CY 2023 Real World Testing
Plan for Visiontree
This is the real world test plan for CY 2023 for the Visiontree VTOC v5.0 certified modular EHR solution. It provides the real world test measurements and metrics that meet the intent and objectives of ONC's Condition of Certification and Maintenance of Certification requirement for real world testing (§ 170.405 Real world testing) to evaluate compliance with the certification criteria and interoperability of exchanging electronic health information (EHI) within the care and practice setting which it is targeted for use.
As ONC has stated in its rule, "The objective of real world testing is to verify the extent to which certified health IT deployed in operational production settings is demonstrating continued compliance to certification criteria and functioning with the intended use cases as part of the overall maintenance of a health IT's certification." We have worked toward this objective in designing our test plan and its subsequent real world testing measurements and metrics.
This document builds toward the final testing measurements and metrics we will use to evaluate our product interoperability within production settings. Within each measure, we document planned testing methodology, associated ONC criteria, justification for measurement, expected outcomes from the testing, care settings applied for this measure, and if applicable the number of clients to use the our real world testing approach, including how our test cases were created, our selected methodology, the number of client/practice sites to use, and our general approach and justification for decisions.
We have included our timeline and milestones for completing the real world testing in CY 2023 and information about compliance with the Standards Version Advancement Process updates.
This Real World Testing plan is complete with all required elements, including measures that address all certification criteria and care settings. All information in this plan is up to date and fully addresses the health IT developer's Real World Testing requirements.
- Developer Name
- Product Name(s)
- Visiontree Optimal Care (VTOC)
- Version Number(s)
- Additional Software Used
- Mirth Mail by Mirth Corporation
- Certified Health IT Criteria
- Product List (CHPL) ID and Link(s)
- Developer Real World Testing Page URL
Timeline and Milestones for Real World Testing CY 2023
- Begin communication with clients to ask for their support and participation in real world testing. The goal is to have a sufficient number of clients committed for real world testing by the end of 1Q-2023.
- During the 2nd and 3rd quarter of CY 2023, the real world testing with clients will be scheduled and performed. It is expected that a preparatory call will be done with clients to prepare them for testing activities. Results will be documented in the test results section of the test methods and ultimately used to build the test report. If any non-compliances are observed, we will notify the ONC-ACB of the findings and make the necessary changes required.
- During the last quarter of the year, the CY 2023 real world test plan will be completed according to ONC and ONC-ACB requirements and expectations. Test plan will be prepared for submission before the end of the year.
Standards Version Advancement Process (SVAP) Updates
For CY 2023, we are not planning to make any version updates on approved standards through the SVAP process. Standard (and version) All standards versions are those specified in USCDI v1.
Real World Testing Measurements
The measurements for our real world testing plan are described below. Each measurement contains:
- Associated ONC criteria
- Testing Methodology used
- Description of the measurement/metric
- Justification for the measurement/metric
- Expected outcomes in testing for the measurement/metric
- Number of client sites to use in testing (if applicable)
- Care settings which are targeted with the measurement/metric
In each measurement evaluate, we elaborate specifically on our justification for choosing this measure and the expected outcomes. All measurements were chosen to best evaluate compliance with the certification criteria and interoperability of exchanging electronic health information (EHI) within the certified EHR.
Per the ONC requirements, we have include at least one measurement/metric that addresses each applicable certification criterion in the Health IT Module's scope of certification.
For each measurement, a testing methodology is used. For our test plan, we use the following methodologies.
- This methodology uses the logging or reporting capabilities of the EHR to examine functionality performed in the system. A typical example of this is the measure reporting done for the automate measure calculation required in 315(g)(2), but it can also be aspects of the audit log or customized reports from the EHR. This methodology often provides historical measurement reports which can be accessed at different times of the year and evaluate interoperability of EHR functionality, and it can serve as a benchmark for evaluating real world testing over multiple time intervals.
Care and Practice Settings Targeted and Number of Clients Sites
The VTOC patient portal is primarily targeted to the ambulatory and oncology health settings, and our measures were design for this setting in mind. In each measure, we do also address the care settings targeted and note any necessary adjustment or specific factor to consider with this specific measure. Within each measure, we note the minimum number of clients or client sites we plan to use for this measure evaluation.
RWT Measure #1. Number of Patients Accessing Portal
- Associated Criteria
- Testing Methodology
This measure is tracking and counting how many patients are given login access to their patient portal account and able to access their patient data on the portal over the course of a given interval.
The interval for this measure will be 3 months, and we will record the results of unique patient accessing their accounts during this time.
VTOC is an interoperable, standalone patient portal that integrates with other EHRs, and its primarily use is for patients is to view their health information so that they can be an active participant in their healthcare decisions and treatment.
This measure provides several key data points to confirm and verify real world interoperability.
- It shows our standalone patient portal can successfully integrate with a 3rd party EHR.
- It shows that patient data is shared and available to the patient for accessing on the portal.
- It provides confirmation our portal can work across different care settings.
This measure will provide a numeric value to indicate how often this interoperability feature is being used. An increment to this measure indicates that the EHR can supply patient health data to the patient portal and provide an account for the patient to use in accessing this data.
Measurement Expected Outcome
The measurement will produce numeric results over a given interval. We will utilize various reports and audit logs, including Automated Measure (315.g.2) reports, to determine our measure count.
A successful measure increment indicates compliance to the underlying ONC criteria. It will show that the EHR can submit patient health data to the patient portal on a regular and consistent basis as well provide an account for the patient to use in accessing this data.
Successfully completing this measure also implies users have a general understanding of the EHR functional operations for this EHR Module and an overall support for the user experience.
We will use the measure count to establish a historic baseline of expected interoperability use so it can be used in subsequent real world testing efforts.