CMS has introduced so many new acronyms and programs within the last year that many are scratching their heads. Standards Version Advancement Process (SVAP) and Real World Testing (RWT) are two new Health IT Acronyms. As you know from our previous blogs and our whitepaper on Conditions of Certification (COC), RWT §170.405(a) is a requirement for 2021. Today, we are going to cover all the bases as we discuss Real World Testing.
What is Real World Testing?
Health IT developers must demonstrate interoperability and functionality of their certified module in a real world setting and scenario. This testing verifies the certified module functions outside of the ONC-Authorized Testing Laboratory (ONC-ATL) controlled testing environment. As a continued push towards transparency, these plans and results will be publicly reported on the ONC CHPL. So, consider your customer’s use cases and your product’s functionality and target markets as you approach RWT test plan creation.
Who is required to submit test plans?
If you have a 2015 Edition or 2015 Edition Cures Update EHR or module certified by an ONC-Authorized Certification Body (ONC-ACB) for the criteria listed below, you must submit a test plan. Health IT Modules not yet deployed are still required to be considered in a Real World Testing plan. If a Health IT Developer does not have customers or has not deployed their Certified Health IT Module(s) at the time the Real World Testing plan is due, the Health IT Developer should address prospective use cases for the coming year in their RWT plan.
*Image above shows certified criteria applicable to RWT as well as (x) indicates DHIT certified criteria applicable
When is the deadline for completing RWT plans?
The original requirement set in the Final Rule was to have your testing plan submitted to ONC by December 15, 2020, but like many CoC requirement dates, RWT has been extended. If you certified any of the above criteria prior to August 31, 2021, you must provide 2022 Real World Testing Plans for those criteria to your selected ATL/ACB for ONC CHPL posting by December 15, 2021. Note that your ATL/ACB will no doubt require some lead time to validate your test plans prior to posting, so check with your ATL/ACB for their deadline. If you certify your module after August 31, 2021, then you will have to follow with testing plans to be posted on or before December 15, 2022. As of December 15, your plan will be publicly available on the CHPL. Here is a timeline you should keep handy:
Your test plan must provide the methodologies you’ll use during Real World Testing. Keep in mind that the goal of RWT is to test interoperability functions in a real world setting, so follow these factors when determining the methodology:
- Type of organization. (Eligible Physician (EP), Eligible Hospital (EH), medical specialty, etc.) This is very important to determine early on as you will need to select Trading Partners (will touch on this in a moment).
- Size of the organization using your application
- How many patient records and application users
- System components and integrations
- Volume and types of data exchange to execute Real World Test plan
Trading Partners specific to the intended use cases and setting types of your market: While Health IT Developers are not required to test their Certified Health IT in each and every setting in which it is marketed for use, the expectation is that a developer’s Real World Testing plan will address each type of clinical setting in which their Certified Health IT is marketed. Developers should address their choice of care and/or practice settings to test and provide the justified chosen approach. Where feasible, use real patient data and real production environments, not your testing environments. You are also not able to use open source test platforms or testing platforms that are specific to just your product.
Your timeline needs to be included in your plan. Milestones need to be identified in a timeline as well as details on implementation and data collection. If you have plans for a certain measure to be implemented for the calendar year, you need to include the key outcomes that will reflect in Real World Testing. Again, transparency is the goal and this will keep developers on their toes since the testing results will be publicly provided. Keep in mind you will have your plans tested and you will have to resolve any discrepancies as part of the testing process.
Identify measures that address each applicable measure/metric in your scope of certification. You will need to incorporate at least one metric for each criterion in your testing plan. The applicable criteria shown in the table above are geared towards interoperability, so pass/fail or simple yes/no metrics won’t work. Please understand: This is not a plan that can be quickly be thrown together. Your game plan needs to be thoroughly thought out and prepared. There will not be any quick plug-and-play templates from CMS. When selecting your partner for interoperability, confirm their plans to take you through Real World Testing.
Make a grand slam by submitting your plans. Your ATL/ACB will have deadlines separate from the CoC requirements so make sure to contact them for their deadlines. The CoC requirement is to have all plans posted to a URL hosted by the Health IT developer by December 15 of each year.
What happens if your team has an error? Let’s be realistic, there will probably be an obstacle at some point in this process and you will have to adjust your methodologies and report the adjustment in your results report. No need to develop an entirely new plan. You will need to report on how the results deviated from the original plan and submit a statement indicating revisions to your established approach. This results report should include the adjustments made, the timeline of the revisions, and the results of the new approach. You have 30 days to advise your ATL/ACB of non-conformities. Have a discussion with your ATL/ACB about their process for non-conformities from the start.
DHIT has multiple modules certified for 2015 Edition Cures Update and we’ve partnered with over 50 EHRs, ambulatory and inpatient, to take the lead for their certification. Since 2011 Edition, we have performed many Gap Analyses to provide a clear map to certification for vendors. Please contact us for information on our Real World Testing (RWT) services and consulting.
