Step 6: Implementing Testing Protocols and User Training
With every new HPXML program launch, there are a number of testing activities that need to happen to maximize the opportunity for success. You should plan for at least a couple of months of testing. Work with the software community to schedule each stage of testing. Try to schedule time for your staff to focus on this activity. If you can set a clear plan for testing early on, and stick to your production schedule, this should allow you to get to market in a timely and organized manner.
There are four main testing roles, each of which requires a different group of testers:
HPXML File Testing
This is best completed between your program software provider and third-party software vendors that are exporting to your system. Ideally, your program software provider will supply third-party software vendors with a testing environment in which they can submit test files and receive direct validation error feedback. Activity will likely be between the IT experts from both your program software provider and the third-party software vendors.
Program Compliance Testing
Based on the validation protocols that you have set up, and in order to check program compliance, a group of testers that are intimately familiar with the program requirements will review files in a test environment. The goal is to see if files submitted by the third-party software vendor are being captured in your program management system and that the system is parsing the data out as expected. Usually, this is completed by someone from the implementation or QA team.
End-to-End Testing
Especially if you are using third-party modeling tools, it is important for testers from the program implementation team to run a series of test homes using each of the software systems as if they were a participating contractor. This is to check to see if data entered into each system is making it all the way to your program database as designed. In addition, you will be familiarizing yourself with the user experience and can identify potential problems before releasing to the contractor base.
User Testing
Once you have completed end-to-end testing, it is always good to identify a group of users (contractors or trade allies) who can try the system in a test environment or as a soft program launch. Using a limited number of contractors, you can identify any potential problems before full-scale release.
It is good to designate a program representative responsible for overseeing the testing process. This person should be equipped to make decisions regarding software requirements and functionality on behalf of the program. As things are discovered during the testing process, it will be important to determine what may need to be fixed before launch verses what can be addressed over time. This role will help ensure that the program requirements are being met, while keeping the project on track.
Home Energy Score Testing
The Department of Energy has developed a Home Energy Score integration testing protocol. For programs participating in Home Energy Score, the DOE will provide a series of sample homes to facilitate software testing. Your program implementation team will need to submit these sample homes as if they were a contractor or auditor in the field, much like the end-to-end testing process identified above. Then using the DOE’s Home Energy Score API, submit the homes to the Home Energy Score system. Following the protocols, your program implementation team will be able to identify validation errors or unexpected outcomes in the data transfer process.
For more information on the Home Energy Scores integration testing protocols, contact the Home Energy Score support team at assessor@sra.com.
User Training and Contractor Role Out
After testing is complete, you can launch your new HPXML based systems. As in most implementation steps up to this point, it is extremely important to engage your contractor and trade ally networks to achieve success. When rolling out a new software environment, especially if multiple software products are being introduced into the market at the same time, programs should coordinate software training along with the software vendors. Each software vendor may have their own training practices, webinars, or other resources they prefer to use.
HPXML Contractor Training for Arizona Public Service (Source: EnergySavvy)
The trainings should include a clear understanding of how to download and upload HPXML files. This includes making transparent to all users the data collection requirements that you have determined as a part of your HPXML implementation. If contractors are unclear on what to collect and how the data validation system works, this can create a lot of frustration if contractor files are rejected by your data validation system. Being very transparent about this in training can significantly reduce the tech support requirement for your program implementation team and software vendors, while also increasing contractor satisfaction.
Note: If you are implementing Home Energy Score for the first time, all participating contractors and auditors must complete a mandatory Home Energy Score training. If you are rolling out HPXML and Home Energy Score together, you will want to provide ample opportunity for contractors to participate in training and complete all requirements before launch.
The Home Energy Score website has more information on training requirements.
