Note: STS is publishing this verified case study after the customer review and approval while respecting the organization's policy of anonymity.
In late 2016, this five hospital ID (HID) Sunquest Lab, Micro and Blood Bank customer was moving from v. 6.4 to v. 8.1 – effectively revving up five releases. At the same time, they added major new blood bank functionality in the form of Emergency Release for all five sites.
Following the Sunquest-provided General Testing Guidelines would have required approximately 440 hours per HID, or 2200 hours of time spent testing. Like all modern healthcare organizations, they also needed to validate the interfaces to their EMR and other systems – another 900 or so hours to order and result every possible procedure for each HID.
As is the case with most LIS departments today, the customer has a small IT team. To make matters worse, none of the usual daily activities of an enterprise can be put on hold while the lab tests, validates and implements upgrades. Says the LIS Coordinator “I don’t know how labs are supposed to be able to add major software testing to their routine workload.”
An LIS validation project can be broken into phases:
As anybody who has ever tested software knows, the phases from 2-4 can be iterative. The testing may discover a software "bug" that needs to be corrected by the vendor; shine light on software configuration that needs to be modified; or demonstrate that a new feature doesn’t work quite like the lab envisioned. In these cases, the repetition adds even more time to the estimated testing timeline.
To help meet an aggressive project deadline, the enterprise utilized STS’s Functional and Transactional Testing solutions. STS worked with the customer to tailor the test scenarios for their particular environment, then quickly moved forward to begin the test executions and initial result review.
The automated testing executed over 1,200 Test Cases with 27,000 testing objectives, and generated 10,000 transactions. According to the LIS Coordinator "The depth of testing is far greater than what my team and I could perform in the same amount of time.
"Because we had such an aggressive upgrade schedule, we provided 11 PCs to run simultaneous testing for the five locations. Each automated PC replaces up to three FTEs because the automated testing runs 24 hours a day. In reality, more than that because sometimes it even tested all weekend! Between the Functional and Transactional Testing, STS replaced nearly 3100 hours of estimated manual testing over the project period of 15 weeks - that's the same as just over 5 FTEs devoted solely to testing. There is no way we could have done that without STS's software."
There were a few issues discovered by the automated testing - for example a maintenance issue related to “unknown” blood type, a CBC resulting error at one of the sites also related to maintenance and a differential display error that was corrected by Sunquest. STS reviews the preliminary results and discusses anomalies with the customer.
Using the STS Functional Testing reduced the testing effort by just over 60%. The actual time spent on the Functional Testing portion of the project was approximately:
In addition to the Functional Testing, the customer used the Transactional Testing solution to validate four or five interfaces at each site. That interface validation, which would have required several months of manual effort with the available resources, was completed in less than three weeks.
As of June 2017, this customer is rolling Electronic Crossmatch out to all five sites and using STS to test the new workflow. To improve the review process, STS has introduced a new online review function which requires approximately 2-4 hours for each scenario review which would result in a nearly 50-75% reduction in the review times noted above. Says the LIS Coordinator, “The best part of this new way of reviewing is that everything needed to review is all on one screen, which makes it very easy to find your place if you get interrupted in the middle of a review session. Also, we can now enter comments directly into the test scenario and STS can read those comments to make any corrections or adjustments to the test cases if needed. This is much faster than entering the information in the ticket on the portal.”
Future projects include implementing Sunquest's Transfusion Manager at all five sites. The customer is planning to use STS's automatic testing for that project.
The LIS Coordinator concludes that "Hiring a team of consultants to manually test would have required a time commitment from us for planning, logistics and result review. Overall, the STS solution was more cost effective, reduced the amount of hands on time required by our staff and provided excellent documentation. With the Transaction Testing solution there was a depth of testing we could not have achieved on our own in the same 15 week period."
"Our team was involved in the final review and had complete say over whether the test results were acceptable, which gave us a solid sense of confidence in the quality of the testing. Plus, with the system generated documentation we feel we are ready for any validation questions an inspector can ask!”