PACS downtime can be dramatically reduced by adopting a workflow continuity approach that ensures access to critical functions such as reading and dictating exams in cases of system unavailability, according to researchers from the University of Pittsburgh Medical Center (UPMC).
In an article published online in the Journal of Digital Imaging, a UPMC team led by Brian Kolowitz shared how the institution used a workflow continuity model to reduce its total PACS downtime by 94% and nearly eliminate its planned downtime over its previous PACS failover approach.
"While workflow continuity does not provide 100% of the functionality available during normal operations, it delivers the most critical functionality, which allows radiologists to continue working uninterrupted," the authors wrote. "The solution is relatively low cost, highly effective, and requires only limited changes in the architecture of existing system."
Unacceptable downtime
After transitioning to 24 x 7-subspecialist coverage of radiology services, UPMC found that PACS downtimes were unacceptable to users and maintenance windows were being lost due to creation of an overnight shift, he said.
"As we engaged our vendors, we realized that the focus of the discussions always revolved around high-availability systems," he told AuntMinnie.com. Traditionally, business continuity and high-availability system design approaches have a technology-intensive focus that can result in extremely complex and expensive systems that can still be subject to limitations, he said.
"For example, some advanced business continuity solutions available today offer a level of redundancy that allows our organization to restore full functionality very quickly when one node of the redundant system fails," he said. "However, these systems still do not meet the uptime needs of the organization because these systems require downtimes during regular maintenance activities and major system upgrades where the entire system is unavailable, impacting clinical operations."
After a thorough analysis of its business continuity strategy, the institution decided to pursue a process that provides continuation of critical workflow instead of attempting to create a redundant, mirrored system (J Digit Imaging, July 6, 2012). Other members of the UPMC group included Harry Black, Charles Barkey, and Ilo Romero.
Solutions to workflow continuity come in a variety of forms; the UPMC PACS uses custom application development to make the transition to the Workflow Continuity Server (WCS) appear seamless to users, but custom code is not required, Kolowitz said. The RIS component is comprised of vendor-provided modules with no internal customization.
The WCS PACS and production PACS have orders and results interfaces with both the production and WCS RIS. In addition, a DICOM interface forwards images from the production PACS to the WCS PACS to synchronize the two systems. The WCS RIS and production RIS make use of a special HL7 interface to migrate orders placed on the WCS RIS to the production RIS.
"The production and WCS PACS remain in sync with regards to orders and results," the authors wrote. "In order for the production RIS and WCS RIS to remain in sync, a series of manual procedures needs to occur when moving between the two systems. One-way transaction log shipping is set up from the production RIS to the WCS RIS. Other key identifiers such as accession number are set by the database administrator."
Key components
Regular maintenance and business process redesign are also key components of the workflow continuity approach, Kolowitz said. In the first phase of adopting its strategy, the institution aggressively pushed to bring all server and operating system patches up to date and then keep them up to date.
"This added a level of stability that dramatically resulted in less system downtimes," he said.
Next, the system patching process was taken to the next level by understanding how the PACS vendor performed the patching process and also determining the most optimal way to sequence the tasks, Kolowitz said. This allowed the team to educate the PACS vendor and reorder the way servers are patched.
"Now the PACS is able to be brought back online after the application, database, and most recent storage modules are available," he said. "We were able to demonstrate the positive impact of our suggestions on end-user workflows that gave the vendor the confidence to replicate the procedures with other customers. Smaller organizations without an extensive IT staff can now benefit from our workflow continuity approach."
Less downtime
In looking closely at their data, the researchers found that the first dramatic reduction in interrupted radiologist workflow in 2009 resulted from attention to the regular system patching procedures. In 2010, further reduction in workflow was achieved after the introduction of the WCS.
From 2008 to 2011, the following improvements were seen in PACS downtime:
- Unplanned: 72.1%
- Planned: 99.7%
- Total: 94.4%
The team realized the benefits of their workflow continuity model during a major PACS version upgrade in 2011, which was implemented without any workflow interruption. If the workflow continuity approach and system design had not been in place, users would have experienced almost 94 hours of interrupted workflow across the enterprise, Kolowitz said.
Kolowitz added that their RIS has gone through a major system upgrade since the JDI article was submitted. Thanks to the RIS WCS, users were able to remain working with only one hour of affected workflow, he said.
"This hour includes a 30-minute failover process and 30 minutes to resynchronize back to production," he said. "Without the WCS, the users would have experienced nine hours of impact."
The UPMC experience has also influenced its PACS and RIS vendors. The PACS vendor is now offering the resulting system configuration and procedures from the PACS workflow continuity approach to other PACS customers, he said.
"Our workflow continuity RIS solution, which is not described in detail in the article, is a zero-code option for us," Kolowitz said. "We were able to develop an approach where we reused existing HL7 migration interfaces to achieve the desired result. The vendor took our suggestion for accession number synchronization between the production and workflow continuity RIS and incorporated it into their core product."
Advice
When asked to give advice for other institutions seeking to implement a similar program, Kolowitz said that because users know their organization and its needs better than anyone else, it's up to them to educate the vendor on their specific needs and to understand the technical capabilities of the vendor's technology.
"Never take no as an answer when technical functionality is negatively impacting your operational needs," he said. "Focus your attention on the workflows needed to support critical operations first, and then use these workflows to drive technical discussions geared toward continuity of these workflows."
Kolowitz also recommends keeping vendor discussions focused on overcoming the technical challenges impeding workflow and resisting the urge to discuss what can't be accomplished due to technical limitations.
"The workflow continuity approach leverages technology to facilitate operational needs without needing to create fully redundant systems," he said. "The change in approach changes the technical requirements that allow for the creation of systems that are not technically equivalent but allow for the execution of the same critical workflows."
