Preparing for the worst: PACS disaster recovery

2002 05 02 13 20 17 706

CLEVELAND - If a structure is damaged by flood, fire, or earthquake, chances are insurance will cover at least part of the cost of rebuilding. But help is hardly automatic for PACS disasters. In the event a PACS network is damaged or destroyed, a comprehensive disaster recovery plan might be the only thing separating an institution from catastrophic loss.

"Disaster recovery plans may seem like a luxury now, but they will soon be mandated by HIPAA. The sooner a facility starts thinking about and creating such a plan, the better," stated Dr. Gary Wendt.

Wendt, a researcher at the department of radiology at the University of Wisconsin (UW) Hospitals and Clinics in Madison, WI, presented the results of a disaster recovery plan his group created and tested. He discussed the results this week at the Symposium for Computer Applications in Radiology.

The UW facilities generate about 300,000 radiological procedures per year from a variety of modalities. When the institution issued a request for proposal for a PACS, one of its requests was for a documented disaster recovery plan. The selected vendor, A.L.I. Technologies of Richmond, BC, Canada, was able to supply such a plan.

The PACS relies on a variety of media, both spinning and static, to maintain the integrity of its image archives. The decision to employ different media types was based on pragmatic concerns, according to Wendt.

"If a virus infects your spinning media, such as a RAID or server farm, it can quickly spread over the network to other spinning media, some of which may be the image archive. But it’s very difficult for that virus to infect an off-line tape storage facility," he said.

The UW PACS implementation utilizes a dual on-line archive on physically independent servers with full network backbone bandwidth to the primary and backup archive locations. This configuration has several advantages over a single archive server with tape backup schema.

First, by housing the primary and backup unit in different locations, the damage created by a physical disaster such as fire or flood would be minimized. Wendt said his location wasn't prone to region-wide disasters such as earthquakes. For institutions in areas prone to such occurrences, he advocated consideration of a remote, ASP-model plan for off-site backup.

Second, by employing two servers performing simultaneous archival backup, failover is instantaneous in the event of a disaster. In a single-server setting, it could take hours, if not days, to reload the tape archive onto a new hardware configuration -- and then only if the new hardware were readily available.

Third, the dual-server setup provides an easy architecture for the facility’s IT personnel to perform needed upgrades and maintenance to hardware and software. If an archive needs servicing, one of the units can be taken off-line with no effect to the users of the PACS. Once the server is back online, its partner server can be attended to in the same manner.

Many plans look good on paper, of course, but the UW team wanted to test its disaster recovery plan in a clinical setting. Wendt and his group first ran through their procedure on an off-line test system, and then performed a full database migration of the PACS system database and server hardware.

"The migration took us 6 hours to perform. We had 4 hours of downtime, and 2 hours of restoration time. In the future, we feel we could shave a couple hours off of the downtime, and could be up and running again in 3 to 4 hours," said Wendt.

The most essential component of any disaster recovery plan, according to Wendt, is thorough training of all personnel, as well as extensive documentation, of each step to be taken in the event of a catastrophic PACS failure.

"Having in-house personnel trained in disaster recovery procedures will help speed recovery time because the in-house person can start recovery procedures before the arrival of vendor service personnel,"

By Jonathan S. Batchelor
AuntMinnie.com staff writer
May 3, 2002

Related Reading

Disaster recovery in radiology, Part III: JCAHO requires emergency preparedness, January 31, 2002

Disaster recovery case study #1: The Dallas VA alternative

Disaster recovery case study #2: UCSF’s model for PACS recovery from off-site storage

Disaster recovery in radiology, Part I: Protecting your images and information, January 17, 2002

A roadmap for implementing HIPAA in radiology, July 26, 2001

Copyright © 2002 AuntMinnie.com

Page 1 of 3588
Next Page