Petersburg Pilot

Server crash leaves

city dead in the water

Keith Chaplin

July 29, 2010.

Petersburg city departments were “dead in the water,” for nearly three days due to a recent server failure that occurred on Thursday, July 22.

Some city departments were so severely disabled by the malfunction that they shut their doors on Monday and reported to work at noon on Tuesday.

Information Specialist Greg Ball finally rebooted the server Tuesday afternoon following days of work on the system to fix the problem.

The nine-year old server was in the process of being transferred to a new unit when errors started occurring.

Ball, who oversees the servers and computer systems for the city was moving the active directory from the old server to the new server Thursday afternoon.

“Right in the middle of the copying, the hard drive quit working,” Ball said.

Ball said he called Dell — the manufacturer of the server — and was told they did not support the aged computer anymore.

He also spent some time talking with the company that backs up the server, which is located in Virginia. Right before the crash, Ball had recently completed backing up files for the police department.

On Tuesday, he quickly jumped from computer to computer in his office, surrounded by shelves filled with dozens of dust covered hard drives, monitors, and keyboards as the process of rebooting began.

Last month, Ball warned the Petersburg City Council of the danger of running such an old server. The council listened and approved funding for a new server at the cost of $15,000.

Ball was in the process of transferring information to the new server when the meltdown occurred.

Following the reboot Tuesday afternoon, the system was back online, however the city’s email system was still down.

Tuesday afternoon, the city departments were once again bustling as back logged work began.

The server meltdown showed how dependent the city is on technology. A seemingly innocent machine located in a small office at the end of South Second Street is the lifeblood of the cities inner workings.

Checks would not be able to be cut, bills would not be able to be sent and email messages are never viewed as long as it sits broken.

The finance department could not access financial information, Finance Director Jodi Tow said.

“We’re totally dead in the water,” Tow said. “We can’t even cut a check.”

Tow said that because the server was down, her staff spent Friday cleaning and organizing records. On Monday there was very little to do, so employees utilized paid time off or vacation time.

As a result of the issue, utility bills will likely be late, Tow said.

Up the stairs from the finance office, the library was open to visitors. The library uses a separate server to keep track of books being checked in and out. However, the public Internet terminal was closed due to the server loss.

They were still able to issue library cards.

Below the library, pens and pencils were used in the police department to write up reports and calls for service.

“The younger officers are like ‘what’s a typewriter?’” Police Chief Jim Agner joked.

In the harbor office, billing was not completed and email messages sat unread. However there was a back up — the pencil and paper.

“I’ve always been a pro pencil and paper person,” Harbormaster Glorianne Wollen said. “We can keep our heads above the water.”