Dynamic! security is the solution for the following

Oil revenue data gets baked in Alaska

Marc L. Songini

 

March 21, 2007 () The Alaska Department of Revenue blamed human error for last summer's temporary loss of 800,000 paper document images related to the state's Permanent Fund Dividend (PFD), which makes annual payments to all Alaskan residents from funds generated from the lease of oil fields in the state.

The error occurred last June, when a network specialist trying to work out a storage glitch inadvertently lost most of the PFD documents stored by the agency, including the 800,000 images.

In a statement released today, Norm Snyder, the agency's IT manager, said that the agency has since increased the storage capacity on its arrays from 3TB to 7TB, "making it much easier to restore databases and perform regular tests." The agency also created "a formal written backup plan, where active file groups and logs are backed up daily, and full backups, which include all file groups, are performed quarterly," he said.

Snyder said the data was lost while a network specialist was attempting to fix a malfunctioning EMC Corp. storage array.

The fix required cleaning out a section of corrupted data, which apparently caused the loss of some key SQL Server backup data and files, Snyder said. When the error was discovered, the network specialist tried to access the backup data from tape storage, but found that it "had inadvertently not been selected to be written to tape during the normal backup process." Without it, the database couldn't be brought back up online, Snyder noted.

Working through the next weekend, agency IT staffers were able to restore much of the information lost using an old backup file. Snyder estimated that about 800,000 documents scanned during 2006 were not backed up. The lack of backed-up data was caused "strictly [by] human error and the consequences of not placing a check box next to the [primary database] file, instructing the tape backup software to place the file onto tape," said Snyder.

The paper versions of those documents were rescanned over a two-month period by four part-time employees assigned to the task, Snyder said. The rescanning process cost the state about $200,000, he said.

To avoid future errors, the agency has added additional storage capacity and created a formal written back up plan, Snyder said. The agency will also establish an offsite file replication system in Anchorage.

Agency officials declined further comment.