U.K. Hosting Company Accidentally Deletes Customer Web sites
In an e-mail to customers, 123-reg said the data was removed by a clean-up script that it ran to show the company the number of machines active against a master database. An error on the script caused the database to show zero records for some live virtual private services. For customers, that result created a failure scenario that effectively deleted what was on the host server.
Customer Backups Recommended
123-reg’s virtual private servers are unmanaged, and the company said it always recommends that customers back up their Web site data to safeguard against unexpected issues. The company added that customers who had purchased backup services from 123-reg could be back online anytime, as could anyone using another Web site backup service.
Although 123-reg did backups of its customers’ Web site data, with that many accounts it’s impossible to do a complete restore without eliminating any incremental work done by any of the accounts that had not been deleted, Rob Enderle, principal analyst at the Enderle Group, told us.
“Backups may be done on different schedules with full backups happening occasionally and incremental backups happening more often,” said Enderle. “It is clear their restore process is nasty and likely the result of their getting a cheap storage management product over a more expensive comprehensive offering. The folks who sell the more comprehensive and expensive offerings are likely thinking ‘I told you so.'” Enderle added that this is not an usual problem for Web hosting companies that operate on small budgets.
Most Sites Back On
123-reg hosts about 1.7 million Web sites in the U.K, but the company said only a small percentage of them were deleted by the rogue script. The company posted updates about the error on its status page. 123-reg said today that service for the majority of customers affected has been restored, and that it’s working on getting the rest of the deleted Web sites back online.
The customers who were hit by the deletion weren’t shy about telling the world about their frustrations. One Web site owner wrote on Twitter, “Spoke with @123reg earlier today regarding VPS issues. Was told a week minimum for e-mail update and more than likely all data lost.”
123-reg said it is still investigating the restoration of each VPS on a case-by-case basis and is working individually with customers to keep them informed of the Web site recovery process. 123-reg said that data recovery company Kroll was helping with the process of restoring data.
The lesson to be taken from this incident is that consumers should be willing to pay for a superior storage management provider, even if it’s a more expensive option, Enderle said. “There are other criteria other than just price that you need to use to asses a vendor,” he said.
