April 21st was the date of my annual physical, an event I’d scheduled for first thing in the morning. I’d checked Food Safety News before I went over to sing “Moon River” to my doctor and all was normal.
I returned to the office about an hour letter and checked the company mail. Bill Marler, our publisher and managing partner of our sponsors at the Marler Clark law firm, had emailed all of us about the “Blogs and FSN” with this question: “Can you guys see them on the Internet? Can you get into the backends?”
The answers to both his questions were “no.”
We quickly learned we were no longer to be found on the Internet. After 584 days, Food Safety News, at least for the moment, no longer existed. Bill’s Blog, and Food Poison Journal, and everything else on the Marler Clark publishing platform were also gone.
We next found out that Food Safety News and our sister publications were offline because of a technical snarl at some server farm in Northern Virginia. It’s part of a vast computing network called Amazon’s Elastic Compute Cloud service.
In the first 24 hours, more than 1,000 stories would be written on the problem Amazon had with its “cloud.” There was lots of speculation on whether it would put a damper on the cloud business.
“It is simply amazing how dependent we have all become on the Web — that we do not control,” Marler told the Dow Jones Newswires. Since I think like the boss when it comes to knowing what I do not control, I took the rest of the day off.
I learned from the interviews Bill was giving that our techie people are moving us to servers in Ireland. Bill is also looking at keeping the servers in house.
But in thinking about this prolonged outage — Food Safety News came back on after about nine hours — I’d be surprised if we do not see more of these events in the future. Maybe the next outage will be cause by nothing more than our overburdened electric grid.
Needless to say, the cloud crash was the largest disruption we’ve ever experienced. The only event that comes close to it occurred on Jan 16, 2010 when our story about Russian Prime Minister Putin’s criticism of America’s use of chlorine in chicken processing found its way onto the Drudge Report. In a few minutes, that brought us 30,000 new readers, but we went down because we had more traffic than we could handle. (Moving to the cloud probably solved that.)
So while we do regret the inconvenience the outage caused readers, it was an event that was out of our hands. Like I said, after figuring out what was happening, I took the day off.
Hope you did too!