This morning I discovered a small operational problem in our service. Our keyword search service (which uses Lucene) wasn't receiving updates from our spiders. To diagnose it, I figured I needed Ian (who looks after our search), Kevin, who does our spider, and Aaron, who implemented Spread for us -- that's what we use to communicate the updates between our components.
Trouble was, Ian was about to leave for (a very well-deserved) vacation, Kevin was on CalTrain on his way to the office, and Aaron out somewhere.
As coincidence would have it, Ian wanted a document from me so he could do some work on the plane. Nice vacation, Ian. So he IMed me to ask me for it. Since I thought he was long gone, I was elated -- I immediately asked him to take a look at the problem. He had a few minutes before he left, and did some quick diagnosis. He IMed me some details, and then took off.
Fortunately, Kevin can IM on his HipTop. I saw him online, and alerted him to the problem. He asked if I wanted him to hop off the train at the next stop, so he could get online and look at the problem. Yes! I told him. He got off in Palo Alto. His favorite Palo Alto hotspot was down, so he went over to the Apple store. All of this could have been avoided, he told me, if the HipTop had a Control key. Ah, Kevin.
Meanwhile, I called Aaron's cellphone. He was out for breakfast, but said he'd be home in a few minutes. Great, says I.
I collected Ian's notes and emailed them to Aaron and Kevin, then headed into the office. Aaron and Kevin rendezvoused on iChat a few minutes later. After a few minutes of work, they diagnosed the problem and it was fixed by the time I got to work.
What a world we live in.
So, what was the problem?
Posted by: Doug Cutting | December 20, 2004 at 06:59 PM
The post archive MySQL instance had crashed.
Posted by: Adam Hertz | January 02, 2005 at 02:18 PM
Very nice write up. Easy to understand and straight to the point.
Thanks for sharing here..
Posted by: Custom Term papers | December 18, 2009 at 11:58 PM
Coach Outlet OnlineEven if he prevails in the House vote expected on Thursdaysdsfssdfsfs
Posted by: Coach Outlet Online | December 01, 2011 at 12:00 AM