Web Hosting Services
Order Web Hosting
Web Hosting Tutorials
Web Hosting FAQs
Reseller Hosting
Web Hosting Affiliate Program

1996 Reports

13 December 1996

What a day. I apologize to all our customers and their customers for the fiasco that has been on-going since 12:30 PM EST on Friday afternoon. It is now 3 AM on Saturday morning and we are finally back up. In a nutshell, this is what happened...

An electrical problem caused the T1 circuitry to fail. We immediately went to the back-up. Then something failed in a repeater that homes all our datacom lines. So we switched over to the emergency ISDN back-up. Then the impossible happened. The ISDN failed. What occurred was statistically impossible...three simultaneous failures that each took down all three legs of our redundant telco systems.

To BellSouth's credit, they performed far above anything that anyone would have expected. Engineers were all over the place, replacing every piece of equipment between us and the POP. The last engineer left our house shortly after 2 AM on Saturday morning. And the folks at the NOC were amazing. They kept us appraised every hour as to what exactly was happening for the 13 1/2 hours we were down. Everyone at BellSouth deserves every single penny of their salary and they are all invited over on the first warm, spring day for a cookout - even the folks in Atlanta. Nice job folks...and a special thanks to James, Joe, Chris, Ron, Dave, and Garfield (the barking one.)

For those customers who received any email during this outage, it will be delivered within two or three hours. Fortunately, emails do not get lost; they just bounce around for up to three days until they can be delivered.

We are now stable and properly reconfigured. BellSouth will continue testing tomorrow. What will happen is that they will get the ISDN fully operational. We will then go down for about five minutes while we switch TCP/IP over to the ISDN. Then BellSouth will replace physical copper on our T1 lines. When they are properly configured, we will go down for about five minutes again to switch back to T1. At that point, we will be fully up and running. Of course, that's when the power will fail. :)

Speaking of which...I realized that many of our customers do not know the level of redundancy we have built into all our processes (including security measures.) Sometime this up-coming week, I will put together a web page on what we have done - or better yet - what Mikey has created and will post the link here. Come check it out.

Again, sorry if the situation today caused anyone any problems. If it is any consolation, the service we received from BellSouth was so much better than any other provider could have even come close to that there is no comparison.

ps. Just to add insult to injury, two other things happened in Raleigh, North Carolina today that made it memorable in the annals of the computer world. The State of North Carolina government offices (statewide) went down for almost three hours because of a computer malfunction. And to top it all off, the main plant of Cisco Systems, the manufacturer of routers, had a generator fire that knocked out their power and shut them down for a considerable length of time. Again, what a day...


October 21, 1996

Took the secure server down for a few minutes in order to send a secure super-encrypted top secret confidential take-out order to Domino's pizza. Within 500 milliseconds, I had a large pickle and onion pan pizza, with extra cheese, pouring out of the T1. Mmmm! Want some?

Status of Mikey<: Full, but there are leftovers if anyone's interested...


October 15, 1996

The unthinkable happened today. Our web server came to life and cooked us all breakfast this morning! We had eggs and chicken and doritos and... Alright, I'll admit that the food choice was a bit odd, but you've got to give it credit for trying. Anyway, we were all sitting around enjoying the day when Valerie noticed that the server--he asked us to call him George--had unplugged himself from the T1. We ran into the server room and sure enough, the floor was covered with packets that had leaked out. What a mess!

As everyone knows, packets will die if they sit out in the open for too long and sure enough, several of them had expired. We gathered as many of them as we could and put them in the bit bucket so that we could perform CPR (computer packet resuscitation).

As a result of today's activities, you may need to come to PagePlop's corporate headquarters to claim your surviving packets. Funeral services for the deceased packets will be held at the end of the week.

Status of Mikey: I'm very upset by the whole situation. I thought George knew better.


October 4, 1996

Ahhhh! We were down for 4 hours lastnite due to a disgruntled UNIX hacker with a John Deere frontend loader. He took out ten high-speed connections along the East coast before BBNPlanet special agent Bill Dearing took him down with a canister of "Hacker-B-Gone" nerve gas. Cisco router technicians on the scene were further shocked when police, while attempting to remove the frontend loader, inadvertently rolled over a nearby router! The damage was extensive, but easily repairable and we were back up within an hour.

Status of Mikey: When I noticed that we were down, I unplugged our T1 and shook it to see if anything was stuck in there. Then I looked into the T1 and saw this guy on a tractor on the other end tearing everything up. I stared into the wire a little too long, however, because when they fixed everything, a high-speed packet came out a hit me in the eye. I'm okay, but it sure didn't feel good.


September 16, 1996

We were down from 4:00pm to 5:00pm for scheduled maintenance. You know, the standard stuff like new spark plugs and fluid checks. Our next scheduled maintenance is either in three months or 30,000 bits.

Status of Mikey: Smoking!!!


September 10, 1996

Hurricane Fran Update: It's over! I thought we were in real trouble with all that wind and rain, but the T1 never went down and the Macs performed like champs:

Duration of power failure: 110 hours

Number of packets lost: 0

Status of Mikey: Busy, removing wet packets from the T1, drying them off, and carefully re-inserting them (there's nothing worse than getting a soggy packet).


September 7, 1996

Hurricane Fran Update. The eye of the storm is currently about 40 miles to the southwest of us. The wind at the airport was just clocked at 79 mph and the rain is coming down in torrents. The power went out a couple of hours ago, but we're running off the backup generator.

Status of Mikey: Wet. (I'm typing this on my home computer which is currently submerged beneath 6 feet of water. Hey, the scuba gear came in handy :-)


September 5, 1996

The National Weather Service has just placed the coast of North Carolina under a hurricane warning and the entire state under a flash flood warning. Hurricane Fran is currently a category 3 storm with sustained winds of 115 mph. We may be in for a rough ride...

Status of Mikey: (Mikey could not be reached for a comment as he is at the store buying bread, water, and scuba gear.)


August 27, 1996

Today we proved beyond a shadow of a doubt that there is at least one moron who works for Apple (and if I can find out who it is...). This morning, we began experiencing an obscure network problem that manifests itself as email messages that cannot be delivered :-o

Within 30 seconds, our network operations center had me on the phone and we began working on the problem. After 1 hour and 20 minutes, we isolated and corrected the problem, which believe it or not, turned out to be the result of an omission in the operating instructions of our email server.

Status of Mikey: Very unhappy. I'm going back to bed now :-(


August 25, 1996

Received and installed our digital ID from Verisign, giving our secure server the highest level of security currently available. Just in case you're curious, we were issued a "high assurance" Class 3 digital ID which is specifically designed to ensure the security of financial transactions over the Internet.

Status of Mikey: Very happy, and feeling pretty secure about it :-)


August 15, 1996

Opening Day. We all set up lawn chairs in the server room and watched the bits go. There were lots of blinking lights. You should have been here; it was really kewl!

Status of Mikey: Very Happy :-)


August 12, 1996

Test Day. We singled out one test web server and pummeled it with continuous connections for 18 hours straight. We hit the server especially hard during the hours of 12:00 noon to 5:00pm:

Total number of hours: 5

Total number of successful connections: 480,000

Total number of unsuccessful connections: 0

Total number of dropped connections: 0

Total number of very happy Mikeys: 1

Return to the Server Status Menu

 


       Home | Services | Order | Resellers | Affiliates | Server Status
       Tutorials | FAQs | Online Payments | About Us | Contact Us

Copyright © 2024 PagePlop.com Comments.