Skip to main content
 

High Availability

6 min read

This is a story about keeping the lights on...

Bright city lights

There used to be a time, not so long ago, when banks were closed at weekends and on bank holidays, and the only way to do financial transactions was with cash, or otherwise with cheques that used to take 3 working days to clear and could 'bounce'. Today, we can do credit and debit card transactions 24 hours a day, 7 days a week, 365 days a year. Today money flows across the globe in the blink of an eye - pay for some sunglasses in Singapore and your current account will be immediately debited back home here in the UK.

There used to be a time, not so long ago, when getting online meant phoning up another computer. We weren't online all the time - we'd connect once in a while to check our emails, but the rest of the time our telephone line had to be left free so that people could call us. Likewise, computers weren't always available to be connected to - the dial-up number might be engaged because somebody else was connected, or maybe the computer would be switched off or having maintenance done to it. Today, you can access websites 24 x 7 x 365 and you'll never see a message that says the service you're trying to access is offline because of maintenance or some kind of problem. That's what "high availability" means.

So, did we stop turning off the computers, or install some more phone lines or something? Did we get rid of the need to upgrade and do maintenance on the computers? Are the days of engineers having to take a service offline now gone? From a consumer's point of view, that's certainly the way it appears.

In a post 9/11 world, disaster recovery is seen as an essential requirement for business. A terrorist organisation could blow up the headquarters of your bank, but to you as the customer, the computer systems have been designed so that things should function just like normal - business as usual as far as you're concerned. Does that mean that computers are now bombproof? From a consumer's point of view, it certainly seems to be the case.

The reality is that behind the scenes there is a lot of redundancy and failover design so that if anything catastrophic happens, other parts of the system can take over from the parts that have failed. If a computer blows up, another one immediately takes over its work, seamlessly. If a hard disk fails, the data has been copied across a bunch of other ones so no information is ever lost. Software is designed so that it can be upgraded without the users even realising that it's happened - you get new features on the websites you use all the time, but you never notice any interruption in the service. That's high availability in action.

Behind the scenes, there's an army of developers, testers, devops, support analysts, network engineers, sysadmins, database administrators and other flavours of infrastructure engineers, who keep things running smoothly. To keep you plugged into the digital world 24 hours a day, allowing you to send and receive emails, text messages and naughty photos whenever you want, a huge stack of systems have been designed, built and maintained with the principle that they must be "always online". It's a bit like repairing a broken-down car while it's still driving down the road at 100mph.

The net result is that the main skill in IT is not creating the hardware and software anymore, but in keeping the lights on all the time - 100% uptime. Teams of people work in shifts around the clock just waiting for something to go wrong so that they can spring into action and fix it, even though faults are not fatal to the overall functioning of the system, and the users won't even notice that there's been a problem. Computers still fail and hardware still needs replacing. Things need upgrading; things need maintaining, but it all happens without anybody ever seeing a message that says "SERVICE NOT AVAILABLE".

Personally, I do not enjoy sitting waiting for something to go wrong. I'm currently working for a team whose role is to keep the lights on, and it got briefly exciting when the air conditioning failed and a whole datacentre shut itself down, but that was the briefest possible thrill. I'm like a firefighter in this modern world where modern fabrics, improved electrical safety and central heating systems mean that fire is an increasingly rare occurrence in the domestic home. I'm built to fight fires, but everything's built to be so resilient. There are no crises that demand heroics anymore.

I'm pretty much in the wrong job. I deal with machines all day long but I want to deal with people. I'm bored but banking is supposed to be boring - when it gets exciting it means stock market crashes and people not getting paid. I need variety but once you've grasped how to build a computer system, they're all the same - I've built everything from torpedo guidance on nuclear submarines, to bus ticket machines and iPhone apps, and it's all built exactly the same way. I am devastatingly depressed about my job. I think banking is 99% evil, with only 1% of it having anything to do with keeping people's wealth safe from robbers or facilitating transactions that are easier than barter. I need to be solving problems, but I've already solved the same ones a million times, and if I do a good job upfront then there aren't many to solve anyway. It's a dismal existence.

So, I sit at my desk and I get paid an obscene amount of money for doing nothing, just in case something goes wrong... which it very rarely does. I'm highly available, but like a disaster recovery site, hopefully I never have to spring into action, because things are really bad if I'm put to good use. It's really horrible, sitting and waiting for something terrible to happen, and really wanting a crisis to develop because I'm so bored and under-utilised.

I really need to find some kind of app which serves some kind of societal function, beyond stupid distractions from the point of living. Surely the point of living is to spend our brief time on this earth with our family and friends, eating, drinking and making merry, not chasing money and other made-up bullshit.

 

Tags: