Charles Makes a Point (or Two)

Of how CIOs fail.

DISCLAIMER:  This is a little bit of a rant b/c I'm more than a little peeved I STILL can't get my preorder of the iPhone 4 through either Apple or AT&T for delivery.  Standing in line for a full day at an Apple store next week (or whenever they can get stock now) is simply not an option.  I'm not THAT much of a gadget geek. 

I don't meant to be harsh, but this was a colossal failure on the part of both AT&T and Apple's IT planning, change control, capacity planning and incident response teams, IMHO.  Since both companies obviously have separate teams to accomplish these tasks, I have to place responsibility at their CIOs feet.  

If the stories going around the 'Net are accurate, basically neither organization was prepared for the onslaught that fell on them on the 15th.  Both Apple and AT&Ts web sites had minor issues early in the day, but the sites themselves stayed up (at least for me) all day long.  What never worked (again, at least for me, and reportedly for a WHOLE LOT of other people) was the promised ability to order an iPhone 4 for delivery on June 24 to my home.  That was the promise, and so that was the expectation set.  It wouldn't work from the Apple store or the AT&T store.  Why not?

From what I can gather from the reports, the issue wasn't load on the individual sites.  While there were a few issues throughout the day getting to the sites, those issues seemed to be very short lived.  The issue was the integration between the sites and all the people accessing the back end systems.  Since Apple won't allow a US customer to buy an iPhone without it being tied to an AT&T contract, we have to interface back into AT&Ts systems to determine eligibility, which in turn dictates pricing, yadda, yadda, yadda, you put a phone in the cart, agree to extend your indentured servitude to AT&T for 2 more years, bada bing, bada boom, you get a shiny new iPhone 4 on June 24.  At least that's what was promised by Steve Jobs at the WWDC, and on every banner ad AT&T and Apple could find to buy between WWDC and June 15.

The first thing that comes to mind while Apple is busy touting how wonderful they are for selling 600,000 iPhones (apparently NOT through the AT&T web site), I'm left wondering if the CIOs had mandated some true load testing and planning be done between their systems, how many more they would have sold (I know at least two would have been sold for my wife and myself for sure!).   Could it have been the first device to sell a million units on launch day?  Who knows.  We're in the Tootsie Pop commercial, my friends -- the world may never know because IT just makes the servers run for the business, they aren't part of the business.

The next thing that comes to mind is "how in the world does this happen"?  Could these IT departments be ANY MORE disconnected from either of their businesses?  This has been only the single most over-hyped and talked about device in the last decade, from the Gizmodo unveiling to the heavy-handed revenge tactics Apple used, to the exceedingly hyperbolic descriptions of how the phone is going to apparently change everything (I'm pretty sure there'll be an app for cleaning up the Gulf oil spill soon, it's all dependent on that A4 chip, so it's not backwards compatible with the 3Gs or 3G, sorry) to everyone knowing that this is the only device which AT&T is not allowed to cripple with their own branding and application placement (take a look at this Information Week piece to understand what I'm talking about if you don't already).  How do these guys miss this?  I mean, it's not like they've never been through an Apple device launch before.   They don't have any excuses here!  To say that they couldn't have predicted this is naive and untrue.  You spend marketing dollars for a reason, and given the track record of product launches for Apple in recent years, this was completely predictable.  Obviously not a specific number, but it would not have surprised me at all for them to have sold 1 million units -- if the systems had allowed it.  

I've seen some arguments made in defense of AT&T that you can't build out your systems to address theoretical unlimited transactions.  Really?  If you want to turn around the perception that you're a completely inept partner to the "stellar" Apple you do!  It's called capacity planning.  It is not an impossible task to have made some very intelligent predictions on what was going to happen here.  They should have looked at past device launches and seen that 1 million units was not out of the realm of possibility, especially given the (relatively) lackluster response the 3Gs got.  There are a lot of 2G and 3G users out there who have been waiting for this device.  There are plenty of users who have resisted coming over who are now because some of the shortcomings of earlier generations are being addressed.  How do you NOT predict this?  And if you overshoot on your capacity planning, and build out your systems to provide a stellar experience to 1 million preorders but still only do 600,000, guess what?  YOU HAVE 600,000 HAPPY CUSTOMERS!!!!  Instead, you have a muted response from those who were able to get the phone ordered, because even for them it was a pain in the backside, and you have countless others who are questioning whether they want to do business with you.  What a no-brainer.

Of course, then I'm scratching my head over the apparent lack of intelligent change control.  What in the world was the AT&T CIO thinking when it was (reportedly) allowed, the weekend before a massive, exclusive device launch, for the core customer database to undergo significant changes that had not been tested under real load?  I mean, really, didn't anyone besides the WHOLE WORLD know this device launch was coming?  Weren't vacations cancelled at AT&T so all hands could be on deck?  Seriously?  Does anyone believe in change control anymore?   In my IT past, we went into change lockdown before new launches, all end of year events, and just about any event that would drive significant orders until the event was past.  it's not a difficult concept, and it's curious why it isn't followed at AT&T, and why Apple wouldn't demand something like that from its only partner in the iPhone business so neither of them got black eyes.

This is precisely why CIOs shouldn't be providers to the business, they should be PART OF the business.  They should be sitting in these discussions making sure these things are thought of, taking a global view of these various integration points, and speaking up when there are issues.  I am assuming, of course, that they didn't do any of that, and if they didn't, they should probably be fired.  If they did, and were ignored, then they should probably quit, because they clearly aren't seen as a valuable and strategic member of the management team.   

To keep it all in perspective, it is just a stupid gadget.  But this is such a clear case of IT not serving the business well it screams for a detailed case study of what not to do if you want to serve your customers well.