IT Service Week

Last week I wrote a blog about what the cost of Twitter’s downtime means for a business in terms of revenue. This week I’d like to dedicate a small blog about capacity planning. Many of us in the United States are following the Presidential campaign more closely this year in the past than last because it is the first African-American candidate and the the first female candidate running not only for the democratic nominee but the chance to become President.

In addition, a number of changes have occurred since the last presidential election that make this 2008 election even more interesting. With approximately 215 million Internet users in the United States, a candidate must have a website operational for voters to review information about plans, policies, and contribute to the campaign.

I analyzed the domains of the republican candidate John McCain and the two Democratic candidates Obama and Hillary Clinton on compete.com. It was interesting to see that the growth of the traffic to the domains ranged from 434% - 826% of an increase in traffic from 2007. What do you do when your traffic increases 800% within one year? Most web-tier server deployments take 3-6 months in the enterprise from the time of planning to production stabilization. Thus when a candidate or company prepares to release big new to the press they need to have the proper architecture in place or run the risk of losing potential web visitors.

I analyzed each of the domains from an outside look to determine what each of the candidate’s technology teams have done to prepare for the high volume of unique visitors.

barackobama.com
Obama’s site reached a peak of 3 million unique visitors in February has the most unique visitors of the three candidate sites. According to Netcraft, the site was registered by godaddy.com and hosted in a Linux environment by a provider called Panther Express Corp. Panther has moved the site to at least 10 different IP addresses since it’s original location. This probably means that they were not originally scaled for the traffic and as traffic to the site increased they moved the domain to more powerful systems. The website is hosted with PWS Web Server on Linux. The site’s last reboot was 155 days ago so the uptime of the site has been available for the past 3 months without any downtime.

Full report on NetCraft

Another interesting thing I found about Obama’s site was that there is a sub-domain hosted by a different hosting provider with a different operating system and location. The sub-domain photos.barackobama.com is hosted with the FreeBSD operating system and the Apache 1.3.37 Web Server. They have purchased three IP addresses from Pair Networks to host the photos subdomain. When I go to this URL it forwards me to the flickr site for Obama with his campaign photos and more. So without knowing the internal architecture of the domain I’d imagine that this URL is just a forwarding URL and the main site hosted by Panther contains all of the site data.

Full report on NetCraft

hillaryclinton.com
Clinton’s site peaked at 1.5 million unique visitors in February. The domain was registered back in 1997 with Network Solutions and is hosted on one IP address with a Windows 2003 Server using IIS 6.0. The netblock is owned by Paul Holcomb from Palo Alto, CA. The last reboot was 23 days ago and most likely they are using Windows Clustered Services to maintain Clinton’s traffic on IIS and a load balancer between the IP and the Host.

Full report on NetCraft

Clinton also has a sub-domain hosted by a different company. Clinton’s contribution site, contribute.hillaryclinton.com is hosted by a company called Rackspace in San Antonio, TX. The site uses Apache 2.26 with the SSL module on Linux. Again, like the parent domain, this site is hosted on one visible URL so they could be using a load balancer as well.
Full report on NetCraft

johnmccain.com
McCain’s site has the least traffic of the three and peaked at about 750k visitors in the month of February. The domain was registered back in 1998 at godaddy.com and is hosted on an IP block owned by SMARTech Corp. McCain’s domain has the most interesting domain history in that the site was originally hosted by Blue Gravity Communications in 2005 on FreeBSD with Apache Webserver and then moved to SMARTech to be hosted on a Windows 2003 Server with IIS 6.0. The site was then moved again to another IP in the SMARTech server farm, but on another Windows server. Netcraft did not have an uptime graph for the site.

Full report on NetCraft

Conclusion
That was a lot of raw data. What I’ve found is that all three candidates are relying on hosting providers to manage and maintain their websites. Also, it appears that candidate’s have opted to split portions of their site to be hosted on two providers. Who has the best configuration? Well it would be tough to judge the architecture just by using NetCraft and Unique Visitor counts, but I’d say that hillaryclinton.com has made a good decision by splitting the contribution website to be hosted on a separate provider than the rest of the domain. This allows the hosting provider to manage the secure site separately from the rest of the domain. Most likely all three candidate’s are using a Load Balancing hardware device to divide the unique visitors amongst a cluster of servers.

What should a Candidate To-Be do to prepare their architecture for the traffic?
1.)Don’t attempt to host the site yourself, but rely on providers that have experience hosting and scaling large websites.
2.)Ensure that your Contribution website is secured and using a Web Server that utilizes SSL
3.)Use a Load Balancing Device to move traffic from one clustered node to another. If you have your site hosted by a reputable provider they will do this for you.

What does your availability cost your company? This weekend Twitter has been unavailable and leaving it’s users to look for alternatives. Techie blogger, Robert Scoble, makes a statement in his blog that a number of Twitter users are moving to a competitor site called FriendFeed. FriendFeed integrates its feeds from YouTube, Yelp, Flickr, and Twitter. Scoble goes on to argue that many former Twitter users are converting to FriendFeed due to Twitter’s unreliability. With Colin Devroe reporting a downtime of over 12 hours on his blog.

So what does availability management mean to your company? Have you been able to calculate what your user’s expectations are for availability? If you look at the last month’s 894,000 page views of Twitter (source: compete.com) then you can assume that downtime of more than 1 hour would be very severe to future business.

I calculated the cost of downtime for a hypothetical company making $10 million dollars a year in sales revenue.

$10M Annual Revenue
$1,141.55 Sales Revenue Per Hour (10 M / 8760 hrs in a year)

If you multiply that by the 12 hours the tweets were unavailable, the lost revenue would have been $13698.60. This may seem like a small number to the annual revenue, however, if those numbers continue over the course of a year your customers will no longer be doing business with your website.

Here are three things that your company can do to reduce downtime:

1. Implement architecture that utilizes clustering and load balancing strategies to maintain high availability for your end users.
2. Determine Service Level Agreements and train your Support Staff to escalate high priority issues such as a public website being down.
3. Develop a disaster recovery plan and improve your ability to recover from downtime quickly when it does occur.

These strategies can be implemented when you integrate a best practice strategy such as ITIL within your IT operations department.

Here is a Google Gotcha I found out today on the Natural Search Blog.

Go to Google.
Type in Chuck Norris in the Search Page
Hit I’m Feeling Lucky.

In the April 2008 edition of Harvard Business Review, David J. Collis and Michael G. Rukstad, wrote an 11-page article entitled “Can You Say What Your Strategy Is?” While the article specifically focuses on business strategy and decisions that the CEO makes to give the enterprise an overall strategy, some of the concepts could be applied to IT Strategy for an enterprise’s technology department.

“Most executives cannot articulate the objective, scope, and advantage of their business in a simple statement. If they can’t, neither can anyone else,” states the article and follows as their main argument. The article then compares how the CEO of Edward Jones, a St. Louis based Brokerage, uses a simple strategy that all 37,000 employees can understand and articulate to their customers. By using a simple strategy that is under 35 words in length the company has grown to become the fourth largest brokerage firm in the United States and they are on the Fortune’s list of top companies to work for in 2008.

Some of the problems are:

• Mixed Direction from Executive Management
• Customers are frustrated with company not clearly defining their purpose statement
• Should we not cut the price to maintain our strong brand image at the risk of losing the customer

I found these problems resemble many of the same issues the CIO faces when developing an IT Strategy. Often IT Strategies are loosely based on best practices such as ITIL but are too complex for every IT employee to understand and articulate to their customers, the business.

Often IT Strategies are complied of things such as we desire high availability, low total cost of ownership, or being on the forefront of technological systems. With 3-4 different strategies rolled into one canvas, employees from the help desk to the server rack are sent in multiple avenues of direction and are not as productive as expected.

This strategy begins by building a simple purpose statement. If a strategy has a disorganized purpose statement or something so complex that not all of your employees can understand the mission, then it would not matter if you had an IT budget the size of the US Fiscal budget; your strategy will fail.

Strategy begins in each Department

But building an IT Strategy can also take place at the departmental level. Of course, IT may have an overall strategy to supporting the business, it helps to define exactly how each IT unit aligns it’s services to the business. The similarity would be, “Our company provides consumers with low-cost bicycles and high quality customer service.” as the Company Strategy and then the marketing department would have their own strategy, “Our marketing department provides the business with a positive brand image demonstrating our exceptionally skilled customer service staff and low-cost in the marketplace.” You notice that the marketing department strategy aligns with the business strategy. Then further within the marketing department they have a research team that provides market research on the pricing of bicycles sold in the United States by competitors. This department would have a strategy as such, “To provide accurate reflection and research to the business that maintains our brand image of being a low-cost provider of bicycles.” Again, the statement is kept simple but also aligns itself to the company strategy.

Building an IT Purpose Statement

When it comes time to develop a strategy statement, compile a list of what are the most important purposes that IT serves for the business that you are supporting. After you have compiled the list, cut the list down to the three main purposes and these will make up your reason the business NEEDS technology to accomplish it’s business goals.

For our example we are going to use an e-commerce company that sells bicycles online.

Company Statement: “Provide a low-cost bicycle to budget cautious consumers.”

IT Operations Department: “Monitor, maintain, and supply high availability for the high volume of low-cost bicycle sales by our business.”

Everyone in the department should fully understand the strategy and be able to articulate the strategy to their customers in the business. This is where internal branding comes in and can help get your message out to the business users.

With a clear purpose statement, your department will be on the first step to developing a strong strategy that can
better support the business.