Monitoring Cyber Monday – the Good and the Ugly

The holiday season is a time when most people get to relax and enjoy time off of work, but for IT professionals it can be a very hectic time of year. With the rise of e-commerce and online shopping, especially with many retailers putting huge emphasis on their online sales in order to meet profit and revenue targets for the year, the performance of e-commerce sites is crucial to the bottom line of most retailers.

To get a better sense of how things are looking, we’ve been running a study of the performance and availability of more than 130 of the top online retailers, both the pure-plays like Amazon as well as traditional brick and mortar stores such as Walmart and Target, to see how they handled the Cyber Monday rush of traffic, as well as the rest of the online holiday shopping season.

We used Panopta’s monitoring system to track the performance of all sites, performing full HTTP page loads every sixty seconds from our global monitoring network. For companies that had both a brochure-type site and a separate online store we tracked the online store. Alerts for outages were sent to our internal team who further tested to verify the results. All of the data below was extracted directly from our tracking database and graphed for inclusion in this report.

What we found so far has been quite interesting. Many of the big retailers have had no downtime recently, including the Black Friday and Cyber Monday rushes. Others, such as outdoor gear retailer The North Face, had more than six hours of cumulative downtime on Cyber Monday alone. JCPenney and Express also had significant downtimes that affected their visitors and sales. Clearly some of the IT staff behind these sites had very bad days on Monday.

Looking across all of the sites that we monitored, we found some interesting patterns. First, looking at the total downtime we see that there were large spikes on Thursday, Thanksgiving Day and the following day, Black Friday. The weekend was quieter, and then downtime spiked again on Cyber Monday.

Traffic surges to online sites would be expected on Friday and Monday, although the conventional wisdom is that Monday would be the biggest surge as people are shopping from work. The similar results on Friday seem to suggest that bandwidth penetration in the US has reached a point where people are just as likely to shop from home on Friday as from work on Monday.

What’s interesting here is the high amount of downtime on Thanksgiving day, which could be due to another surge in traffic but is also likely attributable to the fact that most of the IT staff had the day off to spend with their family, which means that responses to problems are much slower than normal.

Looking at the average pageload times across all sites, we see some similar behavior – large spikes on Friday and Saturday and another rise on Cyber Monday. Interestingly, Monday was not as bad as Friday and Saturday, suggesting that consumers are spending more time searching and shopping online, perhaps in preparation of buying online or to research purchases that they’ll make later in-person.

As you would expect, sites differed widely in how they handled the start of the shopping season. Of the 135 sites monitored, more than 100 had no downtime while others had significant outages. Looking at those with the worst availability, we see a number of big-name sites that had problems:

Looking at the load times of individual sites we found several interesting things. While the average load times were up more than 30% on Cyber Monday relative to the previous Monday, some sites actually had better load times on Cyber Monday, which suggests that they either added additional resources and were able to handle the rush of traffic with no problems or they streamlined their sites to better handle the increased visitors. Either way, their sites were loading quickly for their visitors.

We also looked at the day-by-day performance of the best and worst performing sites and found interesting patterns in both. Among the fastest sites, there is a clear pattern of consistent behavior across all sites with a spike on Friday and Saturday which then returns to normal on Monday:

The consistency of these sites is impressive, with normal pageload times under half of a second and, while spiking on Thursday and Friday still remaining below one second. These are very quick, responsive sites that studies have shown are most effective for converting visitors to purchasers.

On the other hand, the worst performing sites show less consistency and overall much higher load times:

Daily Pageload Times for Worst Performing SitesFor these sites, the general load times are between one and three seconds, with spikes as high as eight or nine seconds. While we don’t have direct visitor or sales statistics from these sites, it’s reasonable to expect that this performance definitely had an impact on their visitors satisfaction and corresponding purchases.

We’re continuing to monitor these sites throughout the rest of the holiday season and will post updates here as we find additional interesting behavior.

We’re looking forward to additional thoughts and analysis from our readers – please post your ideas in the comments below!

Jason A.