Welcome back to our latest update on the internet’s top retailers. If you are new to our blog, we have been publishing our analysis of passing and failing uptime for ecommerce webpages. Last time, we found over the course of the year, the 26% internet’s top 132 retailer’s suffered from less than 99.9% uptime. Only 19 websites had less than 99.9% uptime this holiday season. Overall, this is a positive sign for retailers across the country and their internet infrastructure. This comes on the heels of our earlier report of fewer retailers having website downtime issues over the 2013 year. Of the 132 retailers we were tracking this season on our holiday page, only 14% had substantial downtime. This is far better than the 35% downtime we have seen this year. With a total of 618 outages and 102 hours of downtime across those 134 retailers, this means performance has improved a great deal compared to 1006 outages and 155 hours of downtime in 2012. This upbeat news, we speculate, can be attributed to a few changes to internet retailer’s webpage infrastructure:

  1. For retailer’s who maintain their own webservers and webservices, the cost of getting better web server hardware and software has fallen from highs earlier in the decade making it more feasible to do more with less. The growth of virtual servers has also expanded retailer’s capabilities of scaling their options up as well.
  2. For retailer’s who have put their sites in the hands of cloud providers, there are now more and more cloud providers giving a myriad assortment of possibilities for retailers to shop around with. This also has the affect of making pricing more competitive for brick and mortar retailer’s who want to have a smaller relationship with their web presence. These expanded options make it simple for retailer’s to shop around for upgrades to their online infrastructure.
  3. More passively, the quality of internet resources has improved greatly since the beginning of this decade making it easier for retailers to create, edit, and control their online web presence.

Despite this, the 19 retailers with less than 99.9% uptime were all retailers with persistent problems that put them on under performing holiday season lists for either 2011 or 2012. The list includes: CDW, J&R, Victoria’s Secret, Shutterfly, Office Depot, Pixmania, Gamefly, Cabelas, Sears, Backcountry, Blockbuster, Dell, Harry and David, Lululemon, Guess, Tiger Direct, Sony, Urban Outfitters, and Banana Republic. For these retailers, there performance was below standard but there is a silver lining in that improvements are simple and easy to start as a new years resolution. Good early steps include beefing up current online infrastructure with additional servers or virtual machines, adding network service monitoring, possibly agent side monitoring, and finally, developing a team system for dealing with outages.

We are excited to announce a new release full of improvements to our system. This release includes:

  • Updated Reporting Engine
  • Improvements to Compound Services
  • Template Support for Agent Manifest Files
  • Additional CPU Metric for the Linux Agent
  • Custom Agent Metrics

Reporting Engine

Our outage reports now include server resource data (collected by the Agent) and the reasons which we capture for outages (connection refuse, 404, 500 etc.). We now also compress our outage reports into a zip file to allow for faster downloads.

CPU Resources

Our Linux monitoring agent now has an additional CPU check. You can now monitor CPU usage by percentage used for all of the cores on your server. You can learn more about this plugin here.

Compound Service

We have improved compound services by allowing you to construct them with individual network service and agent resource checks. If you would like to try out our new compound service check out this article.

Manifest Files

We have improved our Agent manifest file installations by adding the ability to specify server templates within the manifest file. Now, when a server is created after installing the agent with the manifest file, any templates specified will be automatically applied. To learn more, see our updated Agent manifest file documentation.

Custom Agent Metrics

You are now able to create custom metrics to report and monitor via our monitoring agent with a simple command line call. This can be seen as a simpler alternative to writing a custom plugin and allows you to integrate metric collection into your own system. To learn more about this new feature you can see this article.

In the past couple months, we have done a lot to improve our service and we are proud to announce our newest release. There are a ton of new features and improvements that come with this release and we hope you are as excited as we are. This release includes:

  • A new version of our monitoring Agent
  • New integrations
  • More robust server templates
  • The release of Panopta OnSight
  • A new support page


Agent 2.0

We have new versions of both the Linux and Windows Agents. We have streamlined installation of the Agent using Debian and Redhat repositories. The new Agent now supports manifest files that allow for automatic configuration of server resource monitoring upon installation. In addition to all of this, you now have the ability to create as many alert thresholds as you would like (including none if you just want data).

We’re continuing to extend the agent and will have a number of new plugins coming soon. If you would like to try out the new Agent see this article for instructions.


Server Templates

By popular demand we have made our server templates more robust and customizable. Server templates now give you complete control over the monitoring locations to use for each check, including the ability to setup multiple checks from different locations. You can now now add as many default templates to a server group as you wish, allowing these templates to stack on top of each other. Along with the new Agent manifest file and provisioning API, this gives you a number of ways to streamline configuration and enable very powerful automated deployment scenarios.

Check out our updated documentation to learn more about the template functionality, and contact our support team if you’d like to discuss custom deployment options for your infrastructure.


Panopta OnSight

The latest version of Panopta OnSight (formally the “Monitoring Appliance”) is finished! With Panopta OnSight, you can securely monitor your servers with network service checks (HTTP, Ping, FTP etc…) and agent monitoring from behind a firewall. The new OnSight is easier to use and install and supports VMWare, XenServer, Hyper-V and VirtualBox environments. If you would like to try it out today check it our documentation here.


New Support Documentation

We have completely overhauled our support documentation. It is better looking, easier to navigate and more detailed than the last incarnation. There is a lot of new content on the site as well, including introductory a getting started guide and a glossary.


Monitoring Network Expansion

We have also added a number of new monitoring nodes around the world. We are proud to expand our infrastructure to India and South America for the first time. All of our new monitoring nodes are listed below. You can see our full monitoring network here.

  • Sydney, Australia 2:
  • Adelaide, Australia:
  • Brisbane, Australia:
  • Beijing, China:
  • Santiago, Chile:
  • Sao Paulo, Brazil:
  • Istanbul, Turkey:
  • Chennai, India:
  • Seoul, South Korea:



In addition to these larger improvements we have a number of smaller upgrades.

  • OpsGenie_darkWe now integrate with OpsGenie, which allows you to merge notifications from Panopta with other monitoring and alerting systems then manage all of your incidents across a range of platforms and devices. If you would like to send your outage alerts to OpsGenie you can learn how here.
  • cancel-alertsWe have also given you the ability to silence further alerts for all your current outages – great for times when you’re firefighting and want to get some silence so you can think clearly.


We are excited to announce that we are expanding our infrastructure. In one week, on May 28, we will add 7 new monitoring nodes around the world. As our infrastructure expands, so too does our the flexibility and accuracy of our monitoring.

We are very proud to add two monitoring nodes in South America, as well as one in India. This is our first time expanding to either area, and we are looking forward to providing reliable server monitoring to the expanding economies in India and South America.

We have also added three monitoring nodes in Australia. Bandwidth leaving Australia has always been very expensive and by expanding our infrastructure their we can ensure affordable monitoring to our clients on the other side of the world.


The new monitoring node locations and IP addresses are listed below:

  • Sydney, Australia 2:
  • Adelaide, Australia:
  • Brisbane, Australia:
  • Beijing, China:
  • Santiago, Chile:
  • Sao Paulo, Brazil:
  • Istanbul, Turkey:
  • Chennai, India:
  • Seoul, South Korea:

Any of our customers that have firewall restrictions for our monitoring nodes should update their system to account for these new IP addresses.

We will continue to add monitoring nodes periodically. If there is anywhere in the world that you would like us to expand to, then please let us know in comments below. After all, we love to improve our service.

With this winter’s historic snow still piling up here in Chicago, our developers
have been stuck indoors delivering lots of new functionality for all of our
customers. We’ve got a wide range of improvements this time, hitting most of
our major systems so there should be something for everyone in this release. It
looks like spring is still several months away (at least) so there will
definitely be more to come soon – keep an eye on our blog soon for details of
what’s coming up.

American politics is always a hectic affair and the rollout of the Healthcare.gov for Americans everywhere has been a bumpy path. In response to this, we would like to release some facts about the response time and availability of the Healthcare.gov website for bloggers and journalists to use as a resource in their own coverage. Using our own Panopta server monitoring system, we set up network checks on the Affordable Care Act’s Healthcare.Gov website finding it was only available for use by the American public 86% of the time during the month of November!

That 86% availability is, by the standards of any online industry, abysmal. Now, it is understood that the roll out of healthcare.gov was “fumbled” but how and where was healthcare.gov fumbled? We checked the healthcare.gov servers, every minute, to check different aspects of the public facing infrastructure including Authoritative DNS, HTTP availability and content checks.


The holidays are now officially in full swing, from the door wreaths on your neighbor’s door and the lights on the trees on main street, this means its time for another update from the Panopta Holiday Index.

If you have been keeping up with the news, you’ll know that if you have been trying to buy health insurance on the new health insurance exchange you get message like this:


We’re excited to announce a large number of enhancements to Panopta as part of our v3.11 release! This release included a fair amount of backend improvements which brings an even more robust and reliable monitoring experience to you. In addition to those backend improvements, here are some of the user facing enhancements which you can benefit from:

Sign in with Twitter

Twitter Login

Now you can sign into Panopta using your Twitter login in addition to our normal username/password authentication. Eliminate the need for another login you have to remember by linking your twitter account to your Panopta login! To set this up, go to Settings | My Account in the control panel and click the Connect Your Twitter Account button. This will walk you through Twitters approval flow, then you can always login with the button on the login page.

While you’re at it, make sure to follow @Panopta to catch all of our latest news and announcements!

Enhancements to maintenance schedules

Many Panopta customers use the maintenance schedule functionality extensively. Maintenance schedules allow you to configure Panopta with periods of time where your systems/servers will be down intentionally and when you don’t want alerts to be sent. In order to make this even easier to use, we’ve added the ability to search through archived maintenance schedules along with the ability to copy an archived schedule for a brand new maintenance schedule.

In the IT industry, infrastructure fails all the time – it’s a known fact that everyone accepts-and there are thousands of tech professionals who won’t tell you anything about it. Usually, the details of failures are kept private, either never mentioned or sanitized into a generic root cause analysis (RCA) that gives only basic information. Very seldom do you get to see what really happened behind the scenes when things go wrong, which is truly unfortunate.

However, these failures, as the old adage goes, are a great teacher. The battle scars and war stories of seasoned system administrators have built their character and establish the skill to quickly assess and resolve problems that come up. Getting to see problems arise and see how they’re dealt with is the best way for more junior staff to learn their trade. Unfortunately the important problems aren’t textbook and this often requires real fires and good teachers in order to gain real knowledge.

One of our customers recently ran into a series of intermittent hardware problems, which led to a number of outages for their SaaS application over a period of 24 hours. They’ve agreed to allow us to describe the problems they ran into and the steps they took to resolve them, along with the lessons they took away from the event and their plans for improving their infrastructure.