My software is, of course, perfect. (HA! but go with it). The trouble is that the software I write uses other software; I have external dependencies. For example, Textaurant uses Heroku for hosting, Twilio for text messaging, New Relic for monitoring, and Airbrake for error aggregation.

There are a several good reasons to have external dependencies like these, including:
  • Development speed. Using providers for pieces of the solution lets us focus on the core value we provide rather than the common.
  • Better reliability. People who specialize in monitoring, for example (like New Relic), are going to be a lot better at monitoring than someone who doesn't think about this all the time.
But.... there is a big downside.
When your external provider goes away, you're in big trouble. If Heroku goes down, my app is unavailable. If Twilio goes away, I won't be sending any text messages. It doesn't matter that the outage is on the service provider end - to my customers, it's just the application they use not doing what it's supposed to. And that's my problem.
So we have dependencies, which are really useful and also introduce risk. What can we do about it?
Lots.
Let's take a simple example. My hosting provider had an outage on December 3rd that took down our application. What could have prevented us from being unavailable, even as Heroku was having a No Good Very Bad Day? We could:
  1. Add a second hosting provider, writing into a shared database (or db cluster). Properly load balanced, the hosting provider who was not affected could simply have taken over all hosting duties.
  2. Fail over to a secondary hosting provider as soon as we realize we're down, again using a shared database or database cluster.
  3. Use local data storage in the browser to allow users to keep working. It wouldn't provide full functionality, but it would have given us 85%+ of our features, which is a lot better than simply being down.
There are two common themes running through these options: redundancy and cost. We can increase redundancy.... as long as we're willing to pay for it. How far you go toward ensuring redundancy is tempered by how much time and money you're willing to spend. In the end, it's up to you and to your particular needs. Just consider your dependencies... before they go down and make you consider them in a panic!