On May 29th our image uploading service crashed and it left customers without the ability to upload images for a few hours.
I take this very serously and so we’ve taken steps to prevent this from happening again.
So far we’ve setup extra monitoring so the entire team gets push notifications when the service goes down again.
We’ve also upgraded our image uploading server to a more powerful type with more CPU power and memory capacity. This will prevent a similar crash again going forward.
Our analysis also showed that we need to do an overhaul of our infrastructure configuration and setup procedure to be more agile; we need to be able to detect issues pro-actively before they happen and respond to issues faster.
We’ll be starting this project right away.
