Make it Crash

August 21, 2012

Want to find out if an engineer performance tested their application? Tell them to do one thing for you: Make it crash. That’s right, bring the network traffic to a crawl; force it to run out of memory; take away disk space. Have them demonstrate that they know the limits of their application so well, they can make it fall over at will. If they’re bad ass, they’ll whip out a test script and start it up. In a few minutes, the application should crash in a predictable way — predictable if the application is well designed. But really, my point here is to try to make it crash but not be able to.

Under extreme performance conditions, a well designed application will just grind to a halt and stop honoring requests, rather than puke and collapse into an unrecoverable state. This is actually not an easy thing to design, as it requires quite a bit of art to pull off. Within the industry, people don’t talk about this evil little place where the application runs out of system resources as electrons and silicon fight for air in a rapidly drying pool of capacitance. No, we like to ignore this and say stupid things to make ourselves try to forget that the inevitable is just an DDOS attack away.

Bad things will happen. Your poor application will one day leave the nest and and take its place in the cloud. On that day, it will no longer have you to protect it by increasing the heap size of your local app server or closing your email because you’re running out of RAM. It will be in a deployment environment outside of your control for an indefinite period of time, and its only defense against falling over in the face of dwindling resources is what you give it when you’re building it. If it makes you feel any better, most engineers don’t give a sh*t about this, and just let it fall over, but then again most engineers are lazy bastards.


I would like to point out that if we work together today, or have in the past, my opinions may or may not have been influenced by working with you. Most likely they have been, but I have to say that to avoid offending people. You're so vain. I bet you think this site is about you.