- unit tests (in several languages)
- cruise control for auto building and tagging, etc.
- a checkin suite of tests (runs after every build)
- a nightly test set that includes about 25 suites (and about 3000 tests)
- a weekly test set that includes about 8 suites with multi-day tests
- a set of performance tests that runs every night
- some scaling and other tests that run occasionally (real machine hogs!)
- a truly evil test that sets up a system and inserts faults until the system fails or we want to start testing newer code so we take it down and start over (this takes weeks, generally)
- a pretty robust framework to manage and run all this
- I'm probably forgetting something...
I can go on about this stuff for hours (and then there's the other testing we do...). So could you, probably. There's another need though: you need your elevator pitch. This is your chance to sell your testing, and to make people want to dive into details with you.
In 30 seconds, what is your test philosophy? What is it your company does to test? Why?
For us it's basically this:
"We're a storage company: reliability and scalability matter most. People want their data, they want it to always be there, and they want it to be consistently available. Our testing model is designed to prevent regressions, stress our reliability mechanisms, sustain performance, and above all ensure consistency of experience while adding new features and platforms. This is a multi-process, highly multi-threaded grid system, so we have an extensive automated test infrastructure that helps us flush out the deadlocks, races, concurrency problems, etc. that a system like this produces. We supplement this with nightly performance testing and automation for specification conformance. Lastly, we layer in manual testing for the human experience and to examine modes we may not have taught the automated infrastructure to see. Quality is everyone's job here."
I'm not explaining everything here. This is a teaser only, but it's important. You have to start with a summary so that people want to learn more. Starting with details loses people's interest; it's just too much information to really synthesize before they decide whether it's interesting. Hook 'em, then wow 'em with the details.