Testing for a release, our test plan breaks down about like this:
- 90-95% regression tests
- 3-8% new twists on existing stuff
- 2-7% new stuff
Of course this varies a bit across releases. Every once in a while we get a huge new feature, and every once in a while we'll have a release with nothing truly new ("it's everything you had before... better!"). But in general we have a system that has been around a while, is fairly featureful, and has an awful lot of areas that should still work just like they did before.
(TANGENT) In general I like existing features to stay for the most part. I tend to be suspicious of products that are always doing "clean sweeps" or huge changes; I think it implies you didn't really think the product through the first time and you're probably not really thinking it through this time, either. When you had something good, keep it. (END TANGENT)
The downside for testing is that things start to get stale. You've started a new release and you have to test file-level enforcement of ACLs... again. To a certain extent you can automate away some of the boring parts, but not everything. So you fall into a rut, and then you start missing stuff.
So you change it up. You reformat your test plan. You move sections around. We've been doing this for about a year now and it's helped us find things. Some of the things have been new bugs, others have been areas that we weren't covering well at all.
The next step is to rethink the technique your test plan implies. One of our discoveries was that our test plans made a smoke test very hard. The testing provided by the automated checkin suite was fine, and that got an initial smoke test. The nightly automated tests provided some level of coverage in a guaranteed amount of time. But the human testing.... that got deep quickly and got coverage more slowly. Basically, the test plan was encouraging us to test the heck out of, say, volume creation, even as we managed to ignore, say, NIS integration until later in the process. We eventually hit it all, but we could do it better.
So now we're reformatting again. This time we're explicitly calling out a "smoke test" of each section. We'll do all the smoke tests early, and then go deep.
Test plans are living documents. They're a record of what you're doing, but they also guide your thinking. So when your results get stale, change the test plan. When your thinking gets stale, change the test plan. It will change your tests... see what you can find now!