"Automated tests are cheap to run" is a danger phrase.
It's generally true. Compared to a single manual run of a test, a single automated run of the same test probably takes less time, even accounting for a human looking at the result. It also frees the human up to do new and exciting tests. However, even if one automated test is cheap to run, that doesn't mean it's cheap forever.
The slippery slope thought process goes something like this:
- I found a bug and they fixed it! I'll write a test to prove it.
- I can automate this test. I'll add it.
- I should really write a test to show this bug, and when it's fixed we can check it in to prove it.
- New feature! Let's see what tests we need to feel good about our abilities to refactor it.
- (Fast forward 3 or 4 years)
By now your automated test suite is suffering from Rampant Automation. One test is cheap, but you don't have one. By now you have thousands. And for that reason automated tests are no longer cheap. They probably run for hours on end, take a significant amount of time to maintain and analyze, and use several machines.
All this isn't to say that you shouldn't use automated tests. Please feel free. Just make sure that you keep an eye on the totality of your automated tests and make sure they're not getting away from you. It's up to you to decide how much time and machine time you will put into automated tests. When you get close, it's time to clean up your automated tests, find ways to speed things up, remove tests that aren't helpful, and/or evaluate whether you really do want to automate the new test(s) you're considering.
Automated tests are great. Runaway automation is not. Keep an eye out!