The example test I use is the cost of running one of our weekly tests. This is an automated test that gets run once a week and confirms that with 7 servers we can write a lot of data and track the rate. Let's look at the cost of actually running this thing. I should note that all costs are general and based on Boston averages.
Test Duration: 3 days (72 hours)
Machines Used: 51 (50 servers and one client)
Total Machine Hours: 3672
Test Run Frequency: once a week (52 times a year)
Machine Lifespan: 3 years
A server costs about $3000 and lasts for 3 years. This server spends about 3 days a week running this test. It spends about 3 days a week running other tests. It spends about 1 day a week idle (not continuously, of course!). So, the cost of this particular test is about 40% of the cost of the server. We can say that this test will run once a week for the entire life of the server. So our cost per test run is as follows:
$3000 * 40% = $1200 (cost of the server for running this test)
$1200 / 3 (years) / 52 (times per year) = $7.69
So it costs $7.69 per test run per server to run this test.
Cost of Infrastructure
Servers use a lot of electricity. There's also additional electricity for cooling, etc. Then there's the cost of use of the network, etc. Our infrastructure costs us about $13.33 per server per month for power, cooling, lights, it's share of infrastructure (switches and whatnot), etc.
$13.33 * 40% = $5.33 (cost of the electricity to run this test)
$5.33 / 4 (times per month) = $1.33
So it costs $1.33 per test run per server to power/cool/etc this test.
Cost of Administration
This is an internal system, not a production system. However, if the servers go down, development basically stops. So we put some pretty good administrators on it. These servers take about 2 hours to set up (including checking, racking, installing, adding to the network, etc). Then they take about an hour a month of maintenance. So we've got 36 hours of maintenance plus 2 hours to set up over the life of the server. A Linux administrator gets about $65,000/year salary (or about $32.50 an hour) in Boston.
38 hours *40% * $32.50/hour = $494 (cost over three years to administer this server)
$494 / 3 (years) / 52 (times per year) = $3.17 per test run
So the actual cost per test run is as follows:
Subtotal Per Server Per Test Run: $12.19
7 servers per test run: $85.33 per test run
So our "free or nearly free" automated test is costing us a little over $85 each time we run it. Manual tests require a lot of the same things, so generally this is worth it. Just make sure you check your numbers when so that your "should I automate" calculations include the true costs of running the automated tests.