Thoughts on Automated testing


The Bored Wookie recently changed employers and this has led him to ponder the state of automated testing. In my last 3 jobs I have seen the pressure for automated testers to be pressed into manual test roles in order to get releases 'out the door' on time. Often times this involves working overtime because of miscommunications about what customer expectation actually is.

I have to wonder, is there a better way to drive automated testing?

When automated testing gets beyond the unit test level, I've seen QA groups tasked with automating their test case suites or being asked to write API tests. On the surface this seems like a fine idea- after all, who knows the test cases better than the testers themselves? If you have one or two that are handy with batch scripts or a bit of Java and C# it seems like you should be able to crank out automated tests like crazy... right?

You might be surprised at how much failure I've seen. I don't think I've ever seen a large-scale test case automation effort actually succeed. There is a tendency to view QA driven automation efforts as less worthy of support than something produce by a mainline development team. In my experience, it seems as if people want QA to 'run tests' and 'make sure the latest build is not broken' more than they want to create an environment where product quality can improve.

It seems to me that the solution to automated testing is to put it in the hands of 'dev' as opposed to the 'QA/Test'. Dev teams tend to get resources and have better support from management than QA groups. It seems like QA management is perpetually stuck under 'Engineering' management wich means that they have to make their case to Dev management in order to justify qa tasks. Since it seems like Developers don't take QA roles as seriously as their own, this is a problem.

Now, I'm not saying that Developers don't appreciate testers or that management is entirel unsympathetic to QA needs. What I am noticing, though, is that Devs tend to be taken more seriously than QA people and that there is a stigma attached to testing that is hard to shake. After all, if you're a tester it just means  you couldn't hack it as a dev, am-i-right? (I'm being sarcastic here- hard to convey that over the interwebs).

If QA organizations aren't being given what they need to make automated testing successful, the project should be moved to where it can get resources.