Steven Frank: Betas and Testing

Lots of truth in the article, from both ends (running a beta cycle, being in QA, and also giving some advice to people in a beta testing program. My advice for the latter group would be: test, let people know that you’re testing, even if you haven’t found any problems in the areas you’ve explored. Basically, let the people running the beta cycle that you’ve gone fishing in this part of the application and haven’t caught anything.)

But I have to point this out:

I’ve researched automated testing with products like Eggplant, with mixed results. I’m a big advocate of unit testing, but in practice, it always seems to get back-burnered, and is not easy enough to do with predominantly GUI-driven applications.

You can have all the unit tests in the world, but if the logic connecting your view component to everything else doesn’t work, the program doesn’t work. I know this point all too well. So testing (internal QA and beta testing) are very useful.

The beta testing being a check on internal QA: making sure the internal QA does its job well, and also going above internal QA, trying oddball stuff that internal QA wouldn’t think of. When you work with a product day in and day out it’s easy to test and use the app only the way that makes sense to you - or to the development group - but people may use the app differently in the real world. People will zig when you are expecting them to zag. Which is also why you need beta testers, even if you have automated tests.