code testing as an online service?

All software companies battle for the quality of their code. Software programs are prone to crashes, bugs and security holes and as the software grows it needs increasingly more testing so it’s as reliable as possible.

Even if the tools are better, the number of bugs in newly written code has remained constant at around five per feature

says Capers Jones of Software Productivity Research. “Worse”, says Mr. Jones, “only about 85% of these bugs are eliminated before software is put into use.” *

An industry rule of thumb is that a bug which costs $1 to fix on the programmer’s desktop costs $100 to fix once it is incorporated into a build, and thousands of dollars if it is identified only after the software has been deployed. **

So what are software companies to do? Well, two options that a lot of companies are using is (1) dynamic analysis tools (which examine software as it runs to work out where breakdowns happen) and (2) static-analysis tools (which look at code without actually running it to look for fundamental flaws). Unfortunately both of these solutions are quite costly for the software company with five employees.

Companies I have worked with suggest testing should be outsourced. This way you get the benefit of the larger company requirements but for substantially less. You also get a respected third-party ’seal of approval’ to reassure investors that you are producing quality code. A company that I have worked with in the past in Canada is QA Labs.

But wouldn’t it be cool if someone could offer code testing as an online service? With everything else moving to a hosted model this might be the space to provide smaller companies a way to check their code at an even better price point. I brought this idea up with a friend and he suggested it would be great but how would you get past the intellectual property concern of moving your code online to be tested? Hmmmm, good point. Source code IP is to be protected at all costs – that portion of the agreement needs to be air tight …

A company that is heading in this direction is Veracode but I’ve got to say I haven’t used it nor does their site offer how much it would cost.

If you’ve used Veracode what do you think about this idea?

* The Economist Technology Quarterly, March 8^th, 2008, page 19
** The Economist Technology Quarterly, March 8^th, 2008, page 20