Tried & Tested

“Right now, software runs the whole world—from infrastructure to your toaster,” says John Fox, vice president of professional services at Swat Solutions, a software quality consulting company in Plymouth. The ubiquitous nature of software makes testing it for glitches, security breaches, and to see if it’s meeting business needs, crucial. But its omnipresence makes it easy to overlook this step, when creating a custom application or just upgrading to the newest e-mail program.

Companies eager to get new software up and running may give software testing short shrift. “Sometimes people tend to take shortcuts,” says Igor Epshteyn, president and CEO of Coherent Solutions, a software testing company in Minneapolis. Companies may forgo software testing if stakeholders don’t realize the value of a systematic process that helps identify defects, and if the software is meeting its stated purpose.

But fixing software problems after deployment—outside the normal testing stage—is costly. Fixing, relaunching, or patching software can take a tremendous amount of time compared to running tests prior to release. And a 2002 study by the National Institute of Standards and Technology says that software bugs (or errors) cost the U.S. economy about $60 billion each year. Thoroughly testing software saves time and money—and can save your company from some embarrassing blunders.

For example, Fox points to a problem accompanying the recent release of Apple’s new iPhone. “[It seems like] they didn’t performance test their iPhone servers,” he says, “so when everyone came to upload the new version [of the phone’s operating system], they got swamped and their servers went down.” One solution could have been to run load tests using virtual servers that could simulate the crush of customers. Fox says a glitch like this can cause a loss of goodwill and even tarnish a brand. “If you have a bug, and all of a sudden you have 10,000 people calling a live operator . . . the support costs are huge,” he says.

Targeted Testing

Before testing even begins, a list of software requirements must be compiled, which testers compare to the software’s actual performance. “That’s a big piece of software quality: getting the requirements done,” Fox says. “A lot of people skip that unfortunately, or do a shoddy job. And I understand why: It’s a pain to document everything and you’re in a rush.” Software requirements can be thought of as what software can and can’t do. These testing requirements—Does the software provide the proper response to invalid data entered? Do users see all the fields they’re supposed to? Does the system work on both PCs and Macs?—are then methodically run through, either manually by a person sitting in front of a computer pressing keys or automatically by a piece of testing-automation software.