May 29, 2024

Pierreloti Chelsea

Latest technological developments

Testing Software Code is Challenging, Even with Agile. Need a Guide?

Decades ago, around the time of the Y2K bug, several software books emerged whose aim was to improve the software development process for engineers and technicians. I remember one popular book by Steve McConnel: Software Project Survival Guide (SPSG).

The SPSG was a hit because it presented the software development process in straightforward and practical terms. For example, test code consists of three elements: unit, integration, and system testing. During system testing, it recommended independent testers, not the same developers that created the programs. Further, tests are needed to cover 100 percent software’s functionality. There were practical suggestions on the ratio of testers to developers:

  • Microsoft: 1 tester / 1 developer
  • NASA: Life critical software, 10 testers / 1 developer
  • Small, in-house business systems: 1 tester / 4 developers

Of course, system test is just one of the many test types needed to produce good software systems. But let’s stay focused on the system as deals with the top-level concerning in testing.

Fast-forward to today. Much has changed in the realm of software development and testing – think Agile systems. But new and even forgotten best practices in testing have also emerged that are especially valuable for the younger software developer. Consider a recent book by two tech experts with decades of experience in the industry (PayPal, LinkedIn, WePay, Cloudera, Twitter): Chris Riccomini and Dmitriy Ryaboy, “The Missing README: A Guide for the New Software Engineer.”

Software Testing Today

In their chapter on testing, the authors understood that “many software projects were missing certain flavors of tests, while others were inconsistent about the separation—intermingling “unit” and “integration” tests. It’s important to know what these categories mean and the trade-offs between them.”

The authors note that successful software developers must make real-world pragmatic testing decisions. These practitioners shouldn’t get hung up on naming and categorization, and should also refrain from passing judgment if the test setup is not quite right.

The testing chapter is full of useful information relevant to today’s modern systems. For example, the authors suggest using injectable clocks rather than static time methods so testers can control the timing that the code sees in a test. This will help avoid the problem of a code segment that waits 500ms for something. A test will pass if the code runs in 499ms but fail when it runs in 501ms. This is a problem.

ASimpleThrottler Ruby class illustrates the problem. SimpleThrottler invokes a throttle method when the operation count exceeds a threshold, but the clock is not injectable.


class SimpleThrottler

      def initialize(max_per_sec=1000)

@max_per_sec = max_per_sec

@last_sec =

@count_this_sec = 0


      def do_work

            @count_this_sec += 1

            # …


      def maybe_throttle

            if == @last_sec and @count_this_sec > @max_per_sec


            @count_this_sec = 0


      @last_sec =


def throttle

            # …




Complementing the code samples and other practical aids is a list of Do’s and Don’t.



DO use tests to reproduce bugs.


DON’T ignore the cost of adding new

testing tools.

DO use mocking tools to help write

unit tests.

DON’T depend on others to write tests

for you.

DO use code quality tools to verify coverage,

formatting, and complexity.

DO seed random number generators

in tests.

DON’T write tests just to boost code


DON’T depend solely on code coverage

as a measure of quality.

DO close network sockets and file

handles in tests.

DON’T use avoidable sleeps and timeouts

in tests.

DO generate unique file paths and

database IDs in tests.

DON’T call remote systems in unit


DO clean up leftover test state

between test executions.

DON’T depend on test execution order.

As do all the chapters, this test chapter ends with a helpful reference to additional readings called Level Up.

All in all, this work is worth adding to the library of the practical software developer and tester, especially for new engineers and technicians. It’s a distillation of technical/process skills (e.g., code review, Agile dev), best practices (deploying new software), and workplace survival skills (dealing with management, working on teams).

The book – The Missing README -will release next month.

John Blyler is a Design News senior editor, covering the electronics and advanced manufacturing spaces. With a BS in Engineering Physics and an MS in Electrical Engineering, he has years of hardware-software-network systems experience as an editor and engineer within the advanced manufacturing, IoT and semiconductor industries. John has co-authored books related to system engineering and electronics for IEEE, Wiley, and Elsevier.