Understanding Testability and Test Smells

June 14, 2022

Tags: Testability smells , Test smells



If you are familiar with code smells, you probably have heard about test smells. But what about testability smells? It could be a new concept for you. Thoguh testability and test smells seem quite similar due to their names, they are fairly different. In the rest of article, I elaborate on both kinds of smells.

Testability smells

Testability means "how easy it is to write test case" for a given CUT (Class Under Test). Testability smells are the programming practices that reduce the testability of a software system. I will talk about four specific testability smells with a rationale and example.

Hard-wired dependencies

This smell occurs when a concrete class is instantiated and used in a class. A hard-wired dependency creates tight-coupling between concrete classes and reduces the ease of writing tests for the class. Such a hard-wired dependency makes the class difficult to test because the newly instantiated objects are not replaceable with test doubles (such as stubs and mocks). Hence the test will check not only the CUT but also its dependencies (which is undesirable).

In snippet given below, the parse method creates an object of the BindingOperation class (line 4) and calls a few methods (lines 5 and 6). The object cannot be replaced at testing time due to the concrete object creation and its use within this method. Hence, the hard-coded dependency is reducing the ease of writing tests for the class.

private void parse(String name, String namespace, WsdlParser parser) throws WsdlParseException {
  if (WSDL_NS.equals(namespace)) {
    if (OPERATION.equals(name)) {
      BindingOperation operation = new BindingOperation(definitions);
      operation.read(parser);
      operations.put(operation.getQName(), operation);
    }
  }
  //rest of the method
}
Listing 1: Example of hard-coded dependency

Global state

This smell arises when a global variable or a Singleton object is used. Global variables create hidden channels of communication among abstractions in the system even when they do not depend on each other explicitly. Global variables introduce unpredictability and hence make tests difficult to be written by developers. The Builder class in following code snippet is accessible, and hence can be read/written, within the entire project. Such practice makes it difficult to predict the state of the object in tests.

public static class Builder {
  //class definition
}
Listing 2: Example of global state

Excessive dependency

This smell occurs when the class under test has excessive outgoing dependencies. Dependencies make testing harder; a large number of dependencies makes it difficult to write tests for the class under test in isolation. For example, the Error class in the open-source project WSC refers to nine other classes within the project - Bind, BulkConnection, TypeInfo, StatusCode, XmlOutputStream, XmlInputStream, ConnectionException, TypeMapper, and Verbose. Such a high number of dependencies on other classes increases the effort to write tests for this class to be tested in isolation.

Law of Demeter violation

This smell arises when the class under test violates the law of Demeter i.e., the class is interacting with objects that are not specified as class members or parameters to the method. In other words, the class has a chain of method calls such as x.getY().doThat(). Violations of the law of Demeter create additional dependencies that a test has to take care of. For example, lines 4 and 5 of the snippet given below call a method to obtain an object that in turn calls another method on the obtained object. Such method chains introduce indirect dependencies that reduce the ease of writing tests for the class.

public Iterator<Part> getAllHeaders() throws ConnectionException {
    HashSet<Part> headers = new HashSet<Part>();
    for (BindingOperation operation : operations.values()) {
        addHeaders(operation.getInput().getHeaders(), headers);
        addHeaders(operation.getOutput().getHeaders(), headers);
    }
    return headers.iterator();
}
Listing 3: Example of the law of Demeter violation

Test smells

Test smells are violations of best practices in test code. Several [1 , 2, 3, 4, 5] academic attempts have been made to propose and detect test smells, and explore various characteristics. Some of the test smells are defined below.
  • Assertion Roulette: The smell occurs when a test method has multiple non-documented (i.e., without any associated description) assertions.
  • Missing Test: The smell occurs when a test method does not include an assertion. This smell is also known as unknown test.
  • Empty Test: The smell occurs when a test does not have any executable statement.
  • Ignored Test: The smell occurs when a test is ignored (by using @ignore annotation).
  • Constructor Initialization: The smell occurs when a test class use constructor to initialize fields rather than using test setup method.
  • Eager Test: The smell occurs when a test method invokes more than one production methods.
  • Exceptional Handling: The smell occurs when a test method uses try-catch block to assert/fail a check.
  • Conditional Test Logic: The smell occurs when a test method wraps the assert statements within control flow statements.

Difference between testability and test smells

As I said in the beginning, both the kind of smells seem similar due to similar names; however, they are quite different kind of smells. Test smells are violations of best practices in test code where as testability smells are violations of software design principles that make writing tests difficult. The scope of test smells is test code but production code for testability smells.

Detecting testability and test smells

DesigniteJava (version 2.1.0 onwards) supports both testability and test smells. Designite (C#) supports testability smells starting from version 4.0.0; support for test smells in Designite is coming soon.