The Test Suite and Test Cases

The Test Suite

The basic goal of testing is to create a minimum number of test cases to uncover the maximum number of errors for a given amount of effort. The test suite must include test cases to consider the functional, structural and intergration aspects of a class under test. The functional aspect is an external viewpoint that focuses on the behaviour promised by the interface of the class where as the structural perspective is an internal view point and is based on the actual internal program structure. The tester can then determine the coverage for the test aspect in question.

Test Cases

Chow (1978) in his paper titled "Testing software design modeled by finite-state machines", IEEE Transactions on Software Engineering, May 1978, discusses testing software design modelled by finite-state machines (FSMs), which he labels the W-method. He focuses on the testing of control structures that can occur within a design, rather than on implementations. With extended finite state machines (EFSM), the traditional approach for testing FSMs such as the W-method is not adequate.

The extended data aspect represents the data manipulation and has to be tested to check the implementation behaviour. A complete test case both starts and ends at the initial state. Test cases can be produced from EFSM using a combination of data flow testing and control flow testing.

The values of input parameters used have an effect on the test cases. The parameter values may dictate the number of times a loop is repeated, e.g., parameter (argument) values with deposit & withdraw operations for an Account object.. Boundary values, positive & negative testing of parameters are also used as part of test case production.

The number of test cases produced is related to the control structures within a class and the number of substates for each attribute. In small examples such as our ACCOUNT class, we can produce test cases by concentrating on all the individual substates. This will, however, result in a significant increase in the number of test cases for larger systems. So, often, we try to reduce the number of test cases by identifying tests that are more likely to reveal errors, and working out the percentage of errors that may be revealed by the selected tests.

Test case selection usually involves defining the purpose of the test and creating tests to satisfy each test purpose. Test purposes are derived manually such as from use-case descriptions or from checklists of common programming mistakes. Test selection techniques require categorising tests as either structural tests, functional tests or integration tests, and recognising the level of testing supported by this technique.

Functional Test Cases

Functional test cases, also known as black box test cases, are constructed based on the specification of the component. The specification of a class includes a class invariant and specification with pre and postconditions for each method. Coverage is shown as a percentage of the class specification that is covered. A method is executed to verify conformance to its assertions. More test cases are used to check invariants.

Structural Test Cases

Structural test cases, also known as white box test cases, are constructed by identifying individual paths through the code. This allows us to catch errors/bugs that were not detected in functional testing. Coverage is treated as a percentage of the lines of code that have been tested based on the test cases. It is not usually possible to execute all paths of the code, although every line of code should be executed at least once.

Integration Test Cases

Intergration test cases deal with interactions between methods within a class (intra-class testing), and interactions between methods in different classes (inter-class testing). It focuses on checking methods, which may cause errors due to interactions. Test cases are produced by checking values of attributes used or set by these methods, or parameters passed between these methods. During integration tests, care needs to be taken with inheritance, polymorphism, and dynamic binding, which affect the dynamic run-time type of an object in a strongly typed object-oriented language.