Test maintenance

Work in small steps.

Session Outline

Connect: Does fixing the tests take longer than fixing the code?

Have you had this experience that fixing the code takes 5 minutes then you spend days updating the tests? Why does that happen? I would like you to tell me at least 3 reasons.

Concept - Characteristics of Approval testing

Go through the characteristics of approval testing they should have come accross already. Remind them what an approval test is.

Concrete - update some existing tests

Practice using the diff tool to inspect differences and approve those that aren’t bugs. Use the java, python or C++ version of Supermarket Receipt on the ‘with_broken_tests’ branch. Note down any bugs you find and any features you find. Approve output and fix bugs as you find them.

Concept - Approval testing detects both bugs and features

Additional characteristic we have come accross now:

Explain what ‘over-specified’ tests are and why they could be bad.

Concrete - Update tests making one change at a time

Use the java, python or C++ version of Supermarket Receipt starting on the ‘break_tests_incremental’ branch. Follow the instructions in the README file to apply changes one by one. After each change, note down if you find a feature or a bug. Approve tests and change code as necessary to approve the output or fix each bug.

Note: this repo ApprovalTools has some useful little scripts such as approve_all.py You could usefully put it in your /usr/local/bin.

By the end, your code should look the same as the branch “with_broken_tests”. Note observations on sticky notes.

Conclusions - How can you reduce test maintenance?

Note your ideas on sticky notes. Are any of these strategies only relevant for approval testing? Share with the group.