Using Coverage and Combinations with Legacy Code

When you inherit valuable but difficult code that you need to change, this approach can enable you to add some tests without necessarily really understanding the code.

We work through some exercises adding tests to legacy code using the ‘Approvals’ tool. We will work in small groups and each group can agree which programming language to use. The exercises are generally available in Java, C#, Python and C++.

Session Outline

(short break)

(short break)

(short break)

Part 1

Using coverage to add regression tests.

Connect: Code coverage warm-up questions

Look at these warm-up questions. Think for yourself then discuss with a pair.

Connect: Approval Testing Characteristics Stand-Up

Read out some statements. People stand up if they agree they are true. (Or give thumbs up)

Concept: Legacy code and coverage

Slides - what is legacy code, how to test it. Also: get them to configure their IDEs.

Demo: Coverage in IDE and add tests

Explain the starting position for Product Export Refactoring Kata. You want to add tests for XMLExporter until you have 100% coverage. The starting code contains an empty test that doesn’t do anything, but very helpfully, there is some test data available in SampleModelObjects. Write a test case for the first function in XMLExporter, I suggest using Approvals and VerifyXML in particular. Show the coverage. It should be much improved for that function, but not 100%. Leave that for them to fix.

Many IDEs have coverage metrics built in and mark lines of code as covered or not covered in the editor. Before this session make sure you know what tools the team has available to them and take the chance to show them how to use them. For example colour-blind people may need to adjust the IDE settings.

Concrete: Add tests using coverage

Have people work in pairs to do the exercise. Help them to set up their IDEs as necessary.

The last function they try to write tests for has a twist. The output varies with every run since it prints today’s date. Some pairs will be able to work out a way to handle it, others will ask for help. You probably won’t have time to help them fix that in this session, assure them you will tackle it in a later session. Ask them to focus on getting 100% coverage for the other functions for the time being.

Conclusions: Describe an algorithm

Is what we’ve done sufficiently mechanical that you could describe an algorithm for adding tests to existing code?

Write a point by point instruction list about what to do.

Part 2

Using branch coverage and Combination approvals to improve the tests we add this way

Connect: Refactoring safety net

In pairs, discuss weaknesses with the tests we’ve written. Would you feel confident to refactor this code now? Where could you still introduce bugs?

Concept: Branch coverage

Slides explaining what this is. Also: get them to configure their IDEs.

Concept: demo Gilded Rose

Show the Gilded Rose exercise. Extract a function. Switch to Combination approvals.

Concrete: add combination approval tests to Gilded Rose

Get them to extract the function and write the tests using coverage.

Conclusions: Pair Share about Combination Approvals

Part 3

Using mutation testing to further improve the tests we add this way. Talk about limitations of this approach.

Connect: Situations where this won’t work

In pairs, discuss weaknesses with the tests we’ve written. Would you feel confident to refactor this code now? Where could you still introduce bugs the tests wouldn’t find?

Concept: Limitations of coverage

Slides explaining covered != bug free and uncovered != buggy. Also introduce mutation testing.

Concrete: Demo mutation testing

Do it by hand in Gilded Rose.

Concrete: Do mutation testing

Everyone have a go

Conclusions: Look again at the situations where this won’t work

Would you feel confident to refactor this code now? Where could you still introduce bugs the tests wouldn’t find?

Part 4

Review and conclusions

Connect: Pure or not?

What are the rules for what functions count as ‘pure’? Look at the examples in Pure-Or-Not-Quizz and mark which ones are pure and which ones aren’t. If you get stuck, look at the Wikipedia article describing what a pure function is.

Concept: Validate and Add Product - extract function

The starting position for ValidateAndAddProduct-Refactoring-Kata has an ordinary Approval test to start you off. The coverage is not great, and you’d like to increase it. Combination Approvals would be a good approach since it’s one big piece of conditional logic with few side effects.

Demonstrate extracting a pure function to use with Combination Approvals. The function should take arguments which are all the things you need to vary in order to cover all the logic branches in the production code. The return type should be a string which contains all the important outputs and can be used to verify against.

Note: the branch ‘with_tests’ includes an example of how to write this function.

Concrete: Use Combination Approvals

Have them complete the Validate and Add Product exercise in pairs. I recommend leaving your code up on the screen while they work on repeating what you just did. Then they should go on and increase the number of combinations until the coverage is 100%.

Conclusions: walkabout posters & code review

Get people to look through everything we’ve done today and discuss with someone what they’ve learnt. Write answers on the mind-map as sub-nodes. Review warm-up questions

Mind map questions:

Retrospective: next steps

Everyone spends 3 minutes writing notes. What will I

Go round and everyone say something. Yes, and. No questioning other people’s conclusions.