When you inherit valuable but difficult code that you need to change, this approach can enable you to add some tests without necessarily really understanding the code.
We work through some exercises adding tests to legacy code using the ‘Approvals’ tool. We will work in small groups and each group can agree which programming language to use. The exercises are generally available in Java, C#, Python and C++.
(short break)
(short break)
(short break)
Using coverage to add regression tests.
Look at these warm-up questions. Think for yourself then discuss with a pair.
Read out some statements. People stand up if they agree they are true. (Or give thumbs up)
Slides - what is legacy code, how to test it. Also: get them to configure their IDEs.
Explain the starting position for Product Export Refactoring Kata. You want to add tests for XMLExporter until you have 100% coverage. The starting code contains an empty test that doesn’t do anything, but very helpfully, there is some test data available in SampleModelObjects. Write a test case for the first function in XMLExporter, I suggest using Approvals and VerifyXML in particular. Show the coverage. It should be much improved for that function, but not 100%. Leave that for them to fix.
Many IDEs have coverage metrics built in and mark lines of code as covered or not covered in the editor. Before this session make sure you know what tools the team has available to them and take the chance to show them how to use them. For example colour-blind people may need to adjust the IDE settings.
Have people work in pairs to do the exercise. Help them to set up their IDEs as necessary.
The last function they try to write tests for has a twist. The output varies with every run since it prints today’s date. Some pairs will be able to work out a way to handle it, others will ask for help. You probably won’t have time to help them fix that in this session, assure them you will tackle it in a later session. Ask them to focus on getting 100% coverage for the other functions for the time being.
Is what we’ve done sufficiently mechanical that you could describe an algorithm for adding tests to existing code?
Write a point by point instruction list about what to do.
Using branch coverage and Combination approvals to improve the tests we add this way
In pairs, discuss weaknesses with the tests we’ve written. Would you feel confident to refactor this code now? Where could you still introduce bugs?
Slides explaining what this is. Also: get them to configure their IDEs.
Show the Gilded Rose exercise. Extract a function. Switch to Combination approvals.
Get them to extract the function and write the tests using coverage.
Using mutation testing to further improve the tests we add this way. Talk about limitations of this approach.
In pairs, discuss weaknesses with the tests we’ve written. Would you feel confident to refactor this code now? Where could you still introduce bugs the tests wouldn’t find?
Slides explaining covered != bug free and uncovered != buggy. Also introduce mutation testing.
Do it by hand in Gilded Rose.
Everyone have a go
Would you feel confident to refactor this code now? Where could you still introduce bugs the tests wouldn’t find?
Review and conclusions
What are the rules for what functions count as ‘pure’? Look at the examples in Pure-Or-Not-Quizz and mark which ones are pure and which ones aren’t. If you get stuck, look at the Wikipedia article describing what a pure function is.
The starting position for ValidateAndAddProduct-Refactoring-Kata has an ordinary Approval test to start you off. The coverage is not great, and you’d like to increase it. Combination Approvals would be a good approach since it’s one big piece of conditional logic with few side effects.
Demonstrate extracting a pure function to use with Combination Approvals. The function should take arguments which are all the things you need to vary in order to cover all the logic branches in the production code. The return type should be a string which contains all the important outputs and can be used to verify against.
Note: the branch ‘with_tests’ includes an example of how to write this function.
Have them complete the Validate and Add Product exercise in pairs. I recommend leaving your code up on the screen while they work on repeating what you just did. Then they should go on and increase the number of combinations until the coverage is 100%.
Get people to look through everything we’ve done today and discuss with someone what they’ve learnt. Write answers on the mind-map as sub-nodes. Review warm-up questions
Mind map questions:
Everyone spends 3 minutes writing notes. What will I
Go round and everyone say something. Yes, and. No questioning other people’s conclusions.