A Test Strategy Retrospective Experience

At my workplace, we hadn’t ever documented what our test strategy was, let alone worked to identify one.  We worked in an environment that had a single tester and because that tester was always just getting by, there wasn’t much room for techniques beyond feature and function testing.  The environment has changed for the better and we now have a team of testers working with development, taking on more testing activities.  Yes, we could do more, but was it the more of the right testing?  To see where we are and where we want to go, I felt it was time to assemble a test strategy.  I decided to approach a single scrum team to start and used ideas from two of Katrina Clokie’s blog posts to get me started.


In Katrina’s experience, she facilitated two types of discussions.  One of them focused on testers on different scrum teams who had drifted from a once unified test strategy.  The other experience focused on a cross-functional agile team engaging in a test strategy retrospective.  I also found out I wasn’t the only one interested in Katrina’s ideas, @seancresswell already gave it a try and I liked his quadrant approach:



I decided to blend all of this and make it work for my context and share that journey with all of you.


Here is what we started with – a blank test strategy based off of Sean and Katrina’s model.


Each role on the team was assigned a colored sticky note, one for Development, one for Testing, and one for Other (which includes the product owner).  Each person on the team was asked to identify how they contribute to testing on this project today and to place the sticky against the two axis.  When that was done, I would ask the team to consider the testing activities that are missing from the strategy and to plot their importance in the strategy.  I also put up a list of common test techniques and approaches, so that the team has some ideas to work with.  This added a little complexity to the session because the list had both approaches and techniques.  One question that arose was with respect to a unit test.  A unit test may cover a function or a feature or a boundary check and there was doubt about how to reflect that on the chart.  Is that a unit test or a boundary test?  I asked the team to keep it simple and if they felt they needed to identify a certain technique within an approach, that it was fine and we would discuss it during the session.


The team then spent about 10 minutes writing down all of their activities on their respective sticky notes and I asked them to plot them on the chart.  When everyone had their ideas up, this is what we saw:
Filled In
Immediately I saw something that I didn’t expect to see – a sea of orange colored stickies (the color assigned to development).  I was surprised to see the high level of contribution to testing by the developers and that was challenging my assumptions in a good way.  I made sure to highlight and acknowledge their contribution to testing on the team, as I felt this was something they don’t get enough credit for.  If they hadn’t plotted their contibutions, I might have never been aware of them.


I also saw a lot of overlap.  I took a moment to collate all of the common sticky notes while the team had a discussion about each others placement of sticky notes.  In cases where multiple roles identified a technique or approach, I did not collate them.  This resulted in the tester having their approach and the developers have their approach, even if there as a common technique being used.  While I did that, an interesting conversation arose about what makes a good unit test and what makes a good integration test.  There was also disucssion about overlapping efforts between the two.  I used the opportunity to remind the team that transparency and awareness about testing is important for all team members because it was clear we all had different ideas about testing.  Should we have the same questions asked in a unit test and integration test?  These were all good questions that came up, but it wasn’t the right forum to complete the discussion, so we carried on with reviewing our test strategy.  Once the stickies were collated, I worked with the team to replace them on the chart.  Here is what that looked like.


We then discussed each quadrant and talked about the activities that we are doing which we want to reinforce and the activities that we aren’t doing enough of or not at all, which we want to incorporate.  We then talked about how the teams could bring these activities into realization, which would occur as part of the typical agile/scrum ceremonies such as sprint planning and the Three Amigos meeting.


To end the meeting, I asked the team to own the test strategy document and make it a living document – meaning that it always reflects what the strategy is.  Since I had seen great contributions from the team, I encouraged them to place the strategy in a visible place so that stakeholders and other teams could see the great things that are being done.  We now have the test strategy document hanging on the wall close to the scrum team who owns it.


Now that I have a successful test strategy session under my belt and it is public, it will be a conversation starter.  I’ve already had a product manager from another team ask when we are doing it for them.


I want to thank Katrina for sharing her ideas in a way that makes it easy for others to bring into their context, and I also want to thank Sean for sharing his own experience.  From all of that I was able to create my own and have it be something I’m quite proud of.


When are you going to have your own test strategy retrospective?  I look forward to hearing about it when you do.