TDD developer Adriaan Brae writes:
I think the separation of tests into ‘generative’ and ‘elaborative’ is a very good concept. I can definitely feel the difference myself.
I like the generative tests much better – they are part of the creative process. I find elaborative
tests much harder to write. This is probably because I lack some of the QA skills you were talking about in your Article about test pairing with developers. It sounds like that would be an excellent learning experience. I think it would be great to spend, say, 1-2 hours a week pair testing.
I am thinking it would be good to keep the pair testing times short. This would allow for cross-training, but also require that each individual go back and implement that training alone. Longer times pairing might end up with each side dependent on the other for ideas.
(posted with permission)
…managing the human dynamics of the tester/programmer interaction is a key issue
Sometimes we forget that in the end, we are all people on development teams with different styles, habits, mannerisms and interact with others differently. Any pairing relationship is unique, and we have to work together towards a common goal. Some of the generalizations I make based on my own experience may not apply to others. I hope others can share their experiences to help broaden our knowledge.
Brian also comments on critical thinking and disruption:
One can be a critical thinker but not a critical speaker. One can post-process the thoughts and use them to guide progress in a non-disruptive way. That’s not traditionally been the role of testers: it’s been their job to have critical thoughts, to evaluate the product against them, to report the results clearly and dispassionately, and to let someone else decide what to do about it. In an agile project, it seems to me that the tester’s responsibility has to go a bit further.
We still have a lot of work to do as testers when working on Agile projects, but the exploration, inquiry and discovery is a lot of fun. It will be interesting to look back on what we are doing now in several years and see how far we’ve come.
James Bach writes:
“It was interesting to me that the developers who came to the Agile Fusion conference initially professed interest in pairing with testers, but their enthusiasm seemed to wane as they learned that testers are critical thinkers– and critical thinkers are disruptive. When I paired with programmers, I mostly adopted a programmer mindset, because I didn’t want to freak them out.
On the context-driven testing mailing list, James elaborates on the disruptive nature of testing:
Try this experiment. Stand behind someone who is writing a memo. Every time they mistype something, tell them. Also tell them immediately about any grammar mistakes. Also comment on their writing style. Periodically stop them and question the whole purpose of the memo. Speak any risks you think of along the way.
I think you will find that the person you are “helping” gets annoyed. Their train of thought will be broken. The same kind of thing happens if we sit with programmers who are writing code. The same kind of thing happens in requirements and design meetings. There is a value to proceeding speculatively and creatively for protected periods of time, without being assailed by critical thoughts.
(posted with permission)
This sentence really got me thinking about tester/developer pairing, and how we can add value to the development process: “There is a value to proceeding speculatively and creatively for protected periods of time, without being assailed by critical thoughts.” I’m not sure that we could or even should pair constantly with a programmer, but there are times when it seems to work very well.
William Wake writes:
At least half the time in TDD, I feel like I’m doing the design part. I talk of generative tests, that yield design, and elaborative tests, that test it “the rest of the way”; the generative tests are pushing the design forward, and it’s somewhat of a distraction to stop and do elaborative tests too soon.
(posted with permission)
This was in response to a note by me describing some of the challenges facing programmers and testers pairing together in TDD activities. I really like the terms: “generative tests” that yield design, and “elaborative tests” that yield more verification or critiquing tests.
As a tester pairing in TDD efforts, I found some of our test development to be a natural fit, while some of it seemed more difficult. When William emailed me with the above comment, I had an “aha!” moment. The natural fit I’d encountered was in test idea generation when we worked on elaborative test development. The generative test development was new to me. When I was the one driving developing new functionality, I found it hard to write tests for classes and methods that hadn’t been developed yet. I clearly need to do more tester developer pairing in TDD activities, and need to work more on TDD when I’m coding. As my pairing partner says, the only key is practice.
Another area of difficulty is the disruptive nature of testing. Some developers might appreciate the disruption as they create code, but others have told me that they wouldn’t want a tester there during what William calls the generative test development. It seems that developers need a protected time of creativity and development to get something together that can be tested. Having a tester interrupting them in this process may not be helpful. Furthermore, when developers spend time making interfaces more testable, the dedicated tester may not be the best person for them to pair with. The tester may not add a lot of value until the interface has been refactored. At that point, they could step in and see if it is testable or not.
Testers who have strong development and testing skills (and who understand and have insight into TDD, bad coding smells, interface discovery, etc.) should fit well into both categories – generative and elaborative. Most testers I might wager, will fit in better in the elaborative test development aspect of TDD. We are used to Business-Facing Product Critiques, and test idea generation activities. Generative test development will probably be new to many dedicated testers. If you want to pair with a developer doing TDD, this might be something to keep in mind.
It may be a good practice to have developer-tester pairing during the elaborative test development activities of TDD, and not during the generative test development. More testers and developers pairing and reporting their experiences may help us learn if we can do this well.
I encourage testers and developers to pair together, and feel free to let me know how you do with TDD pairing. What did you struggle with? What worked well?
A couple of related favorites:
If you are thinking about test automation, you should read James Bach’s “Test Automation Snake Oil” article.
For more info on test automation tool alternatives, check out Bret Pettichord’s “Home Brew Test Automation” slides.
One aspect of Agile Development that appeals to me, and need not only belong in the Agile Development school of thought, is the value that is placed on face-to-face communication. Paper doesn’t glue projects together; the people who are working on the project do. Collaboration works best when done face-to-face, and a written document cannot make the necessary decisions, or fully express the requirements, risks and mitigating factors of a project. I’m not against documentation, but I agree with James Bach, I’m against wasteful documentation. When the documentation distracts us from testing, we should evaluate what we are capturing in documents.
It seems that some successful development teams prefer “discussion with peers” over written documentation. Tim Van Tongeren posts about communication being more highly valued than documentation. Tim writes:
…the study showed that coordination and communication correlate with higher team performance and product quality
I am reminded of Principle 7 of the Context-Driven Testing School:
Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.
We need to be careful and self-critical about what we do on our testing projects. Every action that does not directly contribute to our testing mission should be scrutinized even more than the actions that do.
Communication and collaboration facilitate decision making, and sharpening our skills with like-minded peers not only pushes us as individuals to perform, but benefits the team and the project as well. Writing project documents is difficult to do as a collaborative effort.
One point to think about is this: perhaps more accurate information about the project has already been captured in the program code and in the tests.
Our perceptive equipment lags behind the conceptual skills that we as humans take for granted. As a consequence, linguistic explanations of concepts or feelings are represented by a limited medium. We often can’t provide a thorough linguistic representation of a concept or perceptual judgement to ourselves or to others. As a result, we frequently speak in metaphorical terms to describe feelings and concepts. If it is hard enough to express our own knowledge and thoughts linguistically, it is even harder to express the knowledge of others, especially in written documents.
Brian Marick has an article on Tacit Knowledge that underscores the underdetermination of project documents. The documents are an expression of the ideas and knowledge of experts, but in and of themselves do not provide the complete picture. I particularly liked the paragraph where he describes collaborating to gain tacit knowledge, and looking at a requirements document:
…as a tool for aligning the unexpressed wishes of the experts with the emerging abilities of the product