Acceptance Testing as 
Document Authoring and Annotation


Ward Cunningham
Cunningham & Cunningham, Inc.

Workshop: Agile Acceptance Testing
Conference: XP/Agile Universe
Sunday August 4, 2002, Chicago IL.
 

The EPISODES[1] pattern language describes only briefly three patterns related to acceptance testing in what was then called "competitive development" and is now known at extreme programming.
 
 

Test Suite
Repository:		 Collect tests from Reference Data, Programming Episodes and normal test development.  Distribute tests to Programming Episodes and Developmental Builds.  Preserve and protect as if it were code.  Make flexibility.
Test Suite 
Browser:		 Drill down from build statistics to suites to cases to variables to inspectors and debuggers.  Import, enter, and compute expected values.  Visualize failure distributions (systematic vs. sporadic).
Test Fixture: Construct or retrieve objects suitable for performing a test.  If a test case is an interrogation, then the fixture is the interrogator who calls up the interrogated and starts asking questions.  Long term fixture maintenance is a development responsibility.  Make sure the fixtures appear in a Development Episode's Investigation Context.

 

At the time of writing these patterns had been realized in a testing framework that ran along side WyCash[2] in a single Smalltalk address space. I have since realized the same patterns at least four more times using Wiki, Word or Excel for the repository, HTML for the browser and Java or Python for the fixtures.

I would now describe the act of acceptance test development as one of joint authorship where the customer and developer collaborate in the writing of satisfied tests. An immediate problem is then choosing an environment where both customer and developer can work productivity. Development is responsible for fixturing (software) while the customer is (ultimately) responsible for test data expressed in terms of these fixtures. HTML is an environment where these interests can meet as shown by Murphy and myself[3].

My current practice is to maintain the Test Suite Repository as a directory of documents using a recent version Microsoft Word (with Excel) which can read and write HTML. A second directory contains a library of Test Fixtures expressed using the module facilities of the programming language. A Test Suite Annotator reads both directories, combines suites with their corresponding fixture, and reports successes and failures as an annotated copies of the original HTML documents. A cleverly constructed annotator can operate on all suites in batch or on any individual suite interactively. Batch results are often collected for daily builds providing a long term browseable history of development.
 

[1] EPISODES: A Pattern Language of Competitive Development Part I, Ward Cunningham, PLoP'95. http://c2.com/ppr/episodes.html

[2] The WyCash Portfolio Management System. Experience Report. Ward Cunningham. OOPSLA'92. http://c2.com/doc/oopsla92.html

[3] Position Paper. OOPSLA '99 Extreme Programming Workshop. Emmet Murphy, ThoughtWorks, LLC http://c2.com/w4/rpxp/files/pos-etm.html (restricted readership)
 

Author's Bio

Ward Cunningham is a founder of Cunningham & Cunningham, Inc. He has also served as Director of R&D at Wyatt Software and as Principle Engineer in the Tektronix Computer Research Laboratory before that. Ward is well known for his contributions to the developing practice of object-oriented programming, the variation called Extreme Programming, and the communities hosted by his WikiWikiWeb. He is active with the Hillside Group and has served as program chair of the Pattern Languages of Programs conference which it sponsors. Ward created the CRC design method which helps teams find core objects for their programs. Ward has written for PLoP, JOOP and OOPSLA on Patterns, Objects, CRC and related topics.