Agile Modeling and eXtreme Programming (XP)

Table of Contents

1. Setting the record straight
2. XP and AM?  
3. AM throughout the XP lifecycle  
4. How do you make this work?  

1. Setting The Record Straight

There are several common misconceptions that people seem to have regarding modeling on an XP project.  The three most common misconceptions are that you don't model on an XP project, that you don't document on an XP project, or that if you do model your only options are the modeling artifacts of the UML.  I'll address these misconceptions in turn in this section, but first I want to explore why they occur so that you can recognize other misconceptions when they arise.  From what I can gather based on the conversations on the AM mailing list the source of these misconceptions is often the result of one or more of the following:

1. Second-hand knowledge of XP.  The Internet is a major source of information for many developers, in particular newsgroups and emails from colleagues.  As people learn a new technique they often join a newsgroup or mailing list, such as the extremeprogramming list, and start monitoring the group/list.  Someone will post something, which may not be accurate, and many people will accept it as official, particularly when they haven't had an opportunity yet to try it out for themselves.  Don't believe everything that you hear.  

2. Questionable sources of information regarding XP.  It's often hard to determine the quality of published material, be it electronic or printed form.  Sometimes honest mistakes are made, that's happened to me more than once, and sometimes people publish misleading information on purpose.  When you're just learning a new subject you often cannot distinguish between high-quality sources of information and questionable ones.  If you base your understanding on questionable sources it is very easy to get the wrong idea about XP. 

3. Difficulty seeing beyond their current environment.  Many developers find themselves in less-than-ideal environments.  XP requires you to adopt practices that are often foreign to your current environment, pair programming and test-first development are new to most organizations, and sometimes these practices simply aren't feasible to adopt.  If you cannot adopt the practice of pair programming then XP isn't going to work for you, but instead of proclaiming that XP doesn't work in their environment many people will instead proclaim that XP doesn't work at all.  The reality is that XP does in fact work in the right situations, it is just that your situation may not be one of the right ones.

4. Too much focus on the word "extreme".  XP's name is both one of its greatest strengths its greatest weaknesses.  Because of the name when some people hear XP's advice to travel light, to reduce the amount of documentation that you create and maintain, that they instead translate it to "create no documentation at all".  That's extreme, right?  Or they'll hear XP's advice to use simple modeling techniques such as user stories and CRC cards and somehow translate that advice to "you don't model at all."  That's extreme, right?  Sigh.

In this section I will set the record straight regarding the three most common issues concerning modeling and XP:

•   Modeling is Part of XP  
•   Documentation happens 
•   XP and the UML?

1.1 Modeling is Part of XP

User stories are a fundamental aspect of XP and artifacts such as Class Responsibility Collaborator (CRC) cards are common to XP efforts.  User stories provide a high-level overview of the requirements for a system -- they are reminders to have a conversation with your project stakeholders regarding their requirements -- and are used to as a primary input into estimating and scheduling, and drive the development of acceptance test cases.   CRC cards are used to explore structure, perhaps for conceptual modeling to understand the problem domain or for design to work through the structure of your software.  User stories and CRC cards are both models, see the Artifacts for AM article, so therefore modeling is clearly a part of XP.  XP developers will also create sketches, often on a whiteboard or a piece of paper, whenever user stories and CRC cards aren't the best option.  In Extreme Programming Explained, the first book written about XP, Kent Beck includes hand-drawn sketches of class diagrams and other free-form diagrams.  In fact, in the second edition he includes a mind map in the inside cover overviewing XP. The bottom line is that modeling is a fundamental aspect of XP, something that I explore in detail in this article.

1.2 Documentation Happens

Documentation is also an important part of XP.  Ron Jeffries offers the following advice:

"Outside your extreme programming project, you will probably need documentation: by all means, write it.  Inside your project, there is so much verbal communication that you may need very little else.  Trust yourselves to know the difference." 

There are several interesting implications of that statement.  First and foremost, the XP community recognizes that documentation should be produced for people external to your team, people that AM would term project stakeholders.  Second, it points out that verbal communication between team members reduces the need for documentation within the team.  This is the result of project team members being co-located, making communication easier, as well as aspects of XP such as Pair Programming and Collective Ownership that promote communication between developers.  As I discuss in the article on Communication documentation is only one form of communication, one that is typically the least effective, that can be easily replaced by more effective techniques such as face-to-face communication.  Third, it recognizes that sometimes you do in fact need internal documentation for your team.  This is consistent with the advice presented in Extreme Programming Installed where the authors point out that information resulting from conversations with your project stakeholders regarding user stories are captured as additional documentation attached to the card.  More on this in the Section A Closer Look At the XP Lifecycle.  Fourth, it suggests that XP team members should know when documentation is required and be allowed to act accordingly.  Fifth, it implies that you should trust the team and give them control over their own destiny.  This can be hard in many organizations.  If the team is untrustworthy then you have a serious problem that needs to be dealt with, this is true regardless of whether they are following XP, or if they are trustworthy but your organizational culture doesn't allow you to act based on that trust then once again you have a serious problem to deal with.  Another problem is that when you are an outsider to an XP team, when you haven't been actively involved in the conversations and interactions that have replaced the need for documentation, that it appears that there isn't enough documentation.  When this is the case, instead of forcing the team to write documentation instead invest the time to determine if they need the documentation that you believe is missing � suggest the documentation to the team, and if there is an actual need for it then they'll create it.  As Ron Jeffries likes to say, "It's called Extreme Programming not stupid programming".  Finally, the most important implication for XP teams is that if you need documentation then write it.

The need for documentation on an XP project is reduced by several of its practices.  First, because of test-first development and a focus on acceptance testing there is always a working test suite that shows that your system works and fulfills the requirements implemented to that point.  For the developers, these tests act as significant documentation because it shows how the code actually works.  When you think about it, this makes a lot of sense.  When you are learning something new do you prefer to read a bunch of documentation or do you look for source code samples?  Many developers prefer to start at source code samples, and the test suite provides these samples.  Second, XP's focus on simplicity and practice of refactoring result in very clean and clear code.  If the code is already easy to understand, why invest a lot of time writing documentation to help you to understand it?  This applies to both internal and external documentation � why add comments to code that is already clear and unambiguous?  If the code isn't so, then refactor it to improve its quality or as a last resort write documentation.  Even though some development environments make it easy to include documentation in your code, Java's Javadoc utility is such an example, you only want to invest in documentation when it makes sense to do so and not just because it is easy.

What confuses many people regarding XP and documentation is that XP doesn't specify potential documents to create during development.  This is unlike the RUP which suggests a slew of potential project artifacts.  Instead, the suggestion is to work together with your project stakeholders in an environment of rapid feedback and trust them to determine the things that they need, not just documents but any type of project enhancements.  Once again, you need to have the courage to trust the people involved with the project.  In the article Agile Documentation I discuss a collection of documents that you may choose to create and provide advice for when to consider creating them.

One of the greatest misunderstandings people have about XP regards concept of traveling light � many people believe that it means you don't create any documentation, but nothing could be further from the truth.  What traveling light actually means is that you create just enough models and documentation, too little or too much puts you at risk.  As I suggest in Agile Documentation a good rule of thumb to ensure that you're traveling light is that you shouldn't create a model or document until you actually need it � creating either thing too early puts you at risk of wasting your time working on something you don't actually need yet. 

An important thing to understand about documentation on an XP project is that it is a business decision, not a technical one.  This is consistent with AM's philosophy regarding documentation, discussed in Agile Documentation.  Jeffries says it best:

"If there is a need for a document, the customer should request the document in the same way that she would request a feature: with a story card.  The team will estimate the cost of the document, and the customer may schedule it in any iteration she wishes."

1.3 XP and The UML

See the article XP and the UML?  Clearly the Wrong Question to be Asking

2. AM and XP?

AM should be tailored into an existing, full lifecycle methodology, in order to improve its approach to modeling.  Because modeling is clearly a part of XP, see above, the potential exists for AM to add value to an XP project.  This assumes of course that there is possible to tailor AM into XP, I believe it is and argue so below, and that you can do so without detracting from what currently exists within XP.  In particular, XP's practices of refactoring and test-first development clearly do a very good job of filling in for two critical goals � promoting clean design and thinking through your design before writing code � that are typically associated with traditional modeling processes.  My experience is that both refactoring and test-first development are complementary to AM and arguably enablers of several AM practices, as I argue below.  In this section I explore the following issues:

•   The potential fit between AM and XP 
•   Refactoring and AM  
•   Test-first development and AM  
•   Which AM practices to adopt? 

2.1 The Potential Fit Between AM and XP

A critical issue that must be addressed is how well AM fits with XP.  Table 1 lists the practices of AM and either maps them to existing principles or practices of XP or discusses the potential fit of the AM practice when it is not explicitly a part of XP.  Because XP was used as a foundation for AM many of the practices map straight to XP.  However, because AM's focus is on modeling several practices are clearly new, hence the potential for AM to bring value to an XP project.

Table 1. Applicability of AM Practices on an XP Project.

The fact that AM's practices are complementary to XP isn't sufficient; there should also be a philosophical alignment between the two methodologies as well.  I believe that there is.  First, AM has adopted the four values of XP � Courage, Simplicity, Communication, and Feedback � and added a fifth one, Humility, one that is clearly compatible with XP.  Second, the principles of AM are closely aligned with those of XP.  Nine of eighteen are adopted directly from XP, and the remaining ones � Software is Your Primary Goal, Enabling the Next Effort is Your Secondary Goal, Model With a Purpose, Multiple Models, Content is More Important Than Representation, Everyone Can Learn From Everyone Else, and maximize stakeholder ROI � are clearly compatible with XP's philosophies.  The three modeling-specific principles may cause a hard-core XP developer to pause for a moment, but on reflection should not prove arguable.  Model With a Purpose advises that you shouldn't work on a model without good cause, Multiple Models says that you have a wide range of techniques available to you that you may choose to apply (including but not limited to CRC cards, user stories, and the diagrams of the UML).

2.2 Refactoring and AM

Refactoring is a technique to restructure code in a disciplined way, a technique that is a fundamental practice of XP.  The basic idea is that you make small changes to your code, called refactorings, to support new requirements and/or to keep your design as simple as possible.  The advantage of refactoring is that it enables programmers to safely and easily evolve their code to fulfill new requirements or to improve its quality.

Is refactoring compatible with AM?  Yes.  Refactoring is a coding technique whereas AM does not address programming-related issues, therefore there is no technical overlap between the two.  What about a conceptual overlap?  AM address design modeling and refactoring addresses design improvement of source code.  This begs the question "What do you do when you have an existing design model and you refactor your code?"  Although it's an interesting question, the real issue is that you have two artifacts, a design model and source code, that describe the design of your system.  One has changed, the source code, now you need to decide whether or not you wish to update the model.  The way that you originally arrived at the model is irrelevant to this issue, you could have gotten there because you took an AM approach to develop it, you could have taken a BDUF approach, or you could adopted an existing model and are coding to it (for example, several organizations have developed persistence frameworks based on the design that I present at http://www.ambysoft.com/persistenceLayer.html).  The issue is irrelevant of the type of design model, be it a UML class diagram, CRC cards, a physical data model, or a procedural structure chart.  The good news is that AM provides advice for how to deal with such a situation, in particular the practice Discard Temporary Models suggests that you should consider whether you really need the design model and then if not get rid of it and the practice Update Only When it Hurts suggests that it's often reasonable to have artifacts such as the design model and the code out of sync.

So how do you apply AM and refactoring together?  Simple.  Apply AM practices as appropriate when you are modeling, use those models as input into your programming efforts, and refactor your code as you normally would have.  If you discover that you need to attempt a major refactoring, get the team together to discuss it, modeling whenever appropriate, then approach the major refactoring as you would have in the past: as a collection of small refactorings. 

Modeling tools that reverse-engineer your code can prove valuable when you are refactoring code, particularly when you are unfamiliar with that code. Many developers think visually, they grasp information communicated via pictures more readily than they do information communicated textually, so CASE tools that quickly import a bit of code and create diagrams from them can be very useful.  It's quite common for CASE tools to import object-oriented source code, perhaps written in Java or C++, and generate UML class diagrams that show the static structure of the code and UML sequence diagrams that depict its dynamic nature.  These diagrams can be used to quickly understand the existing code, the first step in refactoring it.

2.3 Test-First Development and AM

Test-first development is a development practice where you work in very short cycles where you consider a test, write the test and business code for it, get it to work, then continue.  These tests are collected into a development integration testing suite that must be successfully run whenever code is submitted into your shared repository.  This practice is integral to XP.

Is test-first development compatible with AM?  Yes.  Like refactoring, test-first development is more of a coding practice so there is little opportunity for technical overlap.  However, there is room for conceptual overlap because test-first development clearly delves into the realm of detailed design since it provides developers with an opportunity to think through their code before they write it (as well as important feedback regarding their code).  If you've chosen to do a little modeling before writing your code, perhaps to think through an issue larger than a single test case, then that's okay.  In fact, it may even make your test-first development efforts easier, because you've thought things through better.  

How do you apply AM within a test-first development environment?  As with refactoring, simply apply AM practices as appropriate when you are modeling, use those models as input into your programming efforts, and iterate between modeling, testing, and programming as needed.  For more details, read the AMDD article.

2.4 Which AM Practices to Adopt?

Only the ones that add value to what your team is trying to accomplish.  Ideally that will at least be the core practices of AM, therefore it would be fair to claim that you are in fact "doing AM", and perhaps even adopt the supplementary practices as well.  It is important to note that your goal isn't simply to be able to say that you're agile modeling, it is to improve your productivity as software developers.

3. AM Throughout the XP Lifecycle

To explain how the practices of AM can be applied on an XP I will work through a portion of the SWA Online Case Study and show how  modeling is used throughout the XP lifecycle in the article AM and XP: AM Throughout the XP Lifecycle. 

4. How Do You Make This Work?

How should you approach modeling during development on an XP project?  Beck suggests that you should apply the XP practice of Small Initial Investment and draw a few pictures at a time.  He states that the XP strategy is that anyone can design with pictures all they want, but as soon as a question is raised that can be answered with code then the designers must turn to code for the answer.  In other works you should then seek Rapid Feedback to discover whether your pictures are on target by following the AM practice Prove it With Code.

When should you consider modeling during development on an XP project?  Whenever creating a model is more effective that writing code.  In other words, follow the AM principle maximize stakeholder ROI and the AM practice Apply the Right Artifact(s).

How should you model?  Follow AM's practice Use the Simplest Tools and prefer tools such as index cards, whiteboards, and Post It notes over more complicated CASE tools.  Simple tools tend to promote interaction and communication, two factors that are critical to your success.  Although XP favors the use of index cards to record user stories, CRC models, and story tasks there is nothing wrong with using a CASE tool as long as its use provides positive value to your effort.

How should you document?  XP teams prefer to write clean, easy-to-understand source code � their philosophy is that only the source code is in sync with the source code.  However, remember that AM's principle Model With A Purpose states that you should understand the needs of a model/document's audience.  If the audience for documentation is your system's users or your senior management then clean source code isn't going to do it, instead you will need to develop external documentation for this audience.  Your stakeholders should request this documentation and should understand the costs involved, one of which is the fact that any time you spend writing documentation isn't spent writing software, and be willing to accept those costs.