Test First UI experiences

For the last year on my own and for the last six months or so with my development team at the office we have been learning, experimenting, and otherwise playing with or extending our test driven development ideas to UI code, primarily web based UI's. We settled on our own variation of the Model View Presenter pattern that went through many iterations on both pure sample code and eventually on released systems. This effort was not a small one. There is very little consensus on how to test UI code in general, even less on how to do TDD with UI code and far fewer real world examples to draw on for both samples and confidence. In addition, my team wasn't as sold on the idea as I was though they definitely appreciated many of the benefits we were trying to get from the effort. The deciding point had less to do with developers than with the rest of the company who were responsible for manual testing of those things that couldn't be automated and are now more free to concentrate on new development more than on worrying about or executing regression tests.

Now, everybody has done TDD development of UI code here. We are still struggling with choices between dynamic mock frameworks vs hand coded mocks because each represents it's own pain level and that pain doesn't always arrive until you've already committed to one direction or the other. I need to experiment with the NMock2 and Rhino frameworks more to see if the pain is lessoned somewhat, but with NMock we often seem to have paint ourselves into corners that the framework doesn't allow.

But the real reason for this post is that after having gone through the first and second implementations, I would have thought that with the heaviest part of the learning curve behind us, the benefits would feel good enough to make us forget about the awkward parts. The opposite is happening and now I need to reflect on the experience and figure out why. Here is some of the feedback I'm getting on this that I didn't get when we started this process on our domain objects.

• UI code really doesn't get reused that much so designing and testing for reuse is not as easy to justify as it is for domain objects. In that case, the final acceptance tests are much more useful because the same people will request changes and won't mind redoing their tests after the change
• The tests seem to take much longer to write than domain object tests and developing our skills hasn't helped as much as we would have expected to at this point.
  
• The actual code needed to make the UI testable is much more complicated to write, even though it's much simpler to test. There is not a sense that this code is actually better except in cases where complicated interactions in the UI are difficult to test manually. In other words, on simple UI's the result is an anti-benefit, but for some the design really does help implement the code
• Domain objects evolve and become richer and more powerful over time. With UI code, the objects just increase in number
  
• Test suites in the UI seem to make us less agile because you have to drag not only the domain code, but now interfaces, mocks, views and presenters along with you on every change of requirements
  

The last bullet is the salient point here. I have long felt that the most valuable business reason for pursuing TDD and automated testing is general is the impact that trail of tests has on the whole process, simplifying things dramatically for those doing the acceptance testing and shortening the whole process. The extra time spent unit testing has worked as advertised, but with UI based testing, this doesn't seem to hold true. The benefits of regressions are far less if you don't reuse the code. An acceptance test by the project leader can often verify most and in some case more that what the unit test would verify. That statement falls apart in multiple screen wizards or certain types of rich UI scenarios, but that leads to my conclusion from all this.

Why not take take the stance that you should design so your UI *can* be tested, certain key things are tested, and the "hard parts", whatever they are, are tested? The rest is left for acceptance tests, which have to be done anyway. Here are the benefits

• less coding time up front
• Better agile story for changing the UI. There is less time fixing tests that only demonstrate the design, something that's obvious already to the acceptance tester
• When there are complicated UI design issues, application context specific strategies (for company X do Y. For company Z do Y and Z) you still can apply your large arsenal of testing experience and tools to the problem because the design is testable

This point of view seems especially appropriate in the early stages of a design when both the developer and the users are discovering what the application will be and changes are just part of the dialog. I hate the idea of restricting that dialog, but if you insist on building a regression safe barrage of tests for every iteration of your design, it seems to be exactly opposite of the agile goals we're trying to achieve. But even if you have agreed to put off some of the test writing until the design stabilizes, you have also implicitly said you are not going to do TDD. The approach I'm suggesting is still TDD, but statements like "the code is fully tested" is not satisfied by the unit test suite. It would be satisfied by the process however: the project sponsors would not sign off until it's been accepted.

So, let's hear it, we can't be the first people to notice flaws in the TDD story for UI design.