Ensure test coverage when refactoring to mocks

Using real dependencies in your tests will lead to pain. A lot of pain.

Most likely, your test fixtures lack coverage, test the wrong class, be difficult to read and don't do what they say.

Eventually, your team will decide to abandon testing or introduce mocking/stubbing.

Refactoring your existing tests will be a major endeavor.

It will be tedious and repetitive.

You will feel like you are wasting time.

You will have a strong urge to rush.

To help you out, I've outlined the steps needed to refactor a test to mocks. I've noted steps that are easy to skip (yet still important) with *

1. Make a copy of the TestFixture -- call it TestFixtureNameWithMocks -- this will ensure you do not lose test coverage and can commit often.

2. Comment out all tests

3. Delete the setup code

4. Take a moment to read the first test

• Does this test actually test anything? (if not, figure out what it should be testing and get it working right in the original fixture first!)*
• Does it verify state of the CUT?
  
• Does it verify state of a dependent (or dependent of dependent of dependent...)?
• Does it do more than one verification?

6. If there is more than one verification, determine how many tests you actually need.

7. If your test relied on the state of a dependent, verify that behavior is tested in the correct test fixture (i.e. the dependent's!). If not, you need one, use this test as an example. Create a failing test now, before you forget. If you have a rabbit hole of dependents, make a test for the top dependent.*

8. If needed, break out the test into multiple verifications -- select one to do, comment the rest.

9. Uncomment test and rewrite using mocks/stub. You will find that some tests are quite different. Some similar. Some will require refactoring of the CUT or dependents to make mock friendly.

10. Run the test and verify it passes. If it does not, figure out what preconditions are missing -- check the CUT and old test. Keep doing this till it passes.

10. Comment out code that is being tested.*

11. Run test, verify it fails for the right reason. If it does not, check the test and verify your assumptions. Make it fail for the right reason!*

12. Uncomment code, verify it passes. If you didn't change code in 11, you can skip this step -- I am paranoid and usually do it anyway :)

13. Repeat for every test in the fixture (and any new tests you find). Pull out common setup if needed. Rename tests since the original names will most likely need it :P

14. Remove old test fixture and rename new one.

If you made it this far imagine how long rewriting one test takes. In the best case you still need to run each test at least 2 times.

Now imagine doing this for more than one class.

How about for how many tests exist by the time our tests are painful enough to require this change.

If done right, this will take a lot of time. If we do it wrong, we take great risks and have tests that give us a false sense of security.

Clearly, if we want to continue delivering end user value, we cannot do this all at once.

Here is one way to do this:
1. Whenever a test is updated (bug fix, enhancement, refactor), update the test fixture to use mocks.
2. Have a policy where it is ok to have 2 fixtures for 1 CUT (one with mocks and one without) to encourage people to start using mocks w/o the overhead/context switch of a big refactor. Stress that finishing the refactor takes precedence over starting new work.
3. Let everyone on the team know that it is expected to take time and include that in commitments and estimations.
4. Remain disciplined

Good luck!!

The need for speed

I say this a lot, but I love TDD. Besides the list of reasons about how it improves code or simplifies solutions, TDD is fast.

TDD is fun because it goes so quickly. Write a test, make it pass, refactor, run more tests.

I love the flow. I love adding a new interface and using it without worry about implementation. I love knowing after I make a new test, something in the app is different and it works. I love living in the tests and only running the application before checking in.

However, the moment the tests slow, the practices start to slip...

1. Writing code and verifying it works by launching the app, then creating the tests
   
2. Making the test pass before verifying it failed.
3. Not running all the tests before check in.
4. Not refactoring because it takes too long
5. It stops you from working close to 5 because you don't want to wait for the build
   
6. Not adding tests at all because you don't want to break the build and don't want to wait to find out
   
7. It ruins all the fun.

These things all lead to poor test coverage or tests that don't actually do what they say. Yeah, on paper your coverage may look nice, but there's plenty of code you can delete w/o breaking tests. Coverage makes sure a code path is followed, it does not verify the path was actually tested.

So, how do we keep our tests running quickly?

1. Mocking. Reduces the amount of time spent creating dependencies and their dependencies and the rabbit hole of dependencies that nobody can even see.
2. Use a one to one ratio of test assemblies to application assemblies -- don't waste time waiting for things to build that you're not testing.
   
3. Keep interfaces and implementations in separate assemblies. Dependents should only be recompiled if the contract changes, not the implementation. This allows you to complete the usage of a change, without worrying about the implementation. Sure, you can just resharper, but are you really going to add tests for a new method right now?? If you do, you break the rhythm, if you don't, you have to remember, somehow, that there is code that needs to be implemented somewhere that your tests won't pick up... you'll start thinking about the implementation or just forget and break the app without anyone knowing.
4. Simplify. Keep tests short. Only create items in setup that are used in all tests. Avoid factories for creating concrete classes because they hide dependencies and make it too easy to use real implementations instead of mocks. Avoid repeating yourself in tests, don't assert the same thing twice, it doesn't help to have the same failure twice but it does slow everything down.
   

In general, follow the best practices of test writing! They're there for a reason. No point in experiencing the pain of the nice people who figured them out. Take advantage w/o having crappy tests, lots of pain and failure.

When Agile Works

When I started consulting on an "agile" team back in February, my first thought was, "Hey! You guys lied! This isn't agile."

The working practices were reminiscent of James Shore's recent blog post.

There was Scrum but they lacked visibility, risk mitigation and self management.
There was a morning stand up, but it was a status meeting with over 20 people -- mostly watchers.
There was a single product team, but members were broken up into domain groups with their own managers, politics and definitions of success.

There were bad practices, communication breakdowns, over-commitment, over-time, half finished features, inconsistency and plenty of technical debt.

Team members had all the stress and no power to change.

But, there was hope.

Within the first few months the company reorganized and moved towards product hierarchy eliminating the contention between groups.

=>Consistent goals and priorities.

The BAs, design and product owner worked closely with an agile coach to create a product backlog. They began having sprint reviews and retrospectives.

=>Visibility and reflection.

A few people became scrum masters. A few others scrum product owners. They started going to agile conferences and local meetings.

=>Engagement.

I left at the end of July to have Kaylee. Things were changing, but we were still far from a well functioning agile team.

When I returned in November, I heard that our agile coach said this was the best agile team he's been on.

I thought to myself, "I think he's been drinking some kool-aid..."

However, when I came in for sprint review and planning, I was amazed.

So much changed.

Team members were involved.

Practice improvements and innovations were made throughout the day.

Charts and metrics were used, not shown.

People were having the right conversations.

The differences not only showed in the team and interactions, but the product.

They had visible success.

I feel very fortunate to have the unique look into the evolution of this agile team.

When you are on the team, it can seem like things don't change. Our retrospectives are a microscopic view compared to a projects lifespan. If I was working for the past 3 months, I may have not noticed.

Maybe a periodic review of how things were/how things are would be a good motivator and illustration of how cool agile can be when its working.

3 Months Later...

Almost.

Babies are challenging... Giving birth was easier... Forget about the note I was sleeping better after Kaylee was born, that didn't last long!

Ah, but the human condition. We adapt. Disrupted sleep included.

Now that I've adapted, I'm consulting again.

Apparently, I really, really like to code AND its much, much easier than being a mommy! Not quite the same ROI...

Since I do most of my work from home, I get the best of both. I'm quite lucky.

I'm with the same company and working with lots of WPF and TDD. We just started using the Isolator AAA from typemock, which looks nice. I know Roy was very involved with its creation and I usually agree with all his testing methodologies :)

My heart is still with rhinomocks, but hopefully I'll find a place for typemock too!

And being a mom, I have to include some newer pics of my lovely daughter!

TechEd Pair Programing Session Overview

Since the session environment was different from our usual, and the audience's agile experience varied greatly (mostly none or little) we decided to allow the audience to ask questions during the presentation. The most successful pairing environments leverage a strong agile environment, so much of our content relies on certain knowledge of other agile practices.

Since we didn't limit questions we ended up covering the first 5 slides and lots of questions in the first 45 minutes of the class.

Not exactly the definition of a breakout session.

This made some people very happy and some people, well, not.

If you enjoyed the session, I'm really happy it helped! I hope you gained some insight you can take back to your team and good luck!

If you didn't, I hope you left early and gained interesting knowledge elsewhere you could take that back to your team and also, good luck :)

Joe White has posted a great summary of everything covered -- thanks Joe!

Most of the questions we answered would have been covered during the lecture, so if you are worried you missed out on some important information, no worries, you just got it in a different order.

Followup to Testing with Mocks

Thanks to everyone who came to the Testing with Mocks TLC session at TechEd.

Here are some links of things we touched on during the class. Please leave a comment if I missed something :)

Test Smells:
TDD Anti Patterns -- Be sure to read the comments, there are some valuable smells there too!

Books:
The Art of Unit Testing
Working Effectively with Legacy Code
TDD by Example (Kent Beck's book)

Rhino Mocks:
Ayende (creator of rhino mocks)
Google Group
Reference Guide

Other .NET Testing Frameworks:
nMock
TypeMock
Moq

Test Runners:
nUnit
mbUnit

Tools:
Resharper
TeamCity
CruiseControl.NET
CruiseControl.rb

IoC Containers:
Castle MicroKernel/Windsor
Spring.Net
StructureMap

Community:
Alt.Net

I feel like I'm taking crazy pills..

Today, I was trying out a wpf grid control and couldn't figure out how to style or data bind or do anything with it in xaml.

I sent the sales rep an email asking for some examples and I was told data binding, templates and styles were not currently supported (dependency properties it seemed too).

With that, I thanked them for their time, and said databinding and styling were a priority so we could not consider their grid.

I didn't expect to hear much after that, but I was informed that those "bells and whistles" were not as important as the other features only their grid supports...

While I would argue these "extras" are a foundation of WPF, I don't think that should matter.

Why are my priorities not valuable?

When listening to user feedback on our applications and products, we can get defensive.

We're all human and insecure, so I guess that could explain it.

However, I would rather focus on a different issue -- what is guiding our decisions, priorities and emotional responses?

Why is the response to user feedback a reason instead of thank you?

All feedback is an opportunity to create a better product. It is great service your users provide (usually for free!).

One of the greatest technical challenges in Ript was adding the ability to rip Flash. It took many, many spikes. I think most other companies would have put it aside -- it didn't add much value, wasn't going to make us money, isn't something that will make a person use the application and certainly cost time and money.

So, why did Gerry make it a requirement?

During user testing, we noticed people thought they were doing something wrong when they couldn't rip flash. They don't understand that some images are flash and some are images.

Gerry could have said document how to determine if an image is Flash so users will know why they can't do something.

But, she understood that it didn't matter if the behavior was documented. If a person thinks its broken, begins to lose trust or feels they are doing something wrong, they are alienated. They will not feel confidence in themselves or the usefulness of the program.

Before working with Gerry I didn't understand this concept. I would have put flash aside. However, her decision is what made her a great product owner.

Where I'm going, where I've been

I'll start with the where I've been:

After NBC took over Oxygen, our lovely, little agile group got a bit... lost. We lost our product owners and inspiration and little by little, we found other opportunities and now, there is no more group.

Next, I went on to an agile coaching role. While bad experiences can inspire the best writing, this was not the case. I decided to resign quickly.

That leads me to my next bit of news -- I am 20 weeks pregnant. I want to spend every week before embarking on motherhood on a worthwhile and fun adventure!

So, where am I now?

I am consulting in NYC on a motivated and talented team working on WPF applications. Its great to be back in this space!

Where am I going?

No idea. From what I hear, into sleepless nights! I plan on working after the baby, but nothing is official at the moment and I'm happy that way.

What about the hot agilistas?

No worries! Oksana and I will still be at TechEd in June. Pregnant women are hot too! Seriously, 7 months pregnant in June in Florida! Who plans that?

Painless CI?

I recently started a new project and had the fun task of getting the build server up and running. Usually, I dislike the entire process -- creating a nant script, configuring cruisecontrol... so much xml... so little fun!

Since its a new project, first order of business -- rake for builds! Something we experimented with at Oxygen, but didn't add to our practice. If you have the options of putting ruby on your build server, switch to rake if you are not already using it. Its easy to read, easy to build projects and you can reuse code. Such a nice change from the copy and paste nightmare of nant.

After spending 1/2 a day working on cruisecontrol.net and trying to figure out what configuration I needed, I realized -- hey, you can use cruisecontrol.rb -- ruby is already on the build server and we're using rake.

I'd post directions on the extra steps or setup I needed to do, but the directions from the cruisecontrol.rb site left me with a working build -- imagine that?

Tech Ed 2008

Oksana and I will be speaking about pair programming and using mock objects at Tech Ed 2008.

Register for Tech Ed here

The conference is in Orlando in June, so bring your bathing suits!

Gerry Laybourne on agile

From an interview with Gerry (page 6, scroll about halfway down the page):

"...And through that, I really got to know them and decided that I really wanted to understand agile management—which is a very interesting way of managing. It's a very formal discipline that allows the team to own the work. And what happens in software development is that you can give an assignment to a software-development team, and then six months later, they deliver what the assignment was. And you look at it and say, "Oh no, that's not what I wanted." In agile management, you work on a two-week cycle. You're working. You have a "scrum master." You have priorities set. You agree on what the priorities are in the meeting. You review the priorities. You evaluate where you are, and you move to the next step..."

I love that she knows agile is a "very formal discipline."

Whatever happened to sustainable pace?

Who practices sustainable pace?

Development is an art. Its a mental challenge (emotion if you pair). As a day goes by, an individuals definition of quality changes depending on their mood and energy.

Over short periods of time a team can keep up long hours; especially if there is positive momentum or results in sight.

But permanent hours to get more out the door? Does this really work with out sacrificing quality and practices?

How many times are corners cut to meet a deadline? Was it ever too much to write that test first? Was code copied without refactoring to remove duplication? Is there spike code in production cause it worked?

Is that our risk to take?

Can a person control burnout or realize the compromises they make? People have an amazing ability to adapt to any situation. In a bad situation, we forget, we lie (mostly to ourselves), we ignore and we accept things we shouldn't.

Morning Scrum -- Remote Style

This is what life at Oxygen looks like in the morning (from the comfort of my home):

the joy of pair programming

I started speaking about pair programming because I didn't like it and I needed to understand this practice better.

Sometimes I still don't.

But, I admit this:
Pairing is a great practice.

Great, but surprisingly difficult.

The ideal pairing situation requires both people to be expert developers. They need to be open to the other person's idea. And in this case (expert developers with good, strong opinions), its likely to bring pain.

If the pair is not parallel in skill (which is often the case), the stronger person must slow down and take on a mentoring role. Being a mentor when you want to be as productive as possible is frustrating. Many developers are poor teachers, but communication and discussing ideas is key to creating a great solution.

Every person you work with is different and you create a unique pair. You will go at different speeds, create different designs, even make different jokes.

Pairing is a good mechanism for teaching, but the real strength is in design and code. I have seen no other practice keep every team member familiar with every project, design and decision.

The whiteboard is rarely used because the pairing session isn't about code, its about creating. The design decisions are made when the pair needs to make them. Any previous design or decision may not be the best solution when (and if) its needed.

If discussion can be avoided, don't do it. The best way to be productive is to keep coding and evolving. Focus on small steps to deliver functionality quickly and ensures CI success.

With pairing every moment is a code review. If the team switches pairs and owns the code, everyone is familiar with everything. No reviews, no whiteboards, no documentation.

Imagine the possibilities.

Working remotely

For the past few months, I have been working for Oxygen remotely. We learn better ways to adapt to the environment everyday, and overall it has been successful. Ken has posted plenty of pictures of our remoting adventures and the worst part is that I miss everyone.

I love working from home, however, it takes a lot of discipline -- and not just for me. My teammates need to be aware that they need to speak a little louder or stand by a camera and I need to stay away from halo during the day :P

Personally, I find it easy to stay in a work mode while I'm at home. Pairing is quite effective remotely. I tend to keep better focus while I'm remoting with a pair than in person.

Meetings are a lot more challenging. Good microphones and video feeds are a must if you are to have any chance of "being" in the meeting. When I first started working from home, I wasn't able to participate in meetings at all, but with help of some additional media equipment, I can go to every meeting in my pajamas :)

If you are working from home, here is some advice:

1. Have space dedicated to work. This helps you stay focused and keeps you from working too much. If I setup shop in my den, chances of me working extra hours while my husband plays xbox is dramatically increased.
2. Be honest and open about communication limitations. Its better to interrupt if your call drops off or someone is too far from the microphone than miss something.
3. Respect your coworkers during meetings and while they're talking. Just because they can't see you surfing the web while they are talking about some deployment doesn't make it okay.
4. Take breaks, walk the dog, get some exercise. Believe it or not, its easier to sit on the computer during lunch than in the office. Its just as important to walk around when you are at home. And stay away from the snacks in the kitchen!!
5. Get a really good espresso machine.
6. Visit the office. Even if you don't have to come in, its good to keep the bonds strong with teammates. If you are too far to come in on a whim, try to work something out with your company that lets you come in not just when you have to, but when you want as well.

Inspiration

For the past year and a half, I have had the opportunity to work with Gerry Laybourne. She is my hero. She inspires me as a developer, creator and woman. Today, she gave a speech at the Microsoft Women's Conference. I wasn't able to go, but was able to read a copy. Her words are strong, truthful and inspiring. She harnesses everything I love about technology and why being a woman in this field is challenging and special.

Gerry is leaving Oxygen at the end of the year. The idea of creating software at Oxygen without Gerry seems unimaginable. But like any change, sometimes you don't think you're ready until you take the next step.

I hope to one day inspire someone the way Gerry has inspired me.

Fairfield/Westchester code camp

I will be at presenting a session on Testing with Mocks at the the Westchester/Fairfield code camp on November 11.

Register here

The session is a shortened version of the tutorial Oksana and I are giving at the Agile Development Practices conference in December.

Hope to see you there!

Good practices are beyond testing

Before using TDD, I was a big fan of separation of concerns, single responsibility and inversion of control. These things have proven over and over again to be priceless.

When I started using TDD I loved is how it enforced these principles. I couldn't maintain or write tests without them. Testing even forced the need for dependency injection and inversion of control containers.

The tests started to validate and enforce important behaviors. Behaviors some people don't appreciate. Sometimes these had been a hard sell.

TDD became an easy justification for best practices. How awesome!

Enter mocking libraries that don't require DI. You can mock everything from constructors to static methods. Mocking methods on the CUT is as easy as creating a new class for that responsibility.

With this, its easier to make a large class and use static utils and real constructors. Now you can test without adding those pesky things that add to the confusion of object oriented code... you know stuff like interfaces, polymorphism, encapsulation...

If we don't need DI or interfaces for testing, then why do we need them? If testing can happen without it, then that behavior isn't driven, right? Why can't I make a big class if its testable?

But these design principles have been around for a long time and for good reason. DI, single responsibility and separation of concerns were not born from testing.

Does Intuitive == Maintainable?

What makes code maintainable? What does it mean to be readable? Why does it matter?

My coding standards and styles change constantly.

I have yet to return to a project after a hiatus and understand it without some investigation -- no matter how intuitive it originally seemed.

If this is so, how can something be maintainable? If a project can be written in a few weeks and then not touched for several months, is there any chance you'd automatically understand or agree with the design?

There are very few solutions that cannot be figured out. Sometimes it takes minutes. Other times a bit longer. A lot of times it involves grimacing, cursing and questioning the sanity of the original developer(s).

But no matter how "awful" a solution may be, at one point it made sense to someone, so with some detective work it can be understood.

Something common of software that is easier to maintain is single responsibility and separation of concerns. These principles are not subjective. They don't change with time, mood or business domain.

If a project follows these principles understanding the entire system isn't necessary, just the part that needs to change. It may take some time to find it, but once found, the solution is simple.

Alt.Net Registration...

Is now open -- register here, see who's coming here

The conference is limited to 100 people, so register quickly.

Hope to see you there!