Ensure test coverage when refactoring to mocks

Using real dependencies in your tests will lead to pain. A lot of pain.

Most likely, your test fixtures lack coverage, test the wrong class, be difficult to read and don't do what they say.

Eventually, your team will decide to abandon testing or introduce mocking/stubbing.

Refactoring your existing tests will be a major endeavor.

It will be tedious and repetitive.

You will feel like you are wasting time.

You will have a strong urge to rush.

To help you out, I've outlined the steps needed to refactor a test to mocks. I've noted steps that are easy to skip (yet still important) with *

1. Make a copy of the TestFixture -- call it TestFixtureNameWithMocks -- this will ensure you do not lose test coverage and can commit often.

2. Comment out all tests

3. Delete the setup code

4. Take a moment to read the first test

• Does this test actually test anything? (if not, figure out what it should be testing and get it working right in the original fixture first!)*
• Does it verify state of the CUT?
  
• Does it verify state of a dependent (or dependent of dependent of dependent...)?
• Does it do more than one verification?

6. If there is more than one verification, determine how many tests you actually need.

7. If your test relied on the state of a dependent, verify that behavior is tested in the correct test fixture (i.e. the dependent's!). If not, you need one, use this test as an example. Create a failing test now, before you forget. If you have a rabbit hole of dependents, make a test for the top dependent.*

8. If needed, break out the test into multiple verifications -- select one to do, comment the rest.

9. Uncomment test and rewrite using mocks/stub. You will find that some tests are quite different. Some similar. Some will require refactoring of the CUT or dependents to make mock friendly.

10. Run the test and verify it passes. If it does not, figure out what preconditions are missing -- check the CUT and old test. Keep doing this till it passes.

10. Comment out code that is being tested.*

11. Run test, verify it fails for the right reason. If it does not, check the test and verify your assumptions. Make it fail for the right reason!*

12. Uncomment code, verify it passes. If you didn't change code in 11, you can skip this step -- I am paranoid and usually do it anyway :)

13. Repeat for every test in the fixture (and any new tests you find). Pull out common setup if needed. Rename tests since the original names will most likely need it :P

14. Remove old test fixture and rename new one.

If you made it this far imagine how long rewriting one test takes. In the best case you still need to run each test at least 2 times.

Now imagine doing this for more than one class.

How about for how many tests exist by the time our tests are painful enough to require this change.

If done right, this will take a lot of time. If we do it wrong, we take great risks and have tests that give us a false sense of security.

Clearly, if we want to continue delivering end user value, we cannot do this all at once.

Here is one way to do this:
1. Whenever a test is updated (bug fix, enhancement, refactor), update the test fixture to use mocks.
2. Have a policy where it is ok to have 2 fixtures for 1 CUT (one with mocks and one without) to encourage people to start using mocks w/o the overhead/context switch of a big refactor. Stress that finishing the refactor takes precedence over starting new work.
3. Let everyone on the team know that it is expected to take time and include that in commitments and estimations.
4. Remain disciplined

Good luck!!

Adding to the team

We've all felt the pain of the interview process.

It sucks. And its not just the candidates out there. Its also the way we find them.

Typical Procedures:

1. Recruiter or ad
2. Phone interview
3. Written questions or code samples
4. In person interview
5. Meetings with other team members
6. References, background checks, drug tests, security clearance, street fighter challenge
   

There certainly is dysfunction here. Many interviews feel like a game and its not played well (or fair).

I think of finding a new team member as a process of filtering:

1. The process is a series of filters each more fine than its predecessor
2. A filter should have definite output
3. Good people are difficult to find. Too fine a filter early in the process is a risk
   
4. Most filters have manual steps. Filters that filter too little are a risk.
   
5. Finding the right person is personal to the team, so any filter that does not involve a team member is less accurate
6. A good fit is unique to a team, so rarely will the same sequence of filters fit multiple jobs.
7. Some criteria may give someone a pass around filters -- personal recommendations, proven methodology expertise and already working for company are a few examples

Before we can setup our filters, we need to know our goals for a candidate. This is a lot harder than it sounds. You may find you don't really know your "goals" beyond a "senior" developer who is a "team player" and "experienced" with our technology and methodologies.

Once we know what is important, we need to figure out how to filter this behavior. We must also look at a persons ability to grow into a goal and the team's desire to teach (which will be biased depending on the person).

Example:

• Goal: We want someone who is an enjoyable pair programmer
• Importance: We only pair so friction here is best avoided
• Ability to Learn: Developers, like all humans, are known to be stubborn in changing their personality. Though people can come around.
• Ideal Filter: Spend a day pairing with them, switching people so at least a few team members weigh in

Clearly this is not a recommended starting point, unless you want to have a stranger on your team everyday poking around in some code in your project. Or maybe it would work if you have a lot of spikes?

Now what?

We can dig deeper into this goal:

1. What kind of traits make someone a good pair?
2. Are there ways we can define questions that will clue us in on these traits? Do any of these questions have answers that we can entrust an outside party to interpret?
   
3. Can we use their past work experience to help?

If we find this goal is best saved for later in the process, we can move on to goals that are better found at coarser levels. Do we required a certain amount of experience or schooling? Do we want people from a similar background (domain, technology, methodologies)? Do we care if they change jobs a lot? Too little? Are they active in learning new technologies? Are they interested in our methodologies? Do they live nearby?

We may not care about individual answers to pass judgment, though a combination of answers is sure to give us something.

How we find people to review is also a filter. If we want someone who is involved in the community, we can use those channels to find people either passively (twitter we're looking, ads on blogs we like) or more actively (read blogs and tweets to find people, find people working on open source projects).

Remember, its not just what you want, its what they want. Make sure your job description includes things that will attract the good fits (or have some people running to the hills), like weekly book readings or Star Trek Fridays.

Finally, there are unique things at every environment. What attracted the team there? What are team member strengths and weaknesses and what is the state of the project -- how can a new team member help most? Do we need another good pair or is it more important we get someone who can help us fix our data layer and quick.

Finding a good fit is a lot of work. If we spend the time to figure out what we want, we can figure out better questions and ways to find them. I'd say its a good exercise, even if its not quite possible to create a systematic filtering process.

Off topic, I'm in favor of speed dating style of interviewing: open house at your company, each team member gets 7 min with each person who shows up. Anyone with all yays moves on to the real interviewing. It may waste a day of everyone's time, but I bet it would be fun and if it worked, it would save a lot of money on recruiters, ads, phone interviews, code reviews and spending time figuring out what "questions" will help us find the best fit.

Can't seem to ever finish a post...

Between working and mommying, I can't seem to finish any of my blog posts -- this is bothering me...

So, I signed up for twitter -- wundarous

Maybe I'll get some ideas out...

or maybe it will be random thoughts that make no sense..

Either way, no excuses not to add a sentence here and there!

The need for speed

I say this a lot, but I love TDD. Besides the list of reasons about how it improves code or simplifies solutions, TDD is fast.

TDD is fun because it goes so quickly. Write a test, make it pass, refactor, run more tests.

I love the flow. I love adding a new interface and using it without worry about implementation. I love knowing after I make a new test, something in the app is different and it works. I love living in the tests and only running the application before checking in.

However, the moment the tests slow, the practices start to slip...

1. Writing code and verifying it works by launching the app, then creating the tests
   
2. Making the test pass before verifying it failed.
3. Not running all the tests before check in.
4. Not refactoring because it takes too long
5. It stops you from working close to 5 because you don't want to wait for the build
   
6. Not adding tests at all because you don't want to break the build and don't want to wait to find out
   
7. It ruins all the fun.

These things all lead to poor test coverage or tests that don't actually do what they say. Yeah, on paper your coverage may look nice, but there's plenty of code you can delete w/o breaking tests. Coverage makes sure a code path is followed, it does not verify the path was actually tested.

So, how do we keep our tests running quickly?

1. Mocking. Reduces the amount of time spent creating dependencies and their dependencies and the rabbit hole of dependencies that nobody can even see.
2. Use a one to one ratio of test assemblies to application assemblies -- don't waste time waiting for things to build that you're not testing.
   
3. Keep interfaces and implementations in separate assemblies. Dependents should only be recompiled if the contract changes, not the implementation. This allows you to complete the usage of a change, without worrying about the implementation. Sure, you can just resharper, but are you really going to add tests for a new method right now?? If you do, you break the rhythm, if you don't, you have to remember, somehow, that there is code that needs to be implemented somewhere that your tests won't pick up... you'll start thinking about the implementation or just forget and break the app without anyone knowing.
4. Simplify. Keep tests short. Only create items in setup that are used in all tests. Avoid factories for creating concrete classes because they hide dependencies and make it too easy to use real implementations instead of mocks. Avoid repeating yourself in tests, don't assert the same thing twice, it doesn't help to have the same failure twice but it does slow everything down.
   

In general, follow the best practices of test writing! They're there for a reason. No point in experiencing the pain of the nice people who figured them out. Take advantage w/o having crappy tests, lots of pain and failure.

When Agile Works

When I started consulting on an "agile" team back in February, my first thought was, "Hey! You guys lied! This isn't agile."

The working practices were reminiscent of James Shore's recent blog post.

There was Scrum but they lacked visibility, risk mitigation and self management.
There was a morning stand up, but it was a status meeting with over 20 people -- mostly watchers.
There was a single product team, but members were broken up into domain groups with their own managers, politics and definitions of success.

There were bad practices, communication breakdowns, over-commitment, over-time, half finished features, inconsistency and plenty of technical debt.

Team members had all the stress and no power to change.

But, there was hope.

Within the first few months the company reorganized and moved towards product hierarchy eliminating the contention between groups.

=>Consistent goals and priorities.

The BAs, design and product owner worked closely with an agile coach to create a product backlog. They began having sprint reviews and retrospectives.

=>Visibility and reflection.

A few people became scrum masters. A few others scrum product owners. They started going to agile conferences and local meetings.

=>Engagement.

I left at the end of July to have Kaylee. Things were changing, but we were still far from a well functioning agile team.

When I returned in November, I heard that our agile coach said this was the best agile team he's been on.

I thought to myself, "I think he's been drinking some kool-aid..."

However, when I came in for sprint review and planning, I was amazed.

So much changed.

Team members were involved.

Practice improvements and innovations were made throughout the day.

Charts and metrics were used, not shown.

People were having the right conversations.

The differences not only showed in the team and interactions, but the product.

They had visible success.

I feel very fortunate to have the unique look into the evolution of this agile team.

When you are on the team, it can seem like things don't change. Our retrospectives are a microscopic view compared to a projects lifespan. If I was working for the past 3 months, I may have not noticed.

Maybe a periodic review of how things were/how things are would be a good motivator and illustration of how cool agile can be when its working.

3 Months Later...

Almost.

Babies are challenging... Giving birth was easier... Forget about the note I was sleeping better after Kaylee was born, that didn't last long!

Ah, but the human condition. We adapt. Disrupted sleep included.

Now that I've adapted, I'm consulting again.

Apparently, I really, really like to code AND its much, much easier than being a mommy! Not quite the same ROI...

Since I do most of my work from home, I get the best of both. I'm quite lucky.

I'm with the same company and working with lots of WPF and TDD. We just started using the Isolator AAA from typemock, which looks nice. I know Roy was very involved with its creation and I usually agree with all his testing methodologies :)

My heart is still with rhinomocks, but hopefully I'll find a place for typemock too!

And being a mom, I have to include some newer pics of my lovely daughter!

My Little Wunda

I'm proud to say I'm a mom to a beautiful daughter born Saturday, August 16th. I'd like to thank my great birthing team of my husband, Vanessa (his sister) and my doula, Amy. Between them and hypnobabies, I was able to have an incredible birthing experience!

To keep up with the academic nature of this blog -- I'd like to share what I've learned so far in my few days of motherhood (note, I never picked up an infant or changed a diaper before Saturday)

1. Giving birth was the most intense, satisfying and romantic experience of my life
2. Its better to let your dog lick your infant than a person kiss their face
3. There's all kinds of great feelings for your own child you could never imagine
4. Wearing your baby is lots of fun and lets you do something while they're sleeping other than watching them sleep
5. Its still early, but so far I've been sleeping better than I have in the past few weeks (my husband however, is learning the joys of interrupted sleep)
6. Every person you talk to will give advice and everyone has different opinions on the same things (that goes for doctors, nurses, grandparents and strangers)
7. Babies like loud noises, cry before they go to the bathroom, fart louder than you could imagine and produce incredible amounts of poop!
8. If all you do is breastfeed, change diapers and sooth an infant, it can take you days to write a single email
9. In a single day, your entire outlook, appreciation and viewpoint changes completely
10. Every coo, cry, movement, poop and breath from your infant is precious

Although I'm sure I'll have my hands full, hopefully, I'll get to write some of the posts I had been putting off for the last few months while I'm away from work -- I've had a lot of adventures in WPF and TDD I'd love to share!