Category Archives: Development Process

another reason % of time writing tests is meaningless

Posted on by
5

Earlier in the month, I blogged a response to – “What would you say is the average percentage of development time devoted to creating the unit test scripts?”.  As I was telling a friend about it, I realized that I missed an important point!

The question also implies that development time is constant.  Or maybe not.   But if I tell you that I spend X% of my time on creating unit tests and your project currently spends Y time on development, I imagine you are considering the development time with unit testing takes Y + Y*X.

However, this isn’t the case.  The real time is Y + Y*X – Z where Z is the time saved in finding the defects in the code faster.  The benefits span past the current project, but this factor matters even if one just measures within the current project.

Or another way of expressing this is that “your Y is different than my Y.”  My time on development has been adjusted by creating software in a different way.

This seems like a convoluted way of expressing it.  Has anyone come across a better explanation?

a response to – “What would you say is the average percentage of development time devoted to creating the unit test scripts?”

Posted on by
2

How long does it take to compile? We don’t ask that. It would be absurd. The fact that people ask how long unit testing takes mean they see it as an optional cost to be incurred. What I want to know is why they don’t ask for a similar accounting of the cost of NOT writing unit tests!

I think the “cost” of unit testing falls into three categories:
1) The early days/training/learning
2) The ongoing cost
3) The cost of not testing/cost of errors

The early days/training/learning
The first time you do something, it takes longer.  This is why we pay experienced people more. It is expected there is a training or learning cost. If an activity is worthwhile, one recoups this cost quickly.  It is called an investment.

The problem I see is that some teams don’t get past this point. They see unit testing taking longer the first time and imagine it will take that long forever. I wonder how these people learned Java or regular expressions or anything else. Except that they wanted to learn the hard tech. If one believes the current buggy state of affairs works or can’t imagine a better way, staying motivated to get over the learning hump is difficult.  This is why having a mentor or coach promoting unit testing is helpful.

The ongoing cost
When writing unit tests as part of the task, it is difficult to measure the
amount of time it takes.  I really can’t tell you what percentage of development time I spend “writing tests” because it occurs at the same time I do the other parts of the task.  I also couldn’t account for the percentages of time I spend typing, thinking, compiling, etc.  These things are occurring simultaneously.

Many of the people who complain the ongoing cost of testing is to high are treating it is a separate activity. They write the code, manually test it, debug, repeat and then write a unit test. If done that way, the entire cost of writing the test is extra. And often resented because they really are “done” before they start writing the tests.  In this case, the unit tests only have value as regression but not as part of the development process.  While I suppose it is better than nothing, I find this a way to make writing tests more costly than it should be.

I’m not saying one has to use TDD (test driven development) to see a benefit. Using the unit tests as a replacement for that manual testing lets it become a cost you need to incur anyway. Granted, it takes a little longer to write a test than test by hand. But this is minimal. It is also offset by way less regression issues and the ability to test error conditions easily. And if you write tests in close proximity to the code, it is even faster.

There’s another measurement issue going on here.  Yes, it takes longer to write tests than test manually ONCE.  If you are writing a script that will only be run once, it isn’t worth it.  How many times have you had an application that was written once and then never touched again?

The cost of not testing/cost of errors

This is the part that really bothers me.  There is a cost to not doing an activity.  It tends to be swept under the rug and treated as a cost of doing business.  It is hard to see opportunity cost.  But it still exists.  The two biggest costs I see are finding errors late and regression errors in future release.

  1. We’ve all see the curve that shows how fixing errors late is much more expensive than finding them in development.  Unfortunately, developers get to claim they are “done” when they’ve really just moved the errors to later.  They are more expensive, but the developer gets to claim they finished the task in X days.  Managers need to stop allowing this.
  2. The future release problem is even harder.  I think more of the value of having unit tests comes from the future.  Some developers claim that they don’t need unit testing because they produce high quality code without it.  Sometimes this is even true.  However, what happens when that developer looks at the code in a year or leaves the team.  The unit tests are code and live on.  My development velocity on maintenance/enhancement tasks is much higher with unit tests because I don’t have to posit what I was thinking a year ago when I wrote the code.

What if my management still needs a cost

This blog post is inspired by someone asking me this question.  “What would you say is the average percentage of development time devoted to creating the unit test scripts?”  While I take issue with the question, I think he is still going to have it.  As a result, here are links to three articles/webpages that use numbers.

  1. Misko Hevery comes up with a figure of 10% cost.  He calls it a 10% tax and points out the benefits that come a tax.  Note that he is writing tests as an integral part of his process and is fluent in doing so.  He also actively dispels the myth that testing takes twice as long.
  2. Brian Johnston discusses costs. I like that he covers the cost of not testing too.   While he picks extreme figures (based on the worst case myth), it does cover risks and things to take into account for your own shops.  Also, he discusses the hardest 15% of tests.  The earlier tests written are the easier ones and cost less.
  3. A variety of opinions wiki’d. While numbers are mentioned, the real value of the page is the caveats for dealing with such numbers.

Conclusion

When talking to management about a cost, make sure they know about the associated benefits.  And the cost of other options.  Doing nothing is an option and has a definite cost!

see part 2 of this blog post

The “Reinventing the Wheel” Anti-Pattern

Posted on by
1
Reinventing the Wheel As a moderator on the JavaRanch, I often come across posts asking how to reinvent features that are available in most application servers. In JDBC for example, I’ve had people ask how to implement their own database connection pooling and how to create their own JDBC driver. Often times the developer is trying to create something that already exists, but they are unsure how to use it. The general rule of thumb I tell programmers is, “If you feel like you are reinventing the wheel, you probably are”.

Review the API
If you ever get the feeling you are inventing the wheel, it’s a pretty good indication you are. In such situations, ask yourself “Is it likely a developer using the same component would need functionality X?”. If the answer is “yes”, then there’s a good chance there’s already such a feature in the API. Most often, reinventing the wheel comes from developers who are too lazy to review the API but not lazy enough not to rewrite the feature they think they are missing. Also, search the web. In some cases, you may need to download a new or updated library to get the feature you want, but this may give you access to even more features.

Your wheel isn’t better
Often called “Reinventing the square wheel”, it is likely the code you are recreating is worse, more buggy, and far less stable than the code you should be using. If you consider it for a moment, it makes sense a method built within the API should be better than a method built on top of the API. The API developer has access to private methods and objects that you do not have access to, and therefore your solution is limited by the public/external methods of the API. Furthermore, the fact the code is part of the API means scores of developers have hopefully reviewed the code for errors and performance enhancements.

That doesn’t necessarily mean the API implementation is better, but whenever I hear a developer say “I have a faster and cooler way of doing this than the way they do it in library” I cringe at the thought of what they may have written. If you do happen to create a better wheel, join the open source project and publish it for others to judge.

There’s always public humiliation
Many of the worst best articles that appear on the The Daily WTF come from programmers reimplementing the most basic functions of a language. One of the things that separates an experienced programmer from a beginner is the ability to recognize what tools in the API are needed for a task and how quickly they can be put together. And with that, I present a list of examples written by real developers and posted online for the world to see:

Any my personal favorite: I’ve heard from some good sources that the next version of SQL will use the word “GIMMIE” instead of “SELECT”