Everyone nowadays is writing tests like crazy, and even using TDD so that tests are always written before actual code. Right? No.
I have used TDD for one project and I wasn’t happy, it was waste of time. Nowadays I write unit tests very seldomly (in my case they are more integration-tests than unit tests) and never follow BDD-style development. That is mostly because I’m a lazy-ass bastard, but there are other reasons as well, such as:
- “Examples” of unit tests suck donkey balls every time. If you are going to keep a presentation about how cool unit tests are, try to show me the practical benefits with a code example, not some lame-ass “Hello world” example or hand-waving. I haven’t seen any convincing real-world cases.
- They seem to be a great way to spend some extra time doing time consuming, not very hard work.. And if you are a sub-contractor/consultant, that is pretty good because it means more billable hours. So good for consultants, not for me.
- You have to think about the “evil” business side too. OK, I believe how TDD/BDD will make my project ultra-maintainable, and unit tests will alert me if something gets broken. But what if my project does not need maintaining because it wasn’t commercially successful? Often I just want to get something out of the door quickly, and I want to minimize my time developing the software.
- Unit tests=more code. This is just a personal belief; I think the less code, the better. I’m a minimalist.
Don’t get this wrong, use BDD/TDD if you really believe in them and think they benefit your project. I just personally haven’t found any use for them in any of my projects or my clients’ projects.
[...] This post was mentioned on Twitter by Hacker News YC and Sean T Allen, hackernews. hackernews said: Why I don’t use TDD or BDD http://bit.ly/9tbUg5 [...]
I bought the peepcodes for rspec, looked at all of the examples and basically came to the same conclusion you did. I also work solo on damn near every project I touch and as a rule don’t trust anyone else’s code.
HOWEVER, a few months ago I started on a project that’d have some pretty high-profile eyeballs on it. If anything changes the way you code or document, it’ll be the idea that someone you really respect will be going through your code later on (either to help or to criticize it). I totally went hardcore on this one, and wrote all of my tests in advance, went crazy with edge scenarios, the whole bit. It was a HUGE waste of time and I only did it because I knew other people would need to dig in.
I learned a few big things in the process:
1. The examples are complete trash when you get into anything beyond the first couple levels of normal app complexity.
2. Mocks vs Stubs — I’m convinced the people using either have no idea what the difference is.
3. Simple stuff doesn’t break, so don’t effing test it!
From here on out, I’m only testing models and their methods along with the more complicated libraries I need to write. I wrote my own in-house gem that includes all of the convenience methods that I like, they’ve all been written from scratch, and they all are tested the way I think they should be. NO dependence on anyone else’s crap whenever possible. Testing controllers and views is pointless in my opinion. There’s a balance that each person needs to just find for himself. If you’re a TATFT guy, fine, and if you don’t test at all, no biggie.
TDD has become a cult, and they’re all just trying to justify THEIR particular waste of time.
How can you judge something that you haven’t tried for a decent time?
Unfortunately you are taking TDD as an isolated thing and your are not considering the important role that it plays in an iterative project with a team of more than a couple of people. Read Extreme Programming Explained and you will understand how Refactoring, Evolutionary Design and TDD go hand in hand to allow code to be supple. Of course if you want to get out of the door quickly you don’t need TDD or even object oriented programming or good code. Surely you can even using a massive php page with lots of query in it. TDD was born for complex systems like payroll or software like the one I’m working on at the moment, a media planning tool for advertising agency. This stuff is full of complex logic that will go tits up without unit tests.
Personally I now use TDD for my personal web projects done in rails because once you grasp the technique it doesn’t require extra effort and actually reduce development time, the boring one (debugging).
David: I have used TDD, and I wasn’t happy with it for that particular two-developer project. I found that we wasted lot of time enforcing the unit testing model while we could have gone without it easily and save lots of time.
Perhaps that states my viewpoint better: I’m not again writing tests with code, I just don’t find the unit testing model that useful. In fact, I write kind of tests for my own projects, but they are not unit tests, but more kind of integration test to help the hand-made testing.
And I gotta admit, I really see the benefits of coded tests for software such as web servers and compilers/parsers. For web applications though, they often seem to be just waste of time since most of the bugs you can only find testing by hand.
It’s hard to justify the big investment in getting up to speed with TDD. I think it took me more than a year to grasp it correctly. It’s not about testing at all but it’s a development process. At the beginning there is so too much overhead because writing tests takes ages and simple web app developed by two people don’t payoff the investment so quickly.
I wouldn’t trash the all technique thought, as usual context makes the difference. Check out Kent’s beck tutorial about TDD http://www.pragprog.com/screencasts/v-kbtdd/test-driven-development you will see how he uses TDD as a development process and not as way to test is code (that come for free).
I would second looking at Kent Beck’s screencasts on PragProg.com. They show how TDD is actually used as a development process. I don’t like his example because in my mind those are integration tests not unit tests but its certainly not a toy example.
What I have found is that only about 10% of programmers who don’t TDD already are capable of getting it. The only way for them to learn is to be shown over and over how to do it with pair programming. It takes about a month of constant pairing before they finally understand what they get from it and start using it by default. 10% isn’t good odds, the other 90% get fired.
TDD is a skill that takes years to learn well IMO. You can understand the principles but you don’t get much of the benefit until you’ve been doing it for a while. In the beginning your coding style doesn’t change and you are still writing big classes which are hard to test and you end up with these crazy mocks and various other ugliness in your tests. After a while however that goes away as you understand how to make the tests clean and evolve the design better. Actually learning to refactor rather than redesign is another complimentary skill which few developers without TDD have.
There will always be companies hiring programmers that don’t do TDD development of course. Those companies don’t know any better. You’ll be employable for years to come. But TDD unfortunately is proving to be highly effective at reducing bug counts with minimal increase in development time over and over in industry. I can’t help you if you’ve decided against the practice, but I can tell you I would never hire a non TDD programmer and its becoming much more common in the industry to expect the skill.
I’m with Jeremias on this. The worked out examples of using TDD “suck donkey balls.” The best ones I could find, from Beck, Martin, and others, end up with incomplete code – incomplete range checking and incomplete assurance that the code does what it’s supposed to do. I gave details in my link.
I looked for any sort of research which shows how much more effective TDD is over a non-TDD process like rapid application development, which is my preferred approach, but there’s very little about the topic. There’s mostly case histories of the “I love it” and “I tried it out but it didn’t seem effective” types, but nothing I would call conclusive.
(You might say it’s not testable, but Prechelt’s “An Empirical Comparison of Seven Programming Languages” shows that at least some topics which people consider to be grounds for holy wars – choice of programming language – can be tested. I think most people ignore the possibility of testing because it’s not something which is easy to do.)
So while “Those companies don’t know any better” I don’t think the TDD people really know better either.
I could never get to do TDD. Never enough time to get into it. Got to deliver fast.
I usually write test after my code is written and I mostly test the complex stuff.
At least it forces me to read my code some time after I’ve written it. Helps find a few bugs before they come live, refactor and solidify the code base.
But TDD continues to be beyond my grasp.
“a fool with a tool is just a fool”.
TDD makes sense if it will be used in a case of software engeneering.
With no idea about software engeneering, TDD will make no sense.
My feeling about writing unit tests, either before or after the fact, is that it is beneficial in a multi-developer project. When you’re the solo developer on a project, you “know” that bit of code is meant to do when you come back to it in 6 months time, or when you change a dependency.
On a multi developer project the developer working on that bit of code (or a dependency) may not be aware of what it’s original purpose was before. The unit test failing is a good way of making that developer aware that they need to be looking further at the code.
I’ve seen numerous instances of tests failing due to a new feature or refactorings being added by a developer later on in the projects’ life cycle – unit tests become even more useful when you have loose coupling between dependencies (such as spring or similar). Without the unit test, that function could have made it into a successful build as the code was working with the new feature, but not the original feature (which the current dev didn’t know to test).
Cheers for the article, though – it’s prompted a useful debate.
In my view, writing the bare minimum test code required and after the code which needs to be tested.
Writing a huge code test harness before the project code, just seems like procrastination if you ask me.
And the larger the test-code requires its own test code. Doesn’t it?
Like you said, if you don’t need to maintain the project, you don’t need TDD or BDD or even best practice or clean code.
I have no problems with Testing and automated testing as well, I have problems with TDD.
I’ve worked, in pairs, and single, with TDD. TDD often is all about unit tests, and this is completely pointless. If you write good, minimalist code, and have the correct tools, there will be very few methods that really need iterative testing. You’re better concentrate on the general architecture and your lines of code rather than pass your time through the useless grind that is unit-testing. I’ve tested middle layers of 2-line long functions, it’s useless. In 4 months it had us discovering only 2 problems. As somebody already said: simple things don’t break. And if you want to refactor all the time, you don’t WANT to have it take 3 times the time by including unit tests.
But I did learn the techniques though, and I do use them when I have to test something really complicated. Regular expressions, for example.
And I also appreciate integration testing. THAT is something useful, because you test whole subsystems, and this is sufficiently complex to break. Automated website testing, or Salesforce trigger testing really find out tons of tricky bugs. As Paul Keeble said: that’s integration (see even functional) testing and it is useful. And it has the added feature of being based on the project’s specifications, so it can be done by independent teams who speak to the client and WANT to find out as much bugs as possible.
So, what I say from experience: TDD s***s, tests are great. Everyone concentates on their job: the programmer on writing clear, short and well-thought code, and the testers on finding out bugs.
I started to write a work example for Test/Behavior Driven development.
It’s work in progress and I write a crapy english but maybe it will amuse you
http://maxblog.bomzhi.de/?cat=11
Sorry about slow approving of comments, I didn’t guess this would spark that big interest + I have approving system in place because wordpress blogs seem to get awful lot of spam…
For a side-note, I would possibly write unit tests & practice cargo cultish ways more easily if my employer would insist. Since I’m self-employed, I have to decide for myself.
I totally agree with you – sure, tests are important, but a whole methodology built around tests? Pah – you need to understand the job that the program is doing. If you do then tests are useful, if you don’t then all the tests in the world won’t save your ass.
I have been developing software for over 30 years. I have been exposed (and over exposed) to most every software methodology devised. I am now semi-retired (EVERY small business wants a web site in this economy), but I still program the way I always have – and it is test driven, in a sense. I have a group of part time testers (humans, not software bots) that are some of most cantankerous quality assurance dudes and dudettes in the world and when they are through with Integration test (I do all Unit tests myself) I have no qualms about submitting the software for Acceptance testing, although it usually takes a week or two for the project tester and I to get back on speaking terms.
In my opinion, TDD is a BIG x 3 waste of time. Get human eyes on the code as soon as possible.
I am glad the comments are reasonable. I guess the idea behind TDD and is to get protected if something goes wrong. “All the tests were OK, so what else could we do?” And hope nobody answers “you could do clean design”.
Doing unit tests can save your ass. If you are on a project with a set of frozen requirements, unit tests might not be that important to you, but if you need to refactor because of changing requirements, unit tests are great to point out the design quirks you missed. In a 1 or 2 person setup, they might not be real important, it depends on the level of communication. For a 4-8 person team, I think it is crucial, I find it hard to effectively communicate with such a large team.
I wish all the developers in my team would “write tests like crazy”, we would have a better safety net when refactoring the code…
“time consuming, not very hard work” : I guess this is the core reason why you did not find unit tests interesting for a project. There are lots of projects for which, indeed, unit tests are not really useful. Websites, GUI clients, DB-related code, etc. have not much use for TDD. But there are other types of projects (business tier, business-related calculation engines, etc.) that get some huge benefits from a code unit test coverage. I have been working for a few years on two to three projects that simply could not have been successful without unit tests.
From my experience, a good sign of usefulness of unit tests is when they become not so simple to write. At some point, you just cannot follow the init + execute + assert pattern, and have to put some more architecture in them. This is why I think you simply have not been working on projects that benefit from unit tests. It you have only seen simple-to-write unit tests, you have not seen a project that really benefits them.
There is another way to recognize projects that need tests : it is when you wish you had some good code coverage for a code refactor. I have seen several code cases where I have started taking a few days writing good tests in order to be able to refactor the System Under Test. In some case, it would simply not have been possible to refactor the code without a code test coverage, or at least not without taking too much risk in terms of code regression.
I agree, though, that TDD is perhaps pushing the tests a little too far. A bit like a project on which it would be required to have 100% code coverage : you then end up spending more time creating tests than you save having them… In fact, the main advantage in TDD is not in the tests themselves, but in the way it helps limiting useless code.
All in all, please do not speak so generally about tests : I understand you have not seen any project that would truely benefit from them, but I can guarantee you these projects exist. And I would hate for some fresh developers to read your post and abandon unit tests, when I have so much difficulty convincing my team members to write some.
Some interesting comments. Originally i was sceptical about TDD. Over time i have really come to love TDD. Not neccessary for the nice test coverage i get but it makes me think about my designs more. If i am finding something difficult to test it usually means im doing something i shouldnt and the design can be improved. When i have nice coverage usually it means my design just looks right. When i need to change something i am confident that my change wont break anything. If a bug does slip through then i add a test that shows the bug and then i fix the bug. Gives me more confidence when passing to QA for testing that my code will work. I also tend to have both unit tests and integration tests so i get the best of both worlds. Yes it takes more time but it is time saved in the end. IMHO.
It depends on your application and the seize and complexity of your project. For the kind of stuff I am working on (over 100 developers, people’s safety involved, HUGE costst of problems found ‘late’) it has a clear advantage for the Business. Obviously at the different levels of testing (Module, module integration, unit …) you find different types of defects. You should not have to find a defect that typically is caused at module level at system level. It just takes too much time. But like I said, this applies to huge and safety related projects.
Too often developers look for the silver bullet, the way we should write code every time, a pattern to follow. As developers we like order, we like neat little pots to put things in. The problem is the world isn’t like that at all and there are times when you need to get products out quickly, to deliver on time and shortcut the budget because the customer (the ones who ultimately pay our bills) wont pay the extra. Sure you might deliver a lesser quality product but thats their decision, their product and their money.
In large projects you need a proper testing strategy, from unit testing through system testing and integration testing to user acceptance testing. TDD is one aspect of this journey but not the only one.
If you are writing rocket guidance systems then waiting for the thing to fall out of the sky will be the wrong thing to do; but writing a new widget for a web store is probably OK to deliver without much in the way of testing.
So where does that leave us, searching for the developer Holy Grail? It leaves us to use our experience as to when we can use the TDD or the JFDI approach. That is even more difficult to teach than TDD.
I like the JFDI approach. Most of the work I get accomplished has a lot to do with JFDI. lol. No but honestly, I find it a lot better to work in teams, and have my partner do my testing while I do theirs…. I’m not sure if this is commonplace but we usually work out all the bugs quicker. It was true in high school and college, and remains true today you can’t proofread your own work! Same applies to programming… just IMHO anyway. Cheers.
TDD is for limp-wristed programmers who should move to San Francisco and wear flowers in their hair.
I’ve been coding for over 25 years and have used every methodology in the books. I’ve worked as a contractor who was hired because I was a guru at using methodologies. Looking back it was all a huge waste of time. Those methodologies drug out projects for years making sure all the steps were followed. Great for me as a contractor. A waste of money for the client.
My salvation and enlightenment came when I went to work on a contract for a chip manufacturing company in Silicon Valley writing internal business applications that they used around the world. That company spends billions of dollars making products that only have an 18 month life span so everything they do is done quickly and accurately. If you do something there, you do it fast and with no errors – period.
The bottom line is: when I got my first assignment and started to write a system development plan, my manager shut down my word processor and told me start coding immediately. As someone entrenched in using methodologies, my head spun, I got light headed, the blood drained out of my brain, and I became a blubbering blank-brained child. How could I code without a spec – impossible! All I had were my notes from meeting with the client.
But now, after hiring on with that company and working for six years as a systems development manager and programmer, I’m happy to say I can crank out high quality software without wasting time on methodologies.
With that siad, though, here’s my methodology (the way we wrote code there): Sit down with your client and write down their needs and wants. Read it back to them and get a verbal agreement. Go code. Test as you code. Write quality code. Document your code interally – lots of comment lines. Test the code and deliver it to the customer with full confidence that you did a good job and it does what they need it to do. Go on to the next project – because if you’re a top notch programmer you won’t have to go back to fix the code in your previous projects.
It is possible. I hired dozens of those programmers who could quickly write quality code that never broke after it was delivered to the client – without using any methodologies other than taking personal responsibility to write quality code the first time.
You could buy the GOOS book. Growing Object Oriented Software – Guided by tests. I think the example there is not so trivial as the ones you see in the web.
And my experience differs from yours. For me TDD was the best practice I learned in many years. But as one said before, you have to stand with it for a period of time.
In my company we already have a metric where a new developer will change your mind about TDD. It is more or less 3 months with heavy pairing. We not lost any soul.
And many who left the company implemented TDD in your new jobs (TDD and ATDD).
In one case of a friend that is in other company, the upper managers went down to know why the project has a quality so better than the others. The only difference was that in that project they are doing ATDD/TDD.
Well, if working without TDD is good for you, then I respect. But I only wish that you give it a serious try, if you had a chance. Everyone I know (here at work) that called TDD is not good, it loss of time etc. They changed their minds after a period of serious try of the practice.
And I have arguments to each of the points that raised (consume more time because you have to write more code etc.) but I will not writ about it because google is your friend and the internet is plenty of argumentations about these points.
One particular useful place where doubts/questions could be placed is at the TDD group in yahoo.
abraços,
Josué.
I believe that test driven development is a very good software construction tool. It is not a silver bullet and, as with everything in life, it must be used appropriately. Recently I managed an engineering team responsible for a very complex client server application with over 2 mloc, written over the course of the last 20 years and used by customers all over the world. Only a couple of folks, out of a team of over 50 engineers, know how the thing was put together; even these folks had a hard time with changing the code; no one really know how the software worked! Adding new features or fixing bugs int his code was a nightmare; a tuck here broke something over there, etc, a night mare. This scenario improved quite a bit after we introduced TDD to validate functionality; it was not unit tests, rather functional tests. We started by writing tests to validate existing functionality; as these tests were written we noticed that everyone now had access to answers relative to the question: how does this work anyway; RTFT was the reply! These tests were harnessed into a body of automated tests that run every 4 hours or so; finding out what broke the software that quickly improved our ability to service our customers. Finally, we started to ask developers to write the tests that would validate the functionality they were about to implement and have them reviewed by their senior peers; these interactions increased the teams overall competency and improved our results. A good thing all around.
I use TDD like this:
I write code and use it. When I find a bug I write a test to duplicate the error in runtime. then I fix the error to make the test work. I call it Lazy Loaded TDD.
If you are a service company: don’t do TDD except if you are paid for that.
If you are a software company: DO TDD, as you have to lower the costs in the long term (New developpers training, bug fixing, evolution, etc.).
At my company we had argued about TDD and testing in general for a while. We decided to test the TDD theory. We took a complex piece of business logic and had one developer who knew and loved TDD and one who hated the idea of testing and had each of them develop the business logic.
The developer who didn’t like testing created a lot of the code, wrote println and assert statements to verify his assertions at various steps of the code, ran the code frequently and looked at the output. He repeated the process until he was confident that it was right.
The developer who did TDD coded an assertion into a test and then wrote code to fulfill the assertion. He then repeated the process, refactoring the code as he went until he was confident that it was right.
Both projects took about the same amount of time to complete. Both worked well and accomplished the intended task. The TDD code was only about 70% the length of the non-TDD code and the TDD code was cleaner and easier to read and understand. The TDD code also ran about 10% faster when profiled.
Lastly the tests were a reusable artifact that would allow us to make changes to the system later if the business rules changed.
As I see it, you can write code without TDD or other unit tests, but if you can do it in the same amount of time, get faster, cleaner code, and have a test suite to boot, why would you choose not to do it?
I do understand that learning TDD takes some time, but it sounds to me like it’s worth it.
I would love to think otherwise, but using TDD for one person projects (which does not need maintanence???) is a complete waste of time.
As someone pointed out earlier, it all depends on where you are using it. In a large organization where people are just exchange-able resources, it makes sense to ensure code base is not completely dependent on individuals and TDD plays an important role there.
If the tests are written correctly and maintained along with product code base, it is not possible for a developer to break any existing features while implementing a new feature or fixing a bug.
It is true that effort/time requirements are tripled. But TDD ensures the best case that a feature once accepted by the QA team shall not break.
To do TDD right takes at least a year or more. And it is not to replace “test” or “qa”. The test suit is not written just for catching the bugs when you make changes. It is just sweet side-effect of TDD. The real value is it force you to think what you are trying to do before you actually write the production code. With BDD (just TDD done in right) will force you to ask questions to customer to prevent redoing things just because you don’t know what customer want (you may think that the customer is stupid). And it will prevent you to write waste code, that is written because we, developers, think/guess that it is nice feature but customer never request.
But…. TDD is not easy. It is the mindset/thinking process change so it requires time, hard-work and of course, believe.
So.. don’t use TDD if the project’s success or failure doesn’t matter to you.
TDD is a discipline that requires quite a lot of practice to get right. The author did not, imho, spend enough time learning the technique to be able to make the claims that he does. At least he qualified his claims with “I haven’t done this long enough to know what I’m talking about.”
I have personally used TDD on a one man project in a larger company to produce the most solid piece of code that company ever produced. That product is now shipping, and is serving a great number of people. Without TDD, I believe that project would have been much more unstable, like the other projects created at that company all were. Management thought I could go faster without writing tests, but I assure you I would have spent much more time debugging had I not been doing TDD.
No, TDD is not a silver bullet. In some cases, it is too much work to set up a suitable environment of mocks to make much sense. Things like device drivers and other smallish pieces of glue code are not very suitable to TDD. I am discouraged to see how difficult it is for many people to learn. It took a lot of effort for me to learn as I had nobody to learn from other than a mailing list. But I do believe in the technique for many types of projects.
When I was younger, I could keep more things in my head simultaneously. As I’m now approaching 50, TDD is the “wisdom” that replaces the “strength” of my youth. For older programmers who are starting to feel the sharpness fade, I highly recommend TDD as a way of continuing to be highly productive without the mental gymnastics required to code with everything in your head.
The value of TDD IMHO is “executable documentation”. It makes old code easier to pick up and run with again. It is hard work. It takes discipline. Just as some programmers will never really grasp lamda functions, some will never catch the TDD fever. If you want to try it, do it on a new “green field” project where you have a year to learn the technique. TDD in a “legacy” environment is painful in the extreme. Or, if you have others to learn from in a pairing environment, take advantage of that to learn the technique.
Once you get that first failing test that you weren’t expecting, you have that light bulb moment of “When would I have found THAT ONE!!” and you realize you are test infected. It saves an inordinate amount of time!
TDD and Unit tests are not exactly the same thing. I do not really believe in TDD. It is much better to start of with a proper design that has been optimized for testing. In one fell swoop this also implies a better quality of your code. But still: it is then still pretty good practice to tests your units. But honestly: just designing the testcases already brings to light alot of thinking errors in the design and code, in my experience.
[...] Why I don’t use TDD or BDD http://blog.kangasbros.fi/?p=30 [...]