Saturday, June 7, 2008

Generating lots of little test cases

When writing code that is largely algorithmic, I find I end up writing specs that sit in a loop repeating the same operations over a set of data.



This works well enough but it has the downside that the tests abort as soon as a single failing case is detected which can lead to vicious cycles of fixing one case only to find you've broken another.



The solution is of course to break out each expectation--inputs and expected outputs--so they are all run and reported individually. Doing so by hand however, is tedious to say the least so why not generate them on the fly instead:



describe Fibonacci do
[[0, 0], [1, 1], [2, 1], [3, 2], [4, 3], [5, 5], [6, 8]].each do |input, output|
it "should generate #{output} for #{input}" do
Fibonacci.calculate(input).should == output
end
end
end



This is such a elegant solution I'm not sure why it only just occurred to me.



6 comments:

dr_bonzo said...

Wow, such a simple solution, thanks for posting it, It's really helpfull

michael neale said...

you should check out quickcheck in haskell. It takes it even further- let mathematics generate the test data for you.

Kristian Domagala said...

Following on from Michael's comment, if you're interested in QuickCheck but want something for Java or Scala, take a look at Reductio at http://reductiotest.org
(Disclaimer: I work for Workingmouse, the company behind Reductio)

HovPod said...

heh, I hoped that this blog would in fact be writing about hovercrafts of the type that hover :)

Stephan said...

Hm, being a tester, I have to say that this 'news' is surprising me a little bit.
It seemed to be so obvious to me (up to now, that is) to separate (test) data from )test) code, that I've never even thought about talking about it.
But then, may be I'm doing test automation for so long now, that my assumptions go wrong sometimes. Apart from that, in case the data is needed in more than one test case/spec, I'd put the data part into some kind of set-up method (assuming that the spec framework (guessing it's RSpec) supports it).
Anyway, thanks for this eye-opener.

Simon Harris said...

With all due respect, I think you've missed the point--perhaps in part due to an unfamiliarity with Rspec? What you've described is exactly what most of us already do. The additional benefits conferred by the approach listed is that each piece of data gets its own test case. That is, the results will contain one test for each expectation rather than a single test that iterates over the data.
Then again, perhaps you already do this and I'm the one that has missed the point :)

Post a Comment