I have dabbled in dozens of open source projects, which might extend as far as filing bugs for a few handfuls. But I have rarely been motivated enough to dive in and figure out what was going on and add a new feature or fix a bug that was annoying me. I guess in the case of py.test I use it so heavily at work that I was “itchy” enough to really want to “scratch” it.
The problem – it is very easy to parametrize tests in py.test (feed different inputs into the same test), which is very useful for test isolation (ideally one assert statement per test) without heaps of repeated code. That’s all great, but there is no easy way to mix passing tests and xfail tests. Xfail means “expected to fail”, and this is a powerful way of writing “demonstration tests” for bugs that you are aware of but haven’t yet fixed. Yep, test driven development!
One way to do it could be to simply copy the test and have a version of it just for xfail cases. However if our test function’s contents are more complicated, this is obviously going to be bad repetition liable to fall out of date. With my patch you can now apply the marker directly to the tuple which has the parametrized values:
Incidentally you can apply any marker, not just xfail. At work we use marks to link tests to issues in our issue tracker (essentialy test metadata), and this would work here too.
As well as enjoying using py.test I like the dev community too. The founder Holger Krekel is undoubtedly a very clever guy (he founded and co-developed PyPy) and a good project leader, exactly what you would want in a BDFL. If you’re not using py.test for testing in Python – why not? :)