LectroTest

What is it?

LectroTest is an automatic, specification-based testing tool for Perl.

Here is how it works. You declare general properties that specify how your software ought to behave, and LectroTest checks the properties by conducting automated random trials.

LectroTest was inspired by Koen Claessen and John Hughes's QuickCheck for the Haskell programming language.

More information

• LectroTest/FAQs – answers to frequently asked questions
• Talk - Free Unit Tests In Perl with LectroTest – these slides from my Perl Mongers talk make a great introduction to LectroTest and specification-based testing.

What's new

Get your LectroTest gear! (2006-06-26) You asked for it, you got it: The LectroTest Emporium! Get t-shirts, hats, mouse pads, and other goofy stuff – all adorned with the fiercely metallic yet lovable LectroTest Robot.

0.3600 released: minor tweaks (2007-08-30). Minor typo fixes in the documentation. Improvements to the internal test suite for the generator library that (for almost all users) will be invisible.

0.3500 released: failure recording and playback added! (2006-05-31). Added failure recording and playback, which are great for building regression-testing suites and for recording failures in the wild. Both Test::LectroTest and Test::LectroTest::Compat take new options that let you tell LectroTest to record failure-causing test cases to files and to play them back later. See Test::LectroTest::RegressionTesting in the documentation for more on this handy new feature.

0.3400 released: minor enhancements (2006-01-18). The test controller's dump method now returns the dumped value as its result so that the method can be used to record values in passing. Added Pod and Pod-coverage tests; tweaked the code to satisfy them.

0.3300 released: minor bug fix. (2005-07-20). Fixed a bug that prevented Test::LectroTest from being listed as a dependency in Makefile.PL files (thanks Steffen Müller). Minor doc improvements.

0.3200 released: New: notes and value dumps. (2005-03-03) Now the test controller lets you attach notes to trials so that if the trial should fail, your notes will appear with the counterexample as a debugging aid. See the docs on Test::LectroTest::TestRunner for more.

Mailing lists. (2005-02-20) I have created two new mailing lists, one for announcements and one for general discussion. See LectroTest/Lists for sign-up and other information.

0.3100 released: Preparing for 1.0. (2005-02-19) After the 0.3XXX series of releases, I am going to release version 1.0. Think of 0.3100 as the first 1.0 pre-release. Changes include full unit-test coverage and minor enhancements in the stand-alone test harness.

0.2010 released: Test::Builder compatibility! (2005-02-16) Now you can mix Test::LectroTest and other Test::X modules such as Test::Simple and Test::More! Test::LectroTest::Compat provides the compatibility magic.

0.2008 released: Now works with older Perls. (2005-02-05) I removed some dependencies on newer Perl features to make LectroTest backward compatible with older Perls. I tested on 5.6.1 (i386-linux; Red Hat 7.2). Other improvements:

• Paste can now be used to join elements of generated lists, which are "flattened" before joining occurs.
• Minor documentation improvements.

Fedora / Red Hat RPMS. (2004-10-16) Now you can install LectroTest with a single command and take advantage of your operating system's built-in package management architecture.

Downloads

• Tarball. Download the code from the "files" box, which should be above and to the right.
• CPAN. Or download from Test::LectroTest on CPAN.
• RPMS. You can also download Fedora Core RPMs from the RPMs section of the site. If you're running a different RPM-based distribution, you can download the .src.rpm and rebuild it for your distro like so:

rpmbuild --rebuild perl-Test-LectroTest-vesrion.src.rpm

See Rebuilding RPMs for more help.

Documentation

Here is a summary of your documentation options:

• Download beautiful, typeset documentation in PDF format
• Browse the Test::LectroTest docs online at CPAN
• Download the slides from introductory talk about LectroTest to the Pittsburgh Perl Mongers: Talk - Free Unit Tests In Perl with LectroTest
• Doanload the slides from my talk on LectroTest tips: Talks/Testing Tips with LectroTest

The idea

Unit testing is like finding needles in a haystack. The haystack is your code's behavior. The needles are erroneous behavior.

Traditional unit testing practices require us to point out to the computer each and every piece of hay in the haystack that we want to check for errors. Hey, computer, check that piece of hay, please. Now this one. Now this one, too. And over here, thank you. And, yes, that one, too... That's a tedious process that grows old fast.

LectroTesting is different. Rather than pointing out the individual pieces of hay, we merely point out the shape of the haystack. We also tell the computer what a needle looks like, in general, so it will know if it sees one along the way.

The computer takes care of the rest. It hunts through the haystack until it finds a needle or we tell it to stop. If it finds a needle, it outputs a counterexample – a specification of the inputs that "broke" our code. We can plug the counterexample into our code to see what went wrong, and we can add it to a list of regression tests for future testing.

That's the idea, in a nutshell. A much better explanation can be found in the slides for my talk on LectroTest.

What does "LectroTesting" look like?

It's simple. Just write a Perl program that uses Test::LectroTest and defines some properties to check. Then run it. For example, here is a property that defines the expected behavior of a function angdiff:

#!/usr/bin/perl -w
use AngularDifference;
use Test::LectroTest;
Property {
    ##[ a <- Int, n <- Int, diff <- Int(range=>[-180,180]) ]##
    my $b = $a + 360 * $n + $diff;
    angdiff($a, $b) == abs($diff);
}, name => "definition via inverse";

To check the property, run the program:

$ perl angdiff.t  # run the program that holds our property
1..1
not ok 1 - 'definition via inverse' falsified in 9 attempts
# Counterexample:
# $a = -1;
# $diff = 1;
# $n = -1;

That's it! LectroTest found a problem and emitted a counterexample to show that our property's claims do not hold. (For a more thorough explanation, see my slides from the Talk - Free Unit Tests In Perl with LectroTest.)

Background blurb

Writing unit tests is an effective coding practice that reduces development time while raising our confidence in the software we create. But, if we're brutally honest, the work is often tedious and time consuming, tempting us to skimp on test cases. Is there anything we can do to reduce the pain while keeping the gain?

Yes! With QuickCheck for the Haskell programming language, Koen Claessen and John Hughes demonstrated that not only can your computer run your test cases, but also it can create them for you!

The first time I used QuickCheck for my Haskell programs, I was sold. It caught problems I never would have thought to look for, let alone test for, and it did so elegantly.

When working in Perl, I missed QuickCheck. In August 2004, I finally got around to doing something about it. The result is LectroTest, a horribly named, automatic, specification-based testing tool for Perl.