LectroTest/FAQs

How does specification-based testing differ from traditional unit testing?

In traditional unit testing, you must write individual test cases that together can be used to provide evidence for the correctness (or incorrectness) of your software. In specification-based testing, you instead declare properties that define the correct behavior of your software, and the computer – not you – writes the test cases automatically.

For example, let us say that we want to test a function sqrt that (if working properly) computes the square root of its sole argument. Using Perl's Test::More, which supports case-based testing, we might create the following suite of tests:

use Test::More tests => 4;
is( sqrt(0), 0, "sqrt(0) == 0" );
is( sqrt(1), 1, "sqrt(1) == 1" );
is( sqrt(4), 2, "sqrt(4) == 2" );
is( sqrt(9), 3, "sqrt(9) == 3" );

We defined four test cases by hand. All of them implicitly test whether sqrt satisfies the definition of square root, namely that the square root of x squared is equal to x, for all x >= 0.

LectroTest lets us declare this fundamental property directly:

use Test::LectroTest;
Property {
   ##[ x <- Float(range=>[0,1_000_000]) ]##
   sqrt( $x * $x ) == $x;
}, name => "sqrt satisfies defn of square root";

Our property says, "For all floating-point numbers x in the range 0–1,000,000, sqrt(x*x) equals x." (We use the range to approximate the set of all non-negative numbers.)

Can I take advantage of LectroTest in my Test::Simple- and Test::More-style test plans?

Yes. Release 0.20_10 of Test::LectroTest introduces Test::LectroTest::Compat, which makes LectroTest compatible with any testing module based on Test::Builder, including as Test::Simple and Test::More. The module defines the holds assertion, which is used to inject property checks into Test::Simple-style test plans. Just define your LectroTest properties as usual and then assert that they hold via holds to include them in your test plan.

Example usage:

use MyModule;  # contains code we want to test
use Test::More tests => 2;
use Test::LectroTest::Compat;
# property specs can now use Test::Builder-based
# tests such as Test::More's cmp_ok()
my $prop_nonnegative = Property {
    ##[ x <- Int, y <- Int ]##
    cmp_ok(MyModule::my_function( $x, $y ), '>=', 0);
}, name => "my_function output is non-negative" ;
# and we can now check whether properties hold
# as Test::Builder-style tests that integrate
# with other T::B tests
holds( $prop_nonnegative );   # test whether prop holds
cmp_ok( 0, '<', 1, "trivial 0<1 test" );  # a "normal" test