.\" -*- mode: troff; coding: utf-8 -*-
.\" Automatically generated by Pod::Man v6.0.2 (Pod::Simple 3.48)
.\"
.\" Standard preamble:
.\" ========================================================================
.de Sp \" Vertical space (when we can't use .PP)
.if t .sp .5v
.if n .sp
..
.de Vb \" Begin verbatim text
.ft CW
.nf
.ne \\$1
..
.de Ve \" End verbatim text
.ft R
.fi
..
.\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>.
.ie n \{\
.    ds C` ""
.    ds C' ""
'br\}
.el\{\
.    ds C`
.    ds C'
'br\}
.\"
.\" Escape single quotes in literal strings from groff's Unicode transform.
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\"
.\" If the F register is >0, we'll generate index entries on stderr for
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
.\" entries marked with X<> in POD.  Of course, you'll have to process the
.\" output yourself in some meaningful fashion.
.\"
.\" Avoid warning from groff about undefined register 'F'.
.de IX
..
.nr rF 0
.if \n(.g .if rF .nr rF 1
.if (\n(rF:(\n(.g==0)) \{\
.    if \nF \{\
.        de IX
.        tm Index:\\$1\t\\n%\t"\\$2"
..
.        if !\nF==2 \{\
.            nr % 0
.            nr F 2
.        \}
.    \}
.\}
.rr rF
.\"
.\" Required to disable full justification in groff 1.23.0.
.if n .ds AD l
.\" ========================================================================
.\"
.IX Title "Test::FAQ 3"
.TH Test::FAQ 3 2026-10-26 "perl v5.44.0" "User Contributed Perl Documentation"
.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
.nh
.SH NAME
Test::FAQ \- Frequently Asked Questions about testing with Perl
.SH DESCRIPTION
.IX Header "DESCRIPTION"
Frequently Asked Questions about testing in general and specific
issues with Perl.
.SS "Is there any tutorial on testing?"
.IX Subsection "Is there any tutorial on testing?"
Test::Tutorial
.SS "Are there any modules for testing?"
.IX Subsection "Are there any modules for testing?"
A whole bunch.  Start with Test::Simple, then move on to Test::More.
.PP
Then go on to <https://metacpan.org/> and search for "Test".
.SS "Are there any modules for testing web pages/CGI programs?"
.IX Subsection "Are there any modules for testing web pages/CGI programs?"
Test::WWW::Mechanize, Test::WWW::Selenium
.SS "Are there any modules for testing external programs?"
.IX Subsection "Are there any modules for testing external programs?"
Test::Cmd
.SS "Can you do xUnit/JUnit style testing in Perl?"
.IX Subsection "Can you do xUnit/JUnit style testing in Perl?"
Yes, Test::Class allows you to write test methods while continuing to
use all the usual CPAN testing modules.  It is the best and most
perlish way to do xUnit style testing.
.PP
Test::Unit is a more direct port of xUnit to Perl, but it does not use
the Perl conventions and does not play well with other CPAN testing
modules.  As of this writing, it is abandoned.  \fBDo not use\fR.
.PP
The Test::Inline (aka Pod::Tests) is worth mentioning as it allows you to
put tests into the POD in the same file as the code.
.SS "How do I test my module is backwards/forwards compatible?"
.IX Subsection "How do I test my module is backwards/forwards compatible?"
First, install a bunch of perls of commonly used versions.  At the
moment, you could try these
.PP
.Vb 4
\&    5.7.2
\&    5.6.1
\&    5.005_03
\&    5.004_05
.Ve
.PP
if you\*(Aqre feeling brave, you might want to have on hand these
.PP
.Vb 4
\&    bleadperl
\&    5.6.0
\&    5.004_04
\&    5.004
.Ve
.PP
going back beyond 5.003 is probably beyond the call of duty.
.PP
You can then add something like this to your \fIMakefile.PL\fR.  It
overrides the ExtUtils::MakeMaker \f(CWtest_via_harness()\fR method to run the tests
against several different versions of Perl.
.PP
.Vb 4
\&    # If PERL_TEST_ALL is set, run "make test" against 
\&    # other perls as well as the current perl.
\&    {
\&        package MY;
\&
\&        sub test_via_harness {
\&            my($self, $orig_perl, $tests) = @_;
\&
\&            # names of your other perl binaries.
\&            my @other_perls = qw(perl5.004_05 perl5.005_03 perl5.7.2);
\&
\&            my @perls = ($orig_perl);
\&            push @perls, @other_perls if $ENV{PERL_TEST_ALL};
\&
\&            my $out;
\&            foreach my $perl (@perls) {
\&                $out .= $self\->SUPER::test_via_harness($perl, $tests);
\&            }
\&
\&            return $out;
\&        }
\&    }
.Ve
.PP
and re\-run your \fIMakefile.PL\fR with the \f(CW\*(C`PERL_TEST_ALL\*(C'\fR environment
variable set
.PP
.Vb 1
\&    PERL_TEST_ALL=1 perl Makefile.PL
.Ve
.PP
now \f(CW\*(C`make test\*(C'\fR will run against each of your other perls.
.SS "If I\*(Aqm testing Foo::Bar, where do I put tests for Foo::Bar::Baz?"
.IX Subsection "If I'm testing Foo::Bar, where do I put tests for Foo::Bar::Baz?"
.SS "How do I know when my tests are good enough?"
.IX Subsection "How do I know when my tests are good enough?"
A: Use tools for measuring the code coverage of your tests, e.g. how many of
your source code lines/subs/expressions/paths are executed (aka covered) by
the test suite. The more, the better, of course, although you may not
be able achieve 100%. If your testsuite covers under 100%, then
the rest of your code is, basically, untested. Which means it may work in
surprising ways (e.g. doesn\*(Aqt do things like they are intended or
documented), have bugs (e.g. return wrong results) or it may not work at
all.
.SS "How do I measure the coverage of my test suite?"
.IX Subsection "How do I measure the coverage of my test suite?"
Devel::Cover
.SS "How do I get tests to run in a certain order?"
.IX Subsection "How do I get tests to run in a certain order?"
Tests run in alphabetical order, so simply name your test files in the order
you want them to run.  Numbering your test files works, too.
.PP
.Vb 3
\&    t/00_compile.t
\&    t/01_config.t
\&    t/zz_teardown.t
.Ve
.PP
0 runs first, z runs last.
.PP
To achieve a specific order, try Test::Manifest.
.PP
Typically you do \fBnot\fR want your tests to require being run in a
certain order, but it can be useful to do a compile check first or to
run the tests on a very basic module before everything else.  This
gives you early information if a basic module fails which will bring
everything else down.
.PP
Another use is if you have a suite wide setup/teardown, such as
creating and delete a large test database, which may be too
expensive to do for every test.
.PP
We recommend \fBagainst\fR numbering every test file.  For most files
this ordering will be arbitrary and the leading number obscures the
real name of the file.  See "What should I name my test files?" for
more information.
.SS "What should I name my tests?"
.IX Subsection "What should I name my tests?"
.SS "What should I name my test files?"
.IX Subsection "What should I name my test files?"
A test filename serves three purposes:
.PP
Most importantly, it serves to identify what is being tested.  Each
test file should test a clear piece of functionality.  This could be
at single class, a single method, even a single bug.
.PP
The order in which tests are run is usually dictated by the filename.
See "How do I get tests to run in a certain order?" for details.
.PP
Finally, the grouping of tests into common bits of functionality can
be achieved by directory and filenames.  For example, all the tests
for Test::Builder are in the \fIt/Builder/\fR directory.
.PP
As an example, \fIt/Builder/reset.t\fR contains the tests for
\&\f(CW\*(C`Test::Builder\->reset\*(C'\fR.  \fIt/00compile.t\fR checks that everything
compiles, and it will run first.  \fIt/dont_overwrite_die_handler.t\fR
checks that we don\*(Aqt overwrite the \f(CW$SIG{_\|_DIE_\|_}\fR handler.
.SS "How do I deal with tests that sometimes pass and sometimes fail?"
.IX Subsection "How do I deal with tests that sometimes pass and sometimes fail?"
.SS "How do I test with a database/network/server that the user may or may not have?"
.IX Subsection "How do I test with a database/network/server that the user may or may not have?"
.SS "What\*(Aqs a good way to test lists?"
.IX Subsection "What's a good way to test lists?"
\&\f(CWis_deeply()\fR from Test::More as well as Test::Deep.
.SS "Is there such a thing as untestable code?"
.IX Subsection "Is there such a thing as untestable code?"
There\*(Aqs always compile/export checks.
.PP
Code must be written with testability in mind.  Separation of form and
functionality.
.SS "What do I do when I can\*(Aqt make the code do the same thing twice?"
.IX Subsection "What do I do when I can't make the code do the same thing twice?"
Force it to do the same thing twice.
.PP
Even a random number generator can be tested.
.SS "How do I test a GUI?"
.IX Subsection "How do I test a GUI?"
.SS "How do I test an image generator?"
.IX Subsection "How do I test an image generator?"
.SS "How do I test that my code handles failures gracefully?"
.IX Subsection "How do I test that my code handles failures gracefully?"
.SS "How do I check the right warnings are issued?"
.IX Subsection "How do I check the right warnings are issued?"
Test::Warn
.SS "How do I test code that prints?"
.IX Subsection "How do I test code that prints?"
Test::Output
.SS "I want to test that my code dies when I do X"
.IX Subsection "I want to test that my code dies when I do X"
Test::Exception
.SS "I want to print out more diagnostic info on failure."
.IX Subsection "I want to print out more diagnostic info on failure."
\&\f(CW\*(C`ok(...) || diag "...";\*(C'\fR
.SS "How can I simulate failures to make sure that my code does the Right Thing in the face of them?"
.IX Subsection "How can I simulate failures to make sure that my code does the Right Thing in the face of them?"
.SS "Why use an \fBok()\fP function?"
.IX Subsection "Why use an ok() function?"
.Vb 6
\&    On Tue, Aug 28, 2001 at 02:12:46PM +0100, Robin Houston wrote:
\&    > Michael Schwern wrote:
\&    > > Ah HA!  I\*(Aqve been wondering why nobody ever thinks to write a simple
\&    > > ok() function for their tests!  perlhack has bad testing advice.
\&    > 
\&    > Could you explain the advantage of having a "simple ok() function"?
.Ve
.PP
Because writing:
.PP
.Vb 2
\&    print "not " unless some thing worked;
\&    print "ok $test\en";  $test++;
.Ve
.PP
gets rapidly annoying.  This is why we made up subroutines in the
first place.  It also looks like hell and obscures the real purpose.
.PP
Besides, that will cause problems on VMS.
.PP
.Vb 3
\&    > As somebody who has spent many painful hours debugging test failures,
\&    > I\*(Aqm intimately familiar with the _disadvantages_. When you run the
\&    > test, you know that "test 113 failed". That\*(Aqs all you know, in general.
.Ve
.PP
Second advantage is you can easily upgrade the \f(CWok()\fR function to fix
this, either by slapping this line in:
.PP
.Vb 1
\&        printf "# Failed test at line %d\en", (caller)[2];
.Ve
.PP
or simply junking the whole thing and switching to Test::Simple or
Test::More, which does all sorts of nice diagnostics\-on\-failure for
you.  Its \f(CWok()\fR function is backwards compatible with the above.
.PP
There\*(Aqs some issues with using Test::Simple to test really basic Perl
functionality, you have to choose on a per test basis.  Since
Test::Simple doesn\*(Aqt use \f(CWpack()\fR, it\*(Aqs safe for \fIt/op/pack.t\fR to use
Test::Simple.  I just didn\*(Aqt want to make the perlhack patching
example too complicated.
.SS "Dummy Mode"
.IX Subsection "Dummy Mode"
> One compromise would be to use a test\-generating script, which allows
> the tests to be structured simply and _generates_ the actual test
> code. One could then grep the generated test script to locate the
> failing code.
.PP
This is a very interesting, and very common, response to the problem.
I\*(Aqm going to make some observations about reactions to testing,
they\*(Aqre not specific to you.
.PP
If you\*(Aqve ever read the Bastard Operator From Hell series, you\*(Aqll
recall the Dummy Mode.
.PP
.Vb 4
\&    The words "power surging" and "drivers" have got her.  People hear
\&    words like that and go into Dummy Mode and do ANYTHING you say.  I
\&    could tell her to run naked across campus with a powercord rammed
\&    up her backside and she\*(Aqd probably do it...  Hmmm...
.Ve
.PP
There seems to be a Dummy Mode with respect to testing.  An otherwise competent
person goes to write a test and they suddenly forget all basic
programming practice.
.PP
The reasons for using an \f(CWok()\fR function above are the same reasons to
use functions in general, we should all know them.  We\*(Aqd laugh our
heads off at code that repeated as much as your average test does.
These are newbie mistakes.
.PP
And the normal \*(Aqcan do\*(Aq flair seems to disappear.  I know Robin.  I
*know* that in any other situation he would have come up with the
\&\f(CWcaller()\fR trick in about 15 seconds flat.  Instead weird, elaborate,
inelegant hacks are thought up to solve the simplest problems.
.PP
I guess there are certain programming idioms that are foreign enough
to throw your brain into reverse if you\*(Aqre not ready for them.  Like
trying to think in Lisp, for example.  Or being presented with OO for
the first time.  I guess writing test is one of those.
.SS "How do I use Test::More without depending on it?"
.IX Subsection "How do I use Test::More without depending on it?"
Install Test::More into \fIt/lib\fR under your source directory.  Then in your tests
say \f(CW\*(C`use lib \*(Aqt/lib\*(Aq\*(C'\fR.
.SS "How do I deal with threads and forking?"
.IX Subsection "How do I deal with threads and forking?"
.SS "Why do I need more than ok?"
.IX Subsection "Why do I need more than ok?"
Since every test can be reduced to checking if a statement is true,
\&\f(CWok()\fR can test everything.  But \f(CWok()\fR doesn\*(Aqt tell you why the test
failed.  For that you need to tell the test more... which is why 
you need Test::More.
.PP
.Vb 1
\&    ok $pirate\->name eq "Roberts", "person\*(Aqs name";
\&
\&    not ok 1 \- person\*(Aqs name
\&    # Failed test at pirates.t line 23.
.Ve
.PP
If the above fails, you don\*(Aqt know what \f(CW\*(C`$person\->name\*(C'\fR returned.
You have to go in and add a \f(CW\*(C`diag\*(C'\fR call.  This is time consuming.  If
it\*(Aqs a heisenbug, it might not fail again!  If it\*(Aqs a user reporting a
test failure, they might not be bothered to hack the tests to give you
more information.
.PP
.Vb 1
\&    is $person\->name, "Roberts", "person\*(Aqs name";
\&
\&    not ok 1 \- person\*(Aqs name
\&    # Failed test at pirates.t line 23.
\&    #        got: \*(AqWesley\*(Aq
\&    #   expected: \*(AqRoberts\*(Aq
.Ve
.PP
Using \f(CW\*(C`is\*(C'\fR from Test::More you now know what value you got and
what value you expected.
.PP
The most useful functions in Test::More are \f(CWis()\fR, \f(CWlike()\fR and \f(CWis_deeply()\fR.
.ie n .SS "What\*(Aqs wrong with ""print $test ? ""ok"" : ""not ok""""?"
.el .SS "What\*(Aqs wrong with \f(CWprint $test ? ""ok"" : ""not ok""\fP?"
.IX Subsection "What's wrong with print $test ? ""ok"" : ""not ok""?"
.SS "How do I check for an infinite loop?"
.IX Subsection "How do I check for an infinite loop?"
.Vb 10
\&    On Mon, Mar 18, 2002 at 03:57:55AM \-0500, Mark\-Jason Dominus wrote:
\&    > 
\&    > Michael The Schwern <schwern@pobox.com> says:
\&    > > Use alarm and skip the test if $Config{d_alarm} is false (see
\&    > > t/op/alarm.t for an example).  If you think the infinite loop is due
\&    > > to a programming glitch, as opposed to a cross\-platform issue, this
\&    > > will be enough.
\&    > 
\&    > Thanks very much!
\&    >
.Ve
.SS "How can I check that flock works?"
.IX Subsection "How can I check that flock works?"
.SS "How do I use the comparison functions of a testing module without it being a test?"
.IX Subsection "How do I use the comparison functions of a testing module without it being a test?"
Any testing function based on Test::Builder, most are, can be quieted so it does
not do any testing.  It simply returns true or false.  Use the following code...
.PP
.Vb 1
\&    use Test::More;     # or any testing module
\&    
\&    use Test::Builder;
\&    use File::Spec;
\&
\&    # Get the internal Test::Builder object
\&    my $tb = Test::Builder\->new;
\&
\&    $tb\->plan("no_plan");
\&    
\&    # Keep Test::Builder from displaying anything
\&    $tb\->no_diag(1);
\&    $tb\->no_ending(1);
\&    $tb\->no_header(1);
\&    $tb\->output( File::Spec\->devnull );
\&
\&    # Now you can use the testing function.
\&    print is_deeply( "foo", "bar" ) ? "Yes" : "No";
.Ve