Hello listers,

Navigating in FPC source codes, I found fpcunit. It's really wonderful. With the tTestCase class, we can generate a report with the success or the failure of our tests.

However, only observing the source code, it's not easy to comprehend how it works. I found a single parcial example, showing the usage of AssertEqual, but it was not enough to understand.

I have never used something like that. I know other languages have similar features, but I have never programed tests in other languages.

Is there documentation on the usage of fpcunit? What is the best way to study it?

regards,

Luciano

2011/12/16 luciano de souza :

Google found this (old) article:
http://www.freepascal.org/olddocs-html/fpcunit.pdf

Vincent

It is old, but still correct.

I will see about refreshing it, and adding it in the FPC documentation.

Michael.

Perhaps becouse I am not an experienced programer, but the code sounds a little complex. However, the effort will bring large bennefits.

Thank you very much for the article. It will help me a lot.

The article is very good, but it does not focuse on all aspects.

I have tried to create my first test example, but it didn't work. The compilation is successful and the program is executed without errors. However, I received no messages, informing the test has failed. Actually, the test should be failed.

This example is seemingly equivalent the minimum example supplied by the article.

As recommended, I only add the unit "test" in the clause "uses", so the test was expected to run. Does someone know what is wrong?

First of all, a unit with a routine to be tested:

unit numbers;
{$mode objfpc}{$h+}
interface
uses
Sysutils;
function calculate(a, b: integer):integer;
implementation
function calculate(a, b: integer):integer;
begin
result := a + b;
end;
INITIALIZATION
END.

Now, my testcase class:

unit test;
{$mode objfpc}{$h+}
interface
numbers, fpcunit, testregistry;
uses
type
TBugTest = class(TTestCase)
published
procedure calculate;
end;
implementation
procedure TBugTest.calculate;
begin
AssertEquals('The calculation is not done', 8, numbers.calculate(5, 4));
end;
INITIALIZATION
RegisterTest(TBugTest);
END.

Now, the runer:

program apptest;
{$mode objfpc}{$h+}
uses
test;
begin
end.

You see no test results, because the program file of your test application is missing some essential things.

If you are using lazarus, simply create a new testing application:

First, make sure that package fpcunitide is installed.

Then, in the 'File-New' dialog, your testproject should be of type

'FPCUNit console test application'
or
'FPCUnit test application'

This will install a test decorator which prints out the result. Then add your test unit to this application.

If you run the console test application on the command-line, specify options '--format=plain' and '--all'

The graphical application will show test results in a tree.

Michael.

Michael,

I have a meaningful peculiarity: I am blind and I use Freepascal only in the console interface, reading all the output by means of my screen reader. Lazarus and Freepascal console IDE are both inaccessible. But, fortunately, for me, it's easy to call the compiler passing parameters at the command line.

I can compile using callings like:
fpc test.pas
fpc -XX test.pas
fpc test.pas -Fu./pas -FE./exe

I really thank Freepascal developers. The reachness of its command line allow me to access the features of the compiler.

For these reasons, it's not possible to open Lazarus and compile the project using its interface. My screen reader has an awful response in this environment.

After reading your message, I ask for a help. A friend told me fpcunit depends on FPCUnitConsole... Well, I don't remember exactly the name, but it's a Lazarus package.

Observe I need to compile in the console. If it's possible to run the executable, but it's not possible to compile in the command line, I won't be able to use FPCUnit.

You said that I need to use '--format=plain' '--all'. I tried: fpc apptest.pas --format=plain -all
And also:
fpc apptest
apptest --format=plain -alll
But, in both cases, I got no result.
I don't know how to embody this parameters in my compilation tasks.

So I ask you: Is it possible to use FPCUnit only in the command line, it means, compiling and executing in console?

The idea of FPCUnit is really wonderful. I really would like to use it, but Lazarus dependences can complicate a lot.

It is possible to do this.

The necessary functionality is contained in the consoletestrunner unit.

I have attached the 'consoletestrunner' unit in a zip file. It is part of Lazarus, but can be used independent of lazarus; there are no dependencies on the LCL.

Copy it to the directory where your source code is.

Then use the following test program:

-----------------------------------------------------------------------
program runtests;
{$mode objfpc}{$H+}
uses
   Classes, consoletestrunner;
var
   App: TTestRunner;
begin
   App := TTestRunner.Create(nil);
   App.Initialize;
   App.Title := 'FPCUnit Console runner.';
   App.Run;
   App.Free;
end.
-----------------------------------------------------------------------

And then add the units containing your tests to the uses clause. Compile this program.

After that, the commandline options for this program will work as I explained.

Michael.

Luciano de Souza wrote

I can compile using callings like: fpc test.pas fpc -XX test.pas fpc test.pas -Fu./pas -FE./exe

You can compile Lazarus projects in the command line using lazbuild which comes with Lazarus, like this:

lazbuild project.lpi

It will build all required projects and otherwise do everything the IDE would do to build the project. The only catch might be if the package requires a package that your Lazarus doesn't know. I don't know which configuration file would need to be modified to tell Lazarus where a package is without opening the IDE.

lazbuild path/packagename.lpk

Mattias

You would believe if I told you I could not compile yet?

It's not lack of effort, believe me. I really can't understand what is failing.

See my compilation script:

@echo off
set compilerpath="c:\lazarus\fpc\2.4.4\bin\i386-win32\fpc.exe" set unitpath="-Fuc:\lazarus\fpc\units\i386-win32\* -Fuc:\lazarus\units\i386-win32\* -Fuc:\lazarus\lcl\units\i386-win32\*" set debugerpath = ""c:\lazarus\mingw\bin\gdb.exe"
Set NormalParam="-XX"
set DebugParam="-g"

if "%1"=="" goto end else goto start

:start
if "%1"=="-n" goto normalmode
if "%1"=="-d" goto debugmode
if %1=="-g" goto debuger
goto end

:normalmode
%compilerpath% %2 %NormalParam% %unitpath%
goto end

:debugmode
%compilerpath% %2 %DebugParam% %unitpath%
goto end

:debuger
gdb %1
goto end

:end

When I do: "make -n {filename}", I got an error. "fileutil is not found".

But "fileutil" is in c:\lazarus\lcl\units\i386-win32" and this path is already added.

Actually, I added this path in fpc.cfg.

Lazbuild seems to simplify compilation using Lazarus. But as a parameter, it receives a LPI file. However, lpi files are created in Lazarus interface. The lack of accessibility of Lazarus is so big that even this task I could not do at Lazarus interface.

If LPI files are really necessary, I must create then manually.

Michael Van Canneyt wrote

The necessary functionality is contained in the consoletestrunner unit.

I have long ago said this must move to the FPC repository, not live in the Lazarus repository.

Luciano de Souza wrote

The idea of FPCUnit is really wonderful. I really would like to use it, but Lazarus dependencies can complicate a lot.

I have been a long time user of FPCUnit and even helped to maintain and improve it over the years. It has some design flaws and limitations though, so is good for smaller projects and less complicated test suites.

For a more feature complete testing suite, I would recommend the FPTest (Free Pascal Testing Framework) project instead. I am the current maintainer of FPTest - which is a fork/continuation of the DUnit2 testing framework created by the late Peter McNabb, but specifically tailored for the Free Pascal compiler. FPTest fixes all of the problems I experienced with FPCUnit. It supports Text and GUI (only fpGUI at the moment, but a LCL UI will follow shortly) test projects out of the box. No dependencies on any IDE (Lazarus IDE or otherwise), and no GUI toolkits are required either. The GUI UI does make the usage (and advanced features) much easier though - but it is by no means a requirement. The Text runner can do everything the GUI runner can.

FPTest has a many more features than FPCUnit: multiple projects support, extended CheckXXX calls (known as AssertXXX in FPCUnit), improved error reporting, warning support, huge self-test testing frameworking, improved decorator tests (which are flawed in FPCUnit), improved Setup/TearDown and SetupOnce/TearDownOnce support etc.

Here is a console sample testing project

---------------------[ project1.pas ]------------------------
program project1;
{$mode objfpc}{$H+}

uses
Classes,
TextTestRunner,

  sample_tests;

begin
  // Register all tests
  sample_tests.RegisterTests;
  RunRegisteredTests;
end.
---------------------------------[ end ]----------------------------------

and the unit containing the actual tests...

-------------------------[ sample_tests.pas ]----------------------
unit sample_tests;
{$mode objfpc}{$H+}
interface
uses
  TestFramework;
type
  TTestCaseFirst = class(TTestCase)
  published
    procedure TestWarning;
    procedure TestOne;
    procedure TestTwo;
    procedure TestThree;
  end;
  TClassA = class(TTestCase)
  published
    procedure TestClassA1;
    procedure TestClassA2; virtual;
  end;
  TClassB = class(TClassA)
  published
    procedure TestClassA2; override;
    procedure TestClassB1;
    procedure TestError;
  end;
procedure RegisterTests;
implementation
uses
  sysutils;
procedure RegisterTests;
begin
  TestFramework.RegisterTest(TTestCaseFirst.Suite);
  TestFramework.RegisterTest(TClassB.Suite);
end;

{ TTestCaseFirst }

procedure TTestCaseFirst.TestWarning;
begin
  // Do nothing here - should cause a Warning
end;
procedure TTestCaseFirst.TestOne;
begin
  Check(1 + 1 = 3, 'Catastrophic arithmetic failure!');
end;
procedure TTestCaseFirst.TestTwo;
begin
  Check(1 + 1 = 2, 'Catastrophic arithmetic failure!');
end;
procedure TTestCaseFirst.TestThree;
var
  s: string;
begin
  s := 'hello';
  CheckEquals('Hello', s, 'Failed CheckEquals');
end;

{ TClassA }

procedure TClassA.TestClassA1;
begin
  fail('TClassA.TestClassA1');
end;
procedure TClassA.TestClassA2;
begin
  Fail('This virtual method should never appear.');
end;

{ TClassB }

procedure TClassB.TestClassA2;
begin
  Fail('Test overridden method');
end;
procedure TClassB.TestClassB1;
begin
  sleep(2264);
  Fail('Test sleep() causing extra time');
end;
procedure TClassB.TestError;
var
  x, y: integer;
begin
  x := 10;
  y := 0;
  Check(x / y = 0, 'Failed on 1');
end;
end.
---------------------------------[ end ]----------------------------------

The source code is freely available from Github using the following command.

git clone git://github.com/graemeg/fptest.git

or

git clone https://github.com/graemeg/fptest.git

If you don't have git installed, you can always grab a source tarball too, using the following URL.

https://github.com/graemeg/fptest/tarball/master

FPTest documentation can be found in the 'docs' directory as HTML files. The FPTest project is still under active development, so things are constantly improved. For this reason I highly recommend you get the source code via git, instead of as a tarball.

Good news is that you are not forced to choose either or testing frameworks. Years ago I included a DUnit/FPTest compatibility interface to FPCUnit. This gives you a nice upgrade path from FPCUnit to FPTest. So if you design your test suites using the CheckXXX calls and not the AssertXXX calls, then later you can easily switch to the FPTest framework without any need for changing your testing code.

I didn't know this project. I'll try.

Marcos Douglas

Graeme, I think you should at least make a wiki page for this project. I also just know about it here.

leledumbo wrote

Graeme, I think you should at least make a wiki page for this project. I also just know about it here.

Good idea - I'll reserve a page in the FPC wiki for this. [ You know programmers, we hate to document things. ;-) ]

Oh yeah... =)

Marcos Douglas

leledumbo wrote

Graeme, I think you should at least make a wiki page for this project. I also just know about it here.

I created a page with some initial information and a couple of screenshots. I'll extend the details and usage later.

http://wiki.freepascal.org/FPTest