Hi,

I use FPCUnit extensively at work and in open source projects like tiOPF, but I hit a bit of a snag - actually a bug in FPCUnit which causes many tests to fail for the wrong reasons. Here is one such case
- in the tiOPF project many database tests fail do to "Setup Once" issues: http://opensoft.homeip.net/tiopf/fpcunit/index.html Michael van Canneyt has also hit this bug before, but now I'm not sure what exactly to do about it.

The bug has to do with running a "setup" or "teardown" only once. Yes FPCUnit has the TestDecorator which adds this support, but if you have a nested test hierarchy, the running once falls over. When doing unit tests that share DB connections this issues becomes a major pain.

Michael had a more in-depth look at the issue and told me that the problem lies in the heart of FPCUnit's design and is not very easy to solve. So much so, that he recommends it is better to simply rewrite FPCUnit than try and patch the existing code. I have not taken such a close look to the issue, but had to modify my test hierarchy to get around the problem (which should really not have been needed in the first place).

Michael, do you still feel the same way about this issue? Dean Zobec, are you still around? FPCUnit was your baby to start with. Do you have any comments?

I'm at cross-roads now. Do I try and solve the problem using the current code, or do I collaborate with Michael and Dean (and anybody else interested) and start a rewrite? If a rewrite is the preferred option, I have a few questions

* how will this affect all the existing FPCUnit users?
* will (or must) the new design have a similar API so our existing test suites will continue to run.
* Do we make it more DUnit compatible? I already created many DUnit compatibility methods, but some things are not possible.
* DUnit2 (hosted in the tiOPF code repository and written by the late Peter McNab) has some major improvements over the existing DUnit. Do we incorporate/port that, or merge the DUnit2 code into a new testing framework that is FPC and Delphi compatible?
* Obviously a timeframe is also important
* Do we use Interfaces (like DUnit and DUnit2) or do we use abstract classes like FPCunit currently does.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

 * DUnit2 (hosted in the tiOPF code repository and written by the late Peter McNab) has some major improvements over the existing DUnit. Do we incorporate/port that, or merge the DUnit2 code into a new testing framework that is FPC and Delphi compatible?

For those that don't know about DUnit2, here is some of the improvements made over the standard DUnit. DUnit2 also used the extensive test suite of the tiOPF project to help it's design and testing. Currently the Windows daily test runs of tiOPF use DUnit2.

http://members.optusnet.com.au/mcnabp/Projects/DUnit2/DUnit2Description.html

More links here:
http://members.optusnet.com.au/mcnabp/Projects/DUnit2/

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

In our previous episode, Graeme Geldenhuys said:

Is it FPC license compatible? DUnit always had the problem it was MPL.

y

I still feel the same way.

I'm at cross-roads now. Do I try and solve the problem using the current code, or do I collaborate with Michael and Dean (and anybody else interested) and start a rewrite? If a rewrite is the preferred option, I have a few questions

For me, a rewrite is the preferred option.

* how will this affect all the existing FPCUnit users?

It should not affect thim, this should be a dogma... After the fix, the decorator stuff should work 'as designed', but the rest should remain the same.

* will (or must) the new design have a similar API so our existing test suites will continue to run.

All code should run as-is.

* Do we make it more DUnit compatible? I already created many DUnit compatibility methods, but some things are not possible.

We'll have to look at this on a per-feature basis.

* DUnit2 (hosted in the tiOPF code repository and written by the late Peter McNab) has some major improvements over the existing DUnit. Do we incorporate/port that, or merge the DUnit2 code into a new testing framework that is FPC and Delphi compatible?

I'm not very comfortable taking over someone else's code, and as Marco points out, there may be a license issue.

* Obviously a timeframe is also important * Do we use Interfaces (like DUnit and DUnit2) or do we use abstract classes like FPCunit currently does.

I'm afraid I have seriously bad experiences with interfaces in FPC, and so would not like to see them used.

Michael.

Michael Van Canneyt wrote

I'm afraid I have seriously bad experiences with interfaces in FPC

How so? I'm not aware of any outstanding bugs with interfaces in FPC. Even the compiler uses them now (only for whole-program optimization though, so usually you won't test them).

Jonas

Marco van de Voort wrote

Is it FPC license compatible? DUnit always had the problem it was MPL.

Umm, I had a look and the code is a fork of the original DUnit, so DUnit2 still falls under the MPL license.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Michael Van Canneyt wrote

For me, a rewrite is the preferred option.

OK, could you then please list all the things that bug you in the current FPCUnit, so we don't duplicate the issues.

It should not affect thim, this should be a dogma... After the fix, the decorator stuff should work 'as designed', but the rest should remain the same.

This is what I expected. :-)

I'm afraid I have seriously bad experiences with interfaces in FPC, and so would not like to see them used.

I have used Interfaces in a few of my projects and found it quite handy and can't say that I experienced any issues. I'm nothing near and expert with them though, but do "get" them. I have only been using the default Delphi/COM based Interfaces (if that's what you call them), not the Corba ones. At the time, the Corba Interfaces still had many issues, but I think they have all been resolved since.

Could you clarify what issues you have experienced. If any bugs remain, maybe now would be a good time to resolve those as well.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Well, e.g.

http://bugs.freepascal.org/view.php?id=12778

In addition to what is described there, the API to retrieve interfaces seems to have changed over time, because I sent code to some other people in the full conviction that it would compile out of the box, but that was definitely not the case. It cost me days of checking, installing, recompiling :(

Another point was trying to get the Mozilla API working. All code works in Delphi, but fails horribly in FPC. I should check this again, maybe it has improved now. Available time is the biggest constraint :(

I'll try to reproduce all problems I had and insert bug reports for them.

Michael.

The main issue is that the decorator stuff has been added as an afterthought. It needs a more fundamental approach. If I remember my conclusions well, we needed something like a basic TTestIterator class, which should be used at all levels. I remember that Delphi has such a class (or interface).

See my other mail.

I must say that I was using mostly CORBA interfaces, to avoid reference count issues. The Mozilla thing used normal COM interfaces.

Michael.

Support for the "implements" keyword indeed is quite buggy, because it was never finished (there are also several failing tests for it in the testsuite).

I was only talking about the interfaces themselves (declaring them, defining classes that implement them via the class header declaration, declaring variables and parameters that are interface types, assigning class instances to interfaces, calling interface methods via class instance and interface variables).

Jonas

Graeme Geldenhuys pravi:

FPCUnit was a port of JUnit, actually a clone in Free Pascal. The Setup and Teardown run only once for a set of tests goes against the philosophy of unit testing actually, as violates the priciple that all the unit tests should be independent from each other. The feature was added as a decorator in JUnit for some marginal cases, with a very "use with care" labeled on it. Where the class under test has strong dependencies on other classes or the other classes are heavy to construct in the usual Setup run for every test, I think it's about time to use Mock Objects.

Anyway the code is quite old, JUnit evolved with the use of interfaces and annotations and fpcunit stood still. I think annotations (in NUnit and JUnit 4) were a big improvement as the tests can be inserted in any class, without a need of inheriting from a TTestcase.

I'm sorry but I have no spare time at the moment to participate in any serious development process, so I can only take a look from time to time to give some suggestions and recommendations.

Dean

Well, I must still create a bugreport for the Mozilla problem. That uses 'plain' interfaces.

Michael.

Michael Van Canneyt wrote

http://bugs.freepascal.org/view.php?id=12778

Indeed, that is a very handy feature of using interfaces. I'm glad you mentioned it, otherwise I was going to have some issues with my tiOPF MVP implementation, which I planed on being interface driven and heavily rely on delegation like that. Umm.... :-(

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

In our previous episode, Jonas Maebe said:

Decal uses interfaces quite heavily, and FPC survives its regressiontest.

In our previous episode, Michael Van Canneyt said:

Well, e.g. http://bugs.freepascal.org/view.php?id=12778

More a delegation problem. Together with 8951 and the pretty old 4842

Another point was trying to get the Mozilla API working. All code works in Delphi, but fails horribly in FPC. I should check this again, maybe it has improved now. Available time is the biggest constraint :(

A lot works nowadays. COM also works mostly also. 2.2.2 and 2.2.4 both improve on this.

Dean Zobec wrote

The Setup and Teardown run only once for a set of tests goes against the philosophy of unit testing actually, as violates the priciple that all the unit tests should be independent from each other.

The tests in tiOPF are independent of each other, but running some 800+ database related tests where each one creates and destroys a DB connections causes the tests to slow down by a *huge* magnitude. The "SetupOnce" and "TearDownOnce" was introduced in DUnit2 to setup and destroy a DB connection per persistence layer (ADO, Oracle, etc.) and greatly increased the time required to run all the tests. I tried to accomplish the same with the TestDecorator, but with a nested test hierarchy (4 or 5 levels deep), TestDecorator doesn't behave as expected and the "SetupOnce" and "TearDownOnce" never got fired. I tried to hack the test suite to get most of the tests to run, but as the unit test output shows, by hack causes other issues and is not a real solution.

I'm sorry but I have no spare time at the moment to participate in any serious development process, so I can only take a look from time to time

Thanks for popping though. :-)

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/