Hello,

I've prepared small test suite based on ALGLIB's Fast Transforms sub-package: tests for FFT, FHT, convolution and cross-correlations, 4 tests in total.

It can be downloaded from
http://www.alglib.net/share/fpc-test-suite.zip

Test suite should be compatible with FreePascal framework: tests start with "t_", additional dependencies start with "u_" (I hope that additional underscores in file names will be OK). Due to its small size this test suite is fast (compilation+execution should take less than minute).

This suite is automatically generated from ALGLIB source, so if everything is alright, then I can supply you with much larger set of tests.

It was tested under x86/x86_64, and in principle it should work under other platforms (ARM, PowerPC, SPARC).

Sergey Bochkanov wrote

I've prepared small test suite based on ALGLIB's Fast Transforms sub-package: tests for FFT, FHT, convolution and cross-correlations, 4 tests in total.

Thanks, I've added them to svn. I cursorily tested them and they at least work under Mac OS X/x86_64 and Mac OS X/PowerPC.

Jonas

Hello, Jonas.

Thanks, I've added them to svn. I cursorily tested them and they at least work under Mac OS X/x86_64 and Mac OS X/PowerPC.

Is it possible to test them under other platforms? SPARC, ARM, PowerPC64?

I am testing full set of ALGLIB units under Win32/Win64/Lin32/Lin64 (x86). After all will be finished, I'll send you new test suite.

However, there is a problem you should know about. ALGLIB uses non-deterministic unit tests. I.e. test sets are generated randomly (instead of being built-in, pre-generated). This approach have both benefits and drawbacks.

From the one side, it allows me to test ALGLIB with much wider set of data (each time test is run, new inputs are tested). From the other side, there is a problem of false positives - test tolerances may be too stringent and sometimes correctly working unit don't match this stringent criterion.

So error message from ALGLIB unit tests may means that: a. there is a bug in ALGLIB
b. there is a bug in compiler
c. there are no bugs, just too stringent criterion in unit test, which sometimes (0.1% of time, for example) doesn't met.

In case (c) I relax error criterion used. But each case of this kind requires careful investigation.

It is almost impossible to get rid of type (c) errors because numerical algorithms are very complex. Some ALGLIB units are more prone to such false alarms, some units are less prone. FFT units I've sent you are the most stable in this respect - false alarms are almost impossible.

So we have two options:
1. to use only the most stable units
2. to give me limited access to SVN so I may run these tests myself and study their results.

I have enough time on my hands so I can help you with tests if you want.

Sergey Bochkanov.

The tests are part of the nightly test suite now, so they are tested on all platforms for which nightly tests are performed. They seem to have worked fine on all properly tested platforms (ARM is not one of those though).

However, there is a problem you should know about. ALGLIB uses non-deterministic unit tests. I.e. test sets are generated randomly (instead of being built-in, pre-generated). This approach have both benefits and drawbacks.

At the very least, the value of randseed should probably be written out on startup so that the test can be reproduced under the same circumstances if it fails.

Everyone has read-only access to the tests in svn, since they're part of the regular fpc svn repository: http://www.freepascal.org/develop.var#svn

To run the testsuite, first perform a "make clean all" in the FPC directory, then change to the tests directory and run

make FPC=compiler_to_compile_testsuite_utilities TEST_FPC=/full/path/ to/fpc/compiler/ppc386 TEST_OPT="-O2 -Cppentiumm" clean all digest

(replace ppc386 with ppcx64 in case of x86_64 platforms, and the contents of TEST_OPT with whatever options you want to use to compile the tests). Afterwards, you an find a list of failed tests in output/ $fulltarget/faillist, and output/$fulltarget/longlong contains the full compiler/test output).

This will always do a full testsuite run though (which takes quite a while to complete, even on a fast machine). There's currently no way to only execute a subset of the testsuite. The alglib tests are in tests/test/alglib.

If you want, I can give you svn write access to tests/test/alglib so you can easily add/change the tests there though.

Jonas

Hello, Jonas.

At the very least, the value of randseed should probably be written out on startup so that the test can be reproduced under the same circumstances if it fails.

My tests report seed values along with names of units. I've stripped all error reporting (except for exit code) from units I've sent you because I didn't know how testsuite handles console output. It is easy to turn it on again.

This will always do a full testsuite run though (which takes quite a while to complete, even on a fast machine). There's currently no way to only execute a subset of the testsuite.

Yes, what's with time? ALGLIB takes a while to execute all tests. About 5 minutes per pass on my 2.4 GHz Pentium Core 2 (virtualized). It is affordable for me, but may be too slow for you.

How much time you can spend on ALGLIB tests? If you say 5 minutes per pass is too long, I can select the most sensitive/fast units. Otherwise, it is better to test all the package.

If you want, I can give you svn write access to tests/test/alglib so you can easily add/change the tests there though.

Yes, that would be the easiest way of keeping ALGLIB tests up to date.

(ARM is not one of those though).

Using an ARM Emulator would be nice...

-Michael