Hi Vincent,

most of what I do is programs that output HTML results. I'm sure I can be of some assistance to you. Can you zip and send the unit test results to me so I can make you a small sample of what I have in mind ?

Kind regards
Ewald Horn
Web: www.nofuss.co.za

Hi,

I have created some tests with fpunit and now I want to present the test results on a html page like
http://hccweb1.bai.ne.jp/tsune-1/NUnitTestResult.html

Graeme Geldenhuys provided me with some sample xsl files and css files, but what seemed to be missing in those is timing results and a complete listing of all executed tests and results per suite. I don't know how to program stylesheets, so my question is:

* Has anybody else converted the xml results to html?
* Can somebody help with the creation of the html page?

Vincent

Thanks for your offer.

In the coming days, I will send you the xml files in private.

Vincent

Hi Vincent,

Graeme Geldenhuys provided me with some sample xsl files and css files, but what seemed to be missing in those is timing results and a complete listing of all executed tests and results per suite. I don't know how to

Oh, I left out the processing of the executed test result.... That could explain why they are missing. :-)
The sample xml file I used didn't have much to report on, so I left it out.

If you can give me a snippet of the XML file (node names and hierarchy) I will amend the XSLT file for you.

Regards,
- Graeme -

Hi Vincent,

Is is easy to add the list of executed tests (5 min job), but the missing feature I believe you are looking for is the timing results. Currently none of the fpcunit results contain timing values. This is something I wanted to add long ago. The GUI TestRunner does show the total time taken, but that is only displayed, not saved anywere.

I will see how far I get with adding timing result today... if done, I will submit a patch in fpc-devel.

Regards,
- Graeme -

Vincent Snijders wrote

Graeme Geldenhuys provided me with some sample xsl files and css files, but what seemed to be missing in those is timing results and a complete listing of all executed tests and results per suite. I don't know how to program stylesheets, so my question is:

Graeme Geldenhuys ha scritto:

At first sight, it seems that the methods StartTest and EndTest of the listeners are the right place where to place the code to do the timings. I guess the right approach would be to add an option at command line or otherwise to add the timing result. Same for the list of the run tests.

Successive steps that are natural is the automation of fpcunit testing: a daemon that checks some svn directories periodically to find if there are changes in the tests, does the update, compiles the tests and publishes the tests (as an html page or sends the results to some registered users via e-mail, irc or othere means). I'm investigating how to best achieve this, I would use lnet for the networking part.

Ciao, Dean

I would use command line tools like svn and scp to do the network part.

Vincent

Dean Zobec wrote

At first sight, it seems that the methods StartTest and EndTest of the listeners are the right place where to place the code to do the timings. I guess the right approach would be to add an option at command line or otherwise to add the timing result. Same for the list of the run tests.

Thanks for the pointers Dean. I didn't get time today, but should over the weekend.

Regards,
- Graeme -

The tests are in Lazarus svn in the test subdirectory.

Vincent

Any progres?

Vincent

Sorry, I got side tracked with work requirements. Who decided there must only be 24 hours in a day! :-)

I already have most of the timing results implemented, just need to rework the way the Text TestRunner uses the XML output. I will finish it in this week.

Regards,
- Graeme -

A sample xml file with test results can be found on http://www.hu.freepascal.org/lazarus/testresults/results.xml

What is missing in these files in the test hierarchy: A future version of the results file will probably include it, so it results.html can show first totals for testsuite, then total for examples, etc.

Vincent

Vincent Snijders wrote

A sample xml file with test results can be found on http://www.hu.freepascal.org/lazarus/testresults/results.xml What is missing in these files in the test hierarchy:

forgot to paste the link :-(
http://www.hu.freepascal.org/lazarus/testresults/tree.xml

A future version of the results file will probably include it, so it results.html can show first totals for testsuite, then total for examples, etc.

Vincent

Hi Vincent,

I haven't forgotten about your updated .xslt file to take into account the timing results. I have been really busy at work, and have a deadline approaching this Friday. After that, I should have a bit more time...

In the mean time, I have been updating my Linux build server for the tiOPF project, which also uses fpcUnit. I am modifying the .xslt file I gave you for my own usage as well. The tiOPF project has over 1500 tests, so I have a lot of data in the results.xml file to play with. As soon as I have the .xslt & .css files complete, I will send you a copy. Maybe we can even add it to the fpcUnit examples in SubVersion.

Does FPC come with a unit that can process xslt files and save the generated result to a file?

Regards,
- Graeme -

Hi Vincent,

Attached is my latest efforts on fpcUnit Test Reports using XSLT. Before, I sent you two versions of the .xsl file. One created a report based on CSS only and the other one based on HTML Tables only. I tested both against my results (around 900 tests) and the CSS version was dog slow!!

I then created from scratch a .xsl file generating HTML Tables (small ones) and uses a small CSS file to format (colors, fonts, etc) the generated HTML. The speed is *much* better.

Attached is a archive containing the CSS, XSL and sample results files. I also included a small script file (fixup_xml.sh) which inserts a single line into the xml results file, to specify the xslt stylesheet to use.

The reason for the script file, is because I didn't what to modify fpcUnit to insert that line automaticaly, as I like to generate different formatted results from the original xml file and need to insert different XSLT stylesheets for that. For example, I generate two different outputs from the same results.xml file. One for HTML results going to my web server and the other, a plain text email, which gets emailed to a newsgroup showing only the summary of the results.

This version is much better than the ones I sent you before. A few things are still missing though, but that is due to the data being missing in the xml results.

* The test listings need to be grouped by test suites which will allow for a better summaries to be generated.
* More attributes need to be added to the test listings to give information like: Did the test pass, fail or give a error. This will allow me to link the summary test listing to the detailed results using the HTML version of the XSLT.
* Ignored/Skipped tests still need support - My version of fpcUnit has not been patched yet.

Enjoy, and please give me some feedback. Good or bad!

Regards,
- Graeme -

I like the result:
http://www.hu.freepascal.org/lazarus/testresults/results.xml

The console testrunner, that I use, will insert the stylesheet information.

This would be my first feature request.

Looking forward to further improvements.

Vincent

Vincent Snijders wrote

I like the result: http://www.hu.freepascal.org/lazarus/testresults/results.xml

I couldn't view it. Got an error that the results.xsl file couldn't be found. Note: I renamed the results.xsl file to fpcunit.xsl, so you will have to update your 2nd line in the results.xml file to point to the right xsl file. Damn, that sounds confusing! :-)

The console testrunner, that I use, will insert the stylesheet information.

I didn't want to amend mine, for in case I accidentally send it in a patch. Also I needed two result output formats.

> * The test listings need to be grouped by test suites which will allow > for a better summaries to be generated. This would be my first feature request.

Mine too. Coming up shortly...

Regards,
- Graeme -

I already did that and tested that. Maybe the browser is showing you an old version?

Make it a command line parameter.

Great.

Vincent

Vincent Snijders wrote

I already did that and tested that. Maybe the browser is showing you an old version?

Ah yes, it view my browser cache. Refresh fixed it. :-)

Nope I didn't want to, because then I need to re-run all my tests again, just to get another output format. Well, I guess I could add and just not use it. That way I could still insert my own stylesheets on a single test run.

Oh, here is my results for tiOPF2. I haven't enabled all tests yet. Those are only half of them. See the reason why I want to fix the Test Listing nodes. Also notice that my summary is Red, not Green, due to the Error count > 0.

http://opensoft.homeip.net/tiopf/fpcunit/results.xml

Graeme.

Hi Vincent,

Small bugfix in the fpcunit.xsl file. On line 58, the $ sign was missing infront of the "failureCount' variable. This caused the Summary line (when failures occured and no errors) to show up green instead of magenta.

Line 58 should look as follows:

<xsl:when test="$failureCount &gt; 0">

Regards,
- Graeme -