Hi everyone, I just fixed an issue with tests/bench/bfloatfunc.pp over at https://gitlab.com/freepascal.org/fpc/source/-/merge_requests/996 failing due to the reduced iteration count.  I've increased it from 1 to 8 when in conformance mode (in IN_TESTS defined), since if it's less than that, not all of the tests get executed and, as a consequence, many of the results are uninitialised and cause false failures. Just wanted to verify that it's good to merge since I'm reluctant to press Merge myself without approval. Kit

Hi,   I add the IN_TESTS conditional code to avoid getting timeout related failures, inside testsuite.   I didn't notice that more than one iteration was needed for bfloatfunc.pp, because I tried it with ITERATIONS=1 and it seemed to finish without failure...   I have no objection to your patch, please go ahead and commit. For tests/bench/bcase.pp, I had to set ITERATIONS To $10000 to avoid errors...   Just one small remark: why do you use '$8' instead of simply '8'? Pierre Le 25/04/2025 à 21:52, J. Gareth Moreton via fpc-devel a écrit : > Hi everyone, > > I just fixed an issue with tests/bench/bfloatfunc.pp over at > https://gitlab.com/freepascal.org/fpc/source/-/merge_requests/996 > failing due to the reduced iteration count.  I've increased it from 1 to > 8 when in conformance mode (in IN_TESTS defined), since if it's less > than that, not all of the tests get executed and, as a consequence, many > of the results are uninitialised and cause false failures. > > Just wanted to verify that it's good to merge since I'm reluctant to > press Merge myself without approval. > > Kit > > _______________________________________________ fpc-devel maillist  -  https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

"Just one small remark: why do you use '$8' instead of simply '8'?" Mostly habit because the tests use "Iteration and 7" to select the index, so I use $8 for the count to indicate that the bitwise nature of it is important. Kit On 25/04/2025 21:31, Pierre Muller via fpc-devel wrote: > Hi, > >   I add the IN_TESTS conditional code to avoid getting timeout > related failures, > inside testsuite. > >   I didn't notice that more than one iteration was needed for > bfloatfunc.pp, > because I tried it with ITERATIONS=1 and it seemed to finish without > failure... > >   I have no objection to your patch, please go ahead and commit. > > For tests/bench/bcase.pp, I had to set ITERATIONS To $10000 > to avoid errors... > >   Just one small remark: why do you use '$8' instead of simply '8'? > > Pierre > > > Le 25/04/2025 à 21:52, J. Gareth Moreton via fpc-devel a écrit : >> Hi everyone, >> >> I just fixed an issue with tests/bench/bfloatfunc.pp over at >> https://gitlab.com/freepascal.org/fpc/source/-/merge_requests/996 >> failing due to the reduced iteration count.  I've increased it from >> 1 to >> 8 when in conformance mode (in IN_TESTS defined), since if it's less >> than that, not all of the tests get executed and, as a consequence, >> many >> of the results are uninitialised and cause false failures. >> >> Just wanted to verify that it's good to merge since I'm reluctant to >> press Merge myself without approval. >> >> Kit >> >> > > _______________________________________________ > fpc-devel maillist  -  > https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel >

If it's okay, can I rename the define from "IN_TESTS" to "CONFORMANCE" or similar?  Technically, the code in "bcase" and "bfloatfunc" are still tests, but by default are benchmarking tests evaluating performace rather than conformance tests verifying that the compiler has produced correct code. Kit On 25/04/2025 22:00, J. Gareth Moreton via fpc-devel wrote: > "Just one small remark: why do you use '$8' instead of simply '8'?" > > Mostly habit because the tests use "Iteration and 7" to select the > index, so I use $8 for the count to indicate that the bitwise nature > of it is important. > > Kit > > On 25/04/2025 21:31, Pierre Muller via fpc-devel wrote: >> Hi, >> >>   I add the IN_TESTS conditional code to avoid getting timeout >> related failures, >> inside testsuite. >> >>   I didn't notice that more than one iteration was needed for >> bfloatfunc.pp, >> because I tried it with ITERATIONS=1 and it seemed to finish without >> failure... >> >>   I have no objection to your patch, please go ahead and commit. >> >> For tests/bench/bcase.pp, I had to set ITERATIONS To $10000 >> to avoid errors... >> >>   Just one small remark: why do you use '$8' instead of simply '8'? >> >> Pierre >> >> >> Le 25/04/2025 à 21:52, J. Gareth Moreton via fpc-devel a écrit : >>> Hi everyone, >>> >>> I just fixed an issue with tests/bench/bfloatfunc.pp over at >>> https://gitlab.com/freepascal.org/fpc/source/-/merge_requests/996 >>> failing due to the reduced iteration count.  I've increased it from >>> 1 to >>> 8 when in conformance mode (in IN_TESTS defined), since if it's less >>> than that, not all of the tests get executed and, as a consequence, >>> many >>> of the results are uninitialised and cause false failures. >>> >>> Just wanted to verify that it's good to merge since I'm reluctant to >>> press Merge myself without approval. >>> >>> Kit >>> >>> >> >> _______________________________________________ >> fpc-devel maillist  -  >> https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel >> >

You can do that if you want, maybe it would also be better to avoid all benchmark results writing in this conformance configuration as there is not enough averaging, and because in many cases the time interval is too short to get any value. Pierre Le 26/04/2025 à 23:50, J. Gareth Moreton via fpc-devel a écrit : > If it's okay, can I rename the define from "IN_TESTS" to "CONFORMANCE" > or similar?  Technically, the code in "bcase" and "bfloatfunc" are still > tests, but by default are benchmarking tests evaluating performace > rather than conformance tests verifying that the compiler has produced > correct code. > > Kit > > On 25/04/2025 22:00, J. Gareth Moreton via fpc-devel wrote: >> "Just one small remark: why do you use '$8' instead of simply '8'?" >> >> Mostly habit because the tests use "Iteration and 7" to select the >> index, so I use $8 for the count to indicate that the bitwise nature >> of it is important. >> >> Kit >> >> On 25/04/2025 21:31, Pierre Muller via fpc-devel wrote: >>> Hi, >>> >>>    I add the IN_TESTS conditional code to avoid getting timeout >>> related failures, >>> inside testsuite. >>> >>>    I didn't notice that more than one iteration was needed for >>> bfloatfunc.pp, >>> because I tried it with ITERATIONS=1 and it seemed to finish without >>> failure... >>> >>>    I have no objection to your patch, please go ahead and commit. >>> >>> For tests/bench/bcase.pp, I had to set ITERATIONS To $10000 >>> to avoid errors... >>> >>>    Just one small remark: why do you use '$8' instead of simply '8'? >>> >>> Pierre >>> >>> >>> Le 25/04/2025 à 21:52, J. Gareth Moreton via fpc-devel a écrit : >>>> Hi everyone, >>>> >>>> I just fixed an issue with tests/bench/bfloatfunc.pp over at >>>> https://gitlab.com/freepascal.org/fpc/source/-/merge_requests/996 >>>> failing due to the reduced iteration count.  I've increased it from >>>> 1 to >>>> 8 when in conformance mode (in IN_TESTS defined), since if it's less >>>> than that, not all of the tests get executed and, as a consequence, >>>> many >>>> of the results are uninitialised and cause false failures. >>>> >>>> Just wanted to verify that it's good to merge since I'm reluctant to >>>> press Merge myself without approval. >>>> >>>> Kit >>>> >>>> >>> >>> _______________________________________________ >>> fpc-devel maillist  -  >>> https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel >>> >> > _______________________________________________ fpc-devel maillist  -  https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel