Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2011-06

fpc build problems on some debian armel buildds

12 messages · 6 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

peter green

Freepascal has been failing sometimes with "einouterror file not found" on armel buildds but i'm not really seeing any pattern to the failures either in terms of upstream versions or in terms of buildds

2.4.0-2 was successfully built on muscat
2.4.0-3 failed on arnold
2.4.0-4 failed on arnold and ancina (the latter being a retry at my request) 2.4.2-1 was sucessfully built on alwyn
2.4.2-2 and 2.4.2-2 were successfully build on ancina 2.4.4-1 failed on antheil

The current FPC package builds fine in qemu for me. I will attempt to try on some real hardware too when I get a chance.

This is worrying for me because I use freepascal on armel and yet it's pretty hard to find a fix for a bug without a meaningful error message and which I cannot reproduce.

Is there a list anywhere of what hardware the buildds run on and/or any other interesting information about their setups so I can try to figure out what if anything the failing buildds have in common. Has anything changed in ancina's configuration recently?

Could some arm users try building fpc and reporting back if they can reproduce the error and if so on what hardware? (note: you must build with -B or you will run into an unrelated failure). If it fails for you can you try running the failing command under strace to try and determine what the file was?

Reply by email

Riku Voipio
to peter green
peter green wrote
The current FPC package builds fine in qemu for me. I will attempt to try on some real hardware too when I get a chance.

Qemu allows unaligned memory accesses, which do not always work on real hardware, especially on armv5 and other older arms.

Is there a list anywhere of what hardware the buildds run on and/or any other interesting information about their setups so I can try to figure out what if anything the failing buildds have in common. Has anything changed in ancina's configuration recently?

All currently running armel buildd's are identical marvell mv78x00 boards. Updates on ancina are the regular "apt-get upgrades" to get latest toolchain etc in sid.

Riku

Reply by email

Steve McIntyre
to peter green: Freepascal has been failing sometimes with "einouterror file not found" on armel buildds…
peter green wrote 22 lines
Freepascal has been failing sometimes with "einouterror file not found" on armel buildds but i'm not really seeing any pattern to the failures either in terms of upstream versions or in terms of buildds 2.4.0-2 was successfully built on muscat 2.4.0-3 failed on arnold 2.4.0-4 failed on arnold and ancina (the latter being a retry at my request) 2.4.2-1 was sucessfully built on alwyn 2.4.2-2 and 2.4.2-2 were successfully build on ancina 2.4.4-1 failed on antheil The current FPC package builds fine in qemu for me. I will attempt to try on some real hardware too when I get a chance. This is worrying for me because I use freepascal on armel and yet it's pretty hard to find a fix for a bug without a meaningful error message and which I cannot reproduce. Is there a list anywhere of what hardware the buildds run on and/or any other interesting information about their setups so I can try to figure out what if anything the failing buildds have in common. Has anything changed in ancina's configuration recently?

muscat is a Thecus N2100, (iop32x with 512MB RAM)

alwyn and antheil are identical Marvell development boards - see http://blog.einval.com/2010/09/27#marvell_buildds for more details.

If you want to know more, http://db.debian.org/machines.cgi lists more details for most of the official machines used by Debian.

If you're getting inconsistent results between antheil and alwyn then
that's worrying. :-(

You can ask for access to abel (the armel porter box) and play with stuff there if it helps.

Steve McIntyre, Cambridge, UK. < liw> everything I know about UK hotels I learned from "Fawlty Towers"

Reply by email

Mark Morgan Lloyd
to Riku Voipio: Qemu allows unaligned memory accesses, which do not always work on real hardware,…
Riku Voipio wrote 15 lines
On Thu, Jun 30, 2011 at 03:33:36PM +0100, peter green wrote: > The current FPC package builds fine in qemu for me. I will attempt to > try on some real hardware too when I get a chance. Qemu allows unaligned memory accesses, which do not always work on real hardware, especially on armv5 and other older arms. > Is there a list anywhere of what hardware the buildds run on and/or any > other interesting information about their setups so I can try to figure > out what if anything the failing buildds have in common. Has anything > changed in ancina's configuration recently? All currently running armel buildd's are identical marvell mv78x00 boards. Updates on ancina are the regular "apt-get upgrades" to get latest toolchain etc in sid.

There were issues with some versions of FPC, related to more than a certain number of parameters (four?) being passed. Jonas wrote the following on the 5th October last year:

-----8<-----

Mark Morgan Lloyd wrote 6 lines
When running 2.4.0 on an ARM system (Debian v5 "Lenny", armel) with limited memory (32Mb RAM + 768Mb swap) and using it to compile a large project (Lazarus 0.9.28.2) I'm seeing intermittent failures which go away if the make is restarted. I've not seen this running on other platforms, and I don't believe it is a problem in the Lazarus sources since the build will eventually complete giving me runnable code.

A couple of days ago I fixed an error in svn trunk for ARMEL that caused the stack to become temporarily unbalanced after performing syscalls with 5 or more parameters (the bug is still there for OABI, but I can't fix that because I don't have access to an OABI machine).

A side-effect of that bug was that if the caller passed the address of its own result as one of the parameters to the system call, it would afterwards return a random value as its result and checks for error results caused random failures like the one you posted (the reproducible case that allowed me to fix it was a similar error).

Jonas
----->8-----

There might be alignment issues on ARM and SPARC but I've only seen those with Lazarus, not with FPC itself.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email

peter green
to Riku Voipio: Qemu allows unaligned memory accesses, which do not always work on real hardware,…
Riku Voipio wrote
Qemu allows unaligned memory accesses, which do not always work on real hardware, especially on armv5 and other older arms.

Do you know specifically how the debian buildds handle unaligned accesses? do they fix them up? do they generate a bus error? do they let them go silently wrong?

Regardless though I don't think that is the issue here as I've finally seen the einouterror in qemu after trying building in the precise path the buildd used (and I managed to reproduce it twice in a row under those circumstances). Unfortunately re-running the failing command manually succeeds so it seems like the long path alone is insufficiant to trigger the bug and there must be something in the environment the build system provides as well. I tried shoving a strace in there but that also seemed to stop it failing so i've no idea how I can track this down further.

Another complication is that this is happening as the first new compiler is being built using the compiler installed on the build system, in other words even if the issue is tracked down and fixed a new manual upload is likely to be needed to fix it.

Reply by email

peter green
to Mark Morgan Lloyd: There were issues with some versions of FPC, related to more than a certain number of…
Mark Morgan Lloyd wrote 21 lines
Riku Voipio wrote: > On Thu, Jun 30, 2011 at 03:33:36PM +0100, peter green wrote: >> The current FPC package builds fine in qemu for me. I will attempt >> to try on some real hardware too when I get a chance. > > Qemu allows unaligned memory accesses, which do not always work on real > hardware, especially on armv5 and other older arms. > >> Is there a list anywhere of what hardware the buildds run on and/or >> any other interesting information about their setups so I can try >> to figure out what if anything the failing buildds have in common. >> Has anything changed in ancina's configuration recently? > > All currently running armel buildd's are identical marvell mv78x00 > boards. Updates on ancina are the regular "apt-get upgrades" to get > latest toolchain > etc in sid. There were issues with some versions of FPC, related to more than a certain number of parameters (four?) being passed. Jonas wrote the following on the 5th October last year:

Do you know if that fix has made it into any stable releases and if so which ones?

Show quoted text 27 lines
-----8<----- On 05 Oct 2010, at 10:05, Mark Morgan Lloyd wrote: > When running 2.4.0 on an ARM system (Debian v5 "Lenny", armel) with > limited memory (32Mb RAM + 768Mb swap) and using it to compile a large > project (Lazarus 0.9.28.2) I'm seeing intermittent failures which go > away if the make is restarted. I've not seen this running on other > platforms, and I don't believe it is a problem in the Lazarus sources > since the build will eventually complete giving me runnable code. A couple of days ago I fixed an error in svn trunk for ARMEL that caused the stack to become temporarily unbalanced after performing syscalls with 5 or more parameters (the bug is still there for OABI, but I can't fix that because I don't have access to an OABI machine). A side-effect of that bug was that if the caller passed the address of its own result as one of the parameters to the system call, it would afterwards return a random value as its result and checks for error results caused random failures like the one you posted (the reproducible case that allowed me to fix it was a similar error). Jonas ----->8----- There might be alignment issues on ARM and SPARC but I've only seen those with Lazarus, not with FPC itself.

Reply by email

Jonas Maebe
to peter green
peter green wrote 15 lines
Mark Morgan Lloyd wrote: > Riku Voipio wrote: >> On Thu, Jun 30, 2011 at 03:33:36PM +0100, peter green wrote: >>> The current FPC package builds fine in qemu for me. I will attempt to try on some real hardware too when I get a chance. >> >> Qemu allows unaligned memory accesses, which do not always work on real >> hardware, especially on armv5 and other older arms. >> >>> Is there a list anywhere of what hardware the buildds run on and/or any other interesting information about their setups so I can try to figure out what if anything the failing buildds have in common. Has anything changed in ancina's configuration recently? >> >> All currently running armel buildd's are identical marvell mv78x00 boards. Updates on ancina are the regular "apt-get upgrades" to get latest toolchain >> etc in sid. > > There were issues with some versions of FPC, related to more than a certain number of parameters (four?) being passed. Jonas wrote the following on the 5th October last year: Do you know if that fix has made it into any stable releases and if so which ones?

It hasn't. It's only available in svn trunk (r16073)

Jonas

Reply by email

Mark Morgan Lloyd
to peter green: Do you know specifically how the debian buildds handle unaligned accesses? do they fix…
peter green wrote 8 lines
Riku Voipio wrote: > > Qemu allows unaligned memory accesses, which do not always work on real > hardware, especially on armv5 and other older arms. > Do you know specifically how the debian buildds handle unaligned accesses? do they fix them up? do they generate a bus error? do they let them go silently wrong?

Google suggests that it's a known issue [1], and that the degree of warning etc. may be controlled using /proc/cpu/alignment [2]

1] http://qemu-forum.ipi.fi/viewtopic.php?f=20&t=5543

2]
http://lecs.cs.ucla.edu/wiki/index.php/XScale_alignment#Have_the_kernel_find_the_problem_for_you

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email

Hector Oron
to peter green: Freepascal has been failing sometimes with "einouterror file not found" on armel buildds…

Hello,

2011/6/30 peter green :

Show quoted text 5 lines
Could some arm users try building fpc and reporting back if they can reproduce the error and if so on what hardware? (note: you must build with -B or you will run into an unrelated failure). If it fails for you can you try running the failing command under strace to try and determine what the file was?

Built on abel.d.o,
dpkg-genchanges -B >../fpc_2.4.4-1_armel.changes

It might be some kind of nasty bug on the build daemon software. Does your package depend on how the environment is setup? I'll upload this binaries at this time unless you state the contrary.

Best regards,

 Héctor Orón  -.. . -... .. .- -.   -.. . ...- . .-.. --- .--. . .-. <free spam> -- Would you like to make a donation for Debian Conference?    ** http://debconf11.debconf.org/payments.xhtml ** </free spam>

Reply by email

Hector Oron
to Hector Oron

Hi,

2011/7/1 Hector Oron :

I'll upload this binaries at this time unless you state the contrary.

Instead of this, I have given back (trigger a rebuild) the package, as it was only built once at the end of may.
Second, if the error reproduces again try to add more diagnostics code around the test, check alignment, environment, trace it, etc... Third, attempt a build on a porterbox (abel or agricola), if the build fails on buildd, but builds on porterbox manually 100% of the time, it can probably be uploaded to the archive. But we should keep chasing the problem.

Best regards,

 Héctor Orón  -.. . -... .. .- -.   -.. . ...- . .-.. --- .--. . .-. <free spam> -- Would you like to make a donation for Debian Conference?    ** http://debconf11.debconf.org/payments.xhtml ** </free spam>

Reply by email

peter green
to Jonas Maebe: It hasn't. It's only available in svn trunk (r16073) Jonas
Jonas Maebe wrote 27 lines
On 30 Jun 2011, at 19:04, peter green wrote: > Mark Morgan Lloyd wrote: > >> Riku Voipio wrote: >> >>> On Thu, Jun 30, 2011 at 03:33:36PM +0100, peter green wrote: >>> >>>> The current FPC package builds fine in qemu for me. I will attempt to try on some real hardware too when I get a chance. >>>> >>> Qemu allows unaligned memory accesses, which do not always work on real >>> hardware, especially on armv5 and other older arms. >>> >>> >>>> Is there a list anywhere of what hardware the buildds run on and/or any other interesting information about their setups so I can try to figure out what if anything the failing buildds have in common. Has anything changed in ancina's configuration recently? >>>> >>> All currently running armel buildd's are identical marvell mv78x00 boards. Updates on ancina are the regular "apt-get upgrades" to get latest toolchain >>> etc in sid. >>> >> There were issues with some versions of FPC, related to more than a certain number of parameters (four?) being passed. Jonas wrote the following on the 5th October last year: >> > Do you know if that fix has made it into any stable releases and if so which ones? > It hasn't. It's only available in svn trunk (r16073)

Is it reasonablly backportable?
Are there any plans for a new stable release any time soon either based on the current stable release branch or freshly branched from trunk?

Show quoted text 6 lines
Jonas_______________________________________________ fpc-devel maillist - http://lists.freepascal.org/mailman/listinfo/fpc-devel

Reply by email

Jonas Maebe
to peter green
peter green wrote 16 lines
Jonas Maebe wrote: > On 30 Jun 2011, at 19:04, peter green wrote: > > >> Mark Morgan Lloyd wrote: >> >>> There were issues with some versions of FPC, related to more than >>> a certain number of parameters (four?) being passed. Jonas wrote >>> the following on the 5th October last year: >>> >> Do you know if that fix has made it into any stable releases and >> if so which ones? > > It hasn't. It's only available in svn trunk (r16073) > Is it reasonablly backportable?

I should apply to the fixes branch without any problems.

Are there any plans for a new stable release any time soon either based on the current stable release branch or freshly branched from trunk?

Not that soon (I guess September or October would be the soonest, end of the year is probably more likely).

Jonas

Reply by email