Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2016-10

teventobject.create fails with error 161

16 messages · 5 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Snorkl e

Hi, Just wondering if anyone has any ideas.
I am using FPC 3.0 by the way.

Anyway, I am using a third party SFTP server lib and it creates a thread inherited from tthread and in the create event of the tthread descendant it does this:
(It uses this thread to read and write data on the socket)

FDataAvailable := TEvent.Create(nil, True, False, '');  //tevent maps to
teventobject
{$IFDEF MSWINDOWS}
  lasterr:=GetLastError;
  if (lasterr  0)   then
    begin
         msg:=SysErrorMessage(lasterr);
         SetLastError(NO_ERROR);  //<--I added this an attempt to recover
from the error
         raise Exception.Create(SCannotCreateEvent+':'+msg);
    end;
{$ENDIF}

after the call to FDataAvailable it always has a handle pointer, but what seems very random like GetLastError will return non 0 always with code 161 which is invalid path name.
When this occurs all new connections will fail, the process becomes useless and I have stop it.

If I remove the call to getlasterror and ignore it, the threads go into 100% cpu usage in the while loops used to read or write when the error does occur.

Users can actually connect but not transfer any data.

Also when I compile as 64bit the odds of this happening go way down and I can sometimes transfer 10000 files before it happens, on 32bit it happens much more frequently.

Apparently this is only a issue on Windows hence the compiler defs for MSWINDOWS.

I did report this issue to the vendor and they said they are looking into it but that could take months.

Anyone have any ideas one why teventobject.create is failing? Even though it does have a hander pointer is it possible it's somehow a bad pointer?

This happens on windows 7 through 10 and even on PCs with lots of resources(32gb ram, SSD hard drive and no corporate bloatware.

Thanks,

T

Jonas Maebe
to Snorkl e
Snorkl e wrote 10 lines
Anyway, I am using a third party SFTP server lib and it creates a thread inherited from tthread and in the create event of the tthread descendant it does this: (It uses this thread to read and write data on the socket) FDataAvailable := TEvent.Create(nil, True, False, ''); //tevent maps to teventobject {$IFDEF MSWINDOWS} lasterr:=GetLastError;

GetLastError() will return the error code of the last Windows API call that has been performed. I doubt TEvent.Create() guarantees not to call any Windows API's between creating an event using a Windows event and returning. E.g., it might allocate or free memory in between, or the FPC RTL may have done that in the context of handling the constructor call.

You should only call GetLastError() immediately after calling a Windows API function yourself. In any other case, you cannot know what may have transpired in between.

Jonas

Snorkl e
to Jonas Maebe

Hi, actually getlasterror only applies to the calling thread, getlastOSerror is the one not to use.
Anyway I think I found the solution.
Just for kicks I put in a name with a / I.e /test_name and getlasterror came back with the same error. So the API is being called somewhere with teventobject.create, anyway instead of leaving name blank I put a giud in for the name and low and behold the error stopped. It appears there is a limit or a bug on creating unnamed events from a single process. When you leave name blank the os must be creating a unique name somehow and it has some issue with a lot of them being created and destroyed. When I compiled as 64bit it took much longer for the error to happen. After I used the guid for the event name I stress tested the server with 10s of thousands of connection and disconnecting and that error never happened again.

So windows limit or bug? Or could it be a bug in FPC?

"Jonas Maebe" wrote 30 lines
On 14/10/16 02:43, Snorkl e wrote: > Anyway, I am using a third party SFTP server lib and it creates a thread > inherited from tthread and in the create event of the tthread descendant > it does this: > (It uses this thread to read and write data on the socket) > > FDataAvailable := TEvent.Create(nil, True, False, ''); //tevent maps to > teventobject > > {$IFDEF MSWINDOWS} > lasterr:=GetLastError; > GetLastError() will return the error code of the last Windows API call that has been performed. I doubt TEvent.Create() guarantees not to call any Windows API's between creating an event using a Windows event and returning. E.g., it might allocate or free memory in between, or the FPC RTL may have done that in the context of handling the constructor call. You should only call GetLastError() immediately after calling a Windows API function yourself. In any other case, you cannot know what may have transpired in between. Jonas _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Sven Barth
to Snorkl e
"Snorkl e" wrote
Hi, actually getlasterror only applies to the calling thread,

getlastOSerror is the one not to use.

GetLastOSError merely calls GetLastError on Windows...

Anyway I think I found the solution. Just for kicks I put in a name with a / I.e /test_name and getlasterror

came back with the same error. So the API is being called somewhere with teventobject.create, anyway instead of leaving name blank I put a giud in for the name and low and behold the error stopped. It appears there is a limit or a bug on creating unnamed events from a single process. When you leave name blank the os must be creating a unique name somehow and it has some issue with a lot of them being created and destroyed. When I compiled as 64bit it took much longer for the error to happen. After I used the guid for the event name I stress tested the server with 10s of thousands of connection and disconnecting and that error never happened again.

So windows limit or bug? Or could it be a bug in FPC?

Do you destroy the event again once it's no longer used? Of course if you require the events the whole time it might indeed be a Windows limitation.

Regards,
Sven

Tomas Hajny
to Snorkl e: Hi, actually getlasterror only applies to the calling thread, getlastOSerror is the one…

On Fri, October 14, 2016 17:44, Snorkl e wrote:

Hi,

Hi, actually getlasterror only applies to the calling thread, getlastOSerror is the one not to use. Anyway I think I found the solution.

.
.

I'm glad that you found potential reason of the problem, but your assumption regarding GetLastError and GetLastOSError is not entirely correct. You're right that GetLastError returns the last error in the current thread, because that is written in the respective MSDN documentation. I don't think that this is in contradiction to the information from Jonas though.

Moreover, the only difference between GetLastError and GetLastOSError is the fact that GetLastError is direct part of the MS Windows API (and thus it behaves exactly as described in MSDN, but it is platform-specific), whereas GetLastOSError is the FPC RTL abstraction for returning error information from the underlying operating system (available across different platforms). As such, you should rely on the behaviour of GetLastOSError only to the level documented in the FPC documentation (in other words, certain aspects described in MSDN documentation for GetLastError may not be valid for GetLastOSError on all FPC supported platforms due to differences in error handling on different platforms).

As an example, direct calls to Windows API bypassing FPC RTL and resulting in an error would change the value returned by GetLastOSError on MS Windows, whereas direct calls to OS/2 API bypassing FPC RTL and resulting in an error would _not_ change the value returned by GetLastOSError on OS/2.

The implementation of GetLastOSError for MS Windows (Win32 or Win64) consists of a direct call to GetLastError and thus the same caveats apply for them on that platform.

Tomas

Show quoted text 33 lines
On Oct 14, 2016 9:23 AM, "Jonas Maebe" wrote: > On 14/10/16 02:43, Snorkl e wrote: > >> Anyway, I am using a third party SFTP server lib and it creates a >> thread >> inherited from tthread and in the create event of the tthread >> descendant >> it does this: >> (It uses this thread to read and write data on the socket) >> >> FDataAvailable := TEvent.Create(nil, True, False, ''); //tevent maps >> to >> teventobject >> >> {$IFDEF MSWINDOWS} >> lasterr:=GetLastError; >> > > GetLastError() will return the error code of the last Windows API call > that has been performed. I doubt TEvent.Create() guarantees not to call > any > Windows API's between creating an event using a Windows event and > returning. E.g., it might allocate or free memory in between, or the FPC > RTL may have done that in the context of handling the constructor call. > > You should only call GetLastError() immediately after calling a Windows > API function yourself. In any other case, you cannot know what may have > transpired in between. > > > Jonas
Snorkl e
to Tomas Hajny

Hi Tomas,

TEventobject.create is the only thing called in the thread constructor, and teventobject.create is indeed calling a API function in order to create the event. There is no other way to find out if teventobject.create failed as teventobject.handle always has a value even if it fails. So what I am getting at is its extremely likely getlasterror applies even though it does not call winapi directly.

Since I solved it by giving the name param a giud, the next thing to ask is why does teventobject fail kind of silently when the name parm is blank over time?

Also keep in mind this is third party code, so I don't really know why they used teventobject as much as they did, I would imagine they didn't want to use critical sections all over the place.

And yes the event object does get destroyed when the thread finishes. I understand what your saying about the differences between getlasterror and getlastoserror.

I do appreciate the comments :-)

"Tomas Hajny" wrote 82 lines
On Fri, October 14, 2016 17:44, Snorkl e wrote: Hi, > Hi, actually getlasterror only applies to the calling thread, > getlastOSerror is the one not to use. > Anyway I think I found the solution. . . I'm glad that you found potential reason of the problem, but your assumption regarding GetLastError and GetLastOSError is not entirely correct. You're right that GetLastError returns the last error in the current thread, because that is written in the respective MSDN documentation. I don't think that this is in contradiction to the information from Jonas though. Moreover, the only difference between GetLastError and GetLastOSError is the fact that GetLastError is direct part of the MS Windows API (and thus it behaves exactly as described in MSDN, but it is platform-specific), whereas GetLastOSError is the FPC RTL abstraction for returning error information from the underlying operating system (available across different platforms). As such, you should rely on the behaviour of GetLastOSError only to the level documented in the FPC documentation (in other words, certain aspects described in MSDN documentation for GetLastError may not be valid for GetLastOSError on all FPC supported platforms due to differences in error handling on different platforms). As an example, direct calls to Windows API bypassing FPC RTL and resulting in an error would change the value returned by GetLastOSError on MS Windows, whereas direct calls to OS/2 API bypassing FPC RTL and resulting in an error would _not_ change the value returned by GetLastOSError on OS/2. The implementation of GetLastOSError for MS Windows (Win32 or Win64) consists of a direct call to GetLastError and thus the same caveats apply for them on that platform. Tomas > > On Oct 14, 2016 9:23 AM, "Jonas Maebe" wrote: > >> On 14/10/16 02:43, Snorkl e wrote: >> >>> Anyway, I am using a third party SFTP server lib and it creates a >>> thread >>> inherited from tthread and in the create event of the tthread >>> descendant >>> it does this: >>> (It uses this thread to read and write data on the socket) >>> >>> FDataAvailable := TEvent.Create(nil, True, False, ''); //tevent maps >>> to >>> teventobject >>> >>> {$IFDEF MSWINDOWS} >>> lasterr:=GetLastError; >>> >> >> GetLastError() will return the error code of the last Windows API call >> that has been performed. I doubt TEvent.Create() guarantees not to call >> any >> Windows API's between creating an event using a Windows event and >> returning. E.g., it might allocate or free memory in between, or the FPC >> RTL may have done that in the context of handling the constructor call. >> >> You should only call GetLastError() immediately after calling a Windows >> API function yourself. In any other case, you cannot know what may have >> transpired in between. >> >> >> Jonas _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
José Mejuto
to Snorkl e
Snorkl e wrote
Since I solved it by giving the name param a giud, the next thing to ask is why does teventobject fail kind of silently when the name parm is blank over time?

Hello,

To me it looks like a memory corruption, in your program or in the library. Have you checked the code with heaptrc activated ?

Snorkl e
to Sven Barth: getlastOSerror is the one not to use. GetLastOSError merely calls GetLastError on…

Hi Sven,

Yes the events get destroyed in the threads destructor. I verified they were being freed up.

It would usually happen when lots of connect disconnects happened in a short time.

Once that error 161 popped it could not be recovered from, all new connections would fail on the teventobject.create.

It would also happen much sooner when compiled as 32bit.

I guess as long a using the GUID works long term it shouldn't be an issue.

"Do you destroy the event again once it's no longer used? Of course if you require the events the whole time it might indeed be a Windows limitation.

Regards,

Sven"

Snorkl e
to José Mejuto: To me it looks like a memory corruption, in your program or in the library. Have you…

Yep, I used heaptr, my code is clean.
Sometimes it would happen after 3 connects/disconnects and sometimes after 500.
With the GUID as the event name it has never happened again and I really stress tested it.

It sounds like windows bug to me, but who knows.

"José Mejuto" wrote 19 lines
El 14/10/2016 a las 20:11, Snorkl e escribió: Since I solved it by giving the name param a giud, the next thing to ask > is why does teventobject fail kind of silently when the name parm is > blank over time? > Hello, To me it looks like a memory corruption, in your program or in the library. Have you checked the code with heaptrc activated ? -- _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Sven Barth
"Snorkl e" wrote
Yep, I used heaptr, my code is clean. Sometimes it would happen after 3 connects/disconnects and sometimes

after 500.

With the GUID as the event name it has never happened again and I really

stress tested it.

It sounds like windows bug to me, but who knows.

Looking at the code of the RTL it could be a FPC bug. FPC calls CreateEvent() basically like this:

Result := CreateEvent(..., PChar(Name));

Now the point is that by Name being a String the result of PChar(Name) is always a valid string consisting only of #0. That however is not equal to Nil which the documentation on MSDN states as necessary for an anonymous event.

Are you able to compile FPC and its RTL yourself? In that case I'd like to ask you to adjust the function BasicEventCreate in $fpcdir/rtl/win/systhrd.inc like this:

var
  n: PChar;
begin
  new(plocaleventrec(result));
  if Length(Name) = 0 then
    n := Nil
  else
    n := PChar(Name);
  plocaleventrec(result)^.FHandle := CreateEvent(EventAttributes,
AManualReset, InitialState, n);
end;

With this and rebuilt RTL, packages and Co I'd ask you to test your code again and report back whether that solves the problem.

Regards,
Sven

Snorkl e
to Sven Barth

Hi Sven,
Yes I can try that this weekend sometime and will respond with results,. Thanks for taking a look at that 😁

"Sven Barth" wrote 46 lines
Am 14.10.2016 20:31 schrieb "Snorkl e" : > > Yep, I used heaptr, my code is clean. > Sometimes it would happen after 3 connects/disconnects and sometimes after 500. > With the GUID as the event name it has never happened again and I really stress tested it. > > It sounds like windows bug to me, but who knows. Looking at the code of the RTL it could be a FPC bug. FPC calls CreateEvent() basically like this: Result := CreateEvent(..., PChar(Name)); Now the point is that by Name being a String the result of PChar(Name) is always a valid string consisting only of #0. That however is not equal to Nil which the documentation on MSDN states as necessary for an anonymous event. Are you able to compile FPC and its RTL yourself? In that case I'd like to ask you to adjust the function BasicEventCreate in $fpcdir/rtl/win/systhrd.inc like this: var n: PChar; begin new(plocaleventrec(result)); if Length(Name) = 0 then n := Nil else n := PChar(Name); plocaleventrec(result)^.FHandle := CreateEvent(EventAttributes, AManualReset, InitialState, n); end; With this and rebuilt RTL, packages and Co I'd ask you to test your code again and report back whether that solves the problem. Regards, Sven _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Snorkl e
to Sven Barth: after 500. stress tested it. Looking at the code of the RTL it could be a FPC bug. FPC…

Sven,
Your idea worked. I downloaded the FPC source and made that change and then built everything to another directory.
modified the fpc.cfg to not point to the FPC dir in the Lazarus folder and made the path changes in the Lazarus IDE.
Compiled the project with the fresh build and did the same stress test and no issues with error 161.

Should I do bug report in the FPC bug tracker? Do you think the fix will be able to be included in the next FPC release?

Thanks again.

Sven Barth wrote 46 lines
Am 14.10.2016 20:31 schrieb "Snorkl e" : > > Yep, I used heaptr, my code is clean. > Sometimes it would happen after 3 connects/disconnects and sometimes after 500. > With the GUID as the event name it has never happened again and I really stress tested it. > > It sounds like windows bug to me, but who knows. Looking at the code of the RTL it could be a FPC bug. FPC calls CreateEvent() basically like this: Result := CreateEvent(..., PChar(Name)); Now the point is that by Name being a String the result of PChar(Name) is always a valid string consisting only of #0. That however is not equal to Nil which the documentation on MSDN states as necessary for an anonymous event. Are you able to compile FPC and its RTL yourself? In that case I'd like to ask you to adjust the function BasicEventCreate in $fpcdir/rtl/win/systhrd.inc like this: var n: PChar; begin new(plocaleventrec(result)); if Length(Name) = 0 then n := Nil else n := PChar(Name); plocaleventrec(result)^.FHandle := CreateEvent(EventAttributes, AManualReset, InitialState, n); end; With this and rebuilt RTL, packages and Co I'd ask you to test your code again and report back whether that solves the problem. Regards, Sven _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Sven Barth
to Snorkl e
"Snorkl e" wrote
Sven, Your idea worked. I downloaded the FPC source and made that change and

then built everything to another directory.

modified the fpc.cfg to not point to the FPC dir in the Lazarus folder

and made the path changes in the Lazarus IDE.

Compiled the project with the fresh build and did the same stress test

and no issues with error 161.

Thank you for confirming.

Should I do bug report in the FPC bug tracker? Do you think the fix will

be able to be included in the next FPC release?

Please file a bug report so that it's not forgotten. Considering that it's a serious bug regarding events (and maybe others that have names, I'll have to check that) I hope that I'll get approval to have it merged to 3.0.x.

Regards,
Sven

Snorkl e
to Sven Barth

Sven,
Bug report added: http://bugs.freepascal.org/view.php?id=30747

Have a great weekend :-)

Sven Barth wrote 27 lines
Am 15.10.2016 09:52 schrieb "Snorkl e" : > > Sven, > Your idea worked. I downloaded the FPC source and made that change and then built everything to another directory. > modified the fpc.cfg to not point to the FPC dir in the Lazarus folder and made the path changes in the Lazarus IDE. > Compiled the project with the fresh build and did the same stress test and no issues with error 161. Thank you for confirming. > Should I do bug report in the FPC bug tracker? Do you think the fix will be able to be included in the next FPC release? > Please file a bug report so that it's not forgotten. Considering that it's a serious bug regarding events (and maybe others that have names, I'll have to check that) I hope that I'll get approval to have it merged to 3.0.x. Regards, Sven _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Snorkl e
to Snorkl e

Sven,
Just a fyi, the mantis bug report form would not let me pick a Severity <http://bugs.freepascal.org/view_all_set.php?sort=severity&dir=DESC&type=2> so its showing up as minor.

Snorkl e wrote 38 lines
Sven, Bug report added: http://bugs.freepascal.org/view.php?id=30747 Have a great weekend :-) On Sat, Oct 15, 2016 at 3:30 AM, Sven Barth wrote: > Am 15.10.2016 09:52 schrieb "Snorkl e" : > > > > Sven, > > Your idea worked. I downloaded the FPC source and made that change and > then built everything to another directory. > > modified the fpc.cfg to not point to the FPC dir in the Lazarus folder > and made the path changes in the Lazarus IDE. > > Compiled the project with the fresh build and did the same stress test > and no issues with error 161. > > Thank you for confirming. > > > Should I do bug report in the FPC bug tracker? Do you think the fix > will be able to be included in the next FPC release? > > > > Please file a bug report so that it's not forgotten. Considering that > it's a serious bug regarding events (and maybe others that have names, I'll > have to check that) I hope that I'll get approval to have it merged to > 3.0.x. > > Regards, > Sven > > _______________________________________________ > fpc-pascal maillist - > http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal >
Sven Barth
to Snorkl e
"Snorkl e" wrote
Sven, Just a fyi, the mantis bug report form would not let me pick a Severity

so its showing up as minor.

Thank you for reporting the bug.
Yes, we've disabled the severity, cause the severity we assume is not usually the severity of the user.

Regards,
Sven