Hi,

trying to use FindFirst and FindNext on files I cannot detect the difference between a failed call because of no matching file and a failure when the file or dir is not listable by the user lacking permissions.

If I'd use fpstat and friends myself this would mean doing anything doubly and having to write some code for each os in question.

Is there any function or procedure that could help me here? Or should the implementation of FindFirst/Next be adapted to respect users credits?

TIA,
Marc

Marc Santhoff wrote:

Hi,

Which platform are you talking about, which FindFirst/FindNext
(Dos/SysUtils) and what is the behaviour expected from you? E.g. with:
---
uses
 Dos;
var
 SR: SearchRec;
begin
 FindFirst ('M:\IT\AuditPro', AnyFile, SR);
 WriteLn (DosError);
 FindClose (SR);
 FindFirst ('M:\IT\AuditPro\.', AnyFile, SR);
 WriteLn (DosError);
 FindClose (SR);
 FindFirst ('M:\IT\AuditPro\X', AnyFile, SR);
 WriteLn (DosError);
 FindClose (SR);
end.
===
running under Win32, where I don't have access to "M:\IT\AuditPro"
directory, but I have access to "M:\IT" (i.e. I can see the AuditPro
directory itself), I get:

0
0
5

(where 5 corresponds to Access Denied).

Note that I don't know whether "M:\IT\AuditPro\X" exists or not (but that is OK, IMHO).

Tomas

Tomas Hajny wrote

Which platform are you talking about, which FindFirst/FindNext (Dos/SysUtils) and what is the behaviour expected from you? E.g. with:

Sorry, I was too deep into the problem.

I'm using FreeBSD as main platform for development but for the future at least Windows and Linux are targets.

That's a result I'd expect. So I'll go digging for some "Error" or "UnixError" variable.

Thanks so far,
Marc

Still, which FindFirst are you talking about - that one from unit Dos, or that one from unit SysUtils?

.
.

That's a result I'd expect. So I'll go digging for some "Error" or "UnixError" variable.

No, you don't need any "UnixError". Dos.FindFirst returns errors in DosError for all platforms (it's a compatibility unit after all). SysUtils.FindFirst should return the error in its return value directly. I don't know whether the implementation for FreeBSD does it correctly at the moment, but that's how it's supposed to work, at least (=> enter it in the bug tracker if it doesn't do work like that).

Tomas

...

That's a result I'd expect. So I'll go digging for some "Error" or "UnixError" variable.

I did and found the implementation of FindFirst/Next on Win32 differing much from the one in rtl/unix/sysutils.pp. The only system error handled there is ENOMEM, so the error number on "permission denied" is -1 aka unknown.

As far as i understand it the function "glob" in
rtl/unix/sysutils.pp:469 is the place where more error handling for this case should be added. (fpc 2.0.2 release)

Is this assumption correct or am I missing something?

And more: are there other places where handling access rights has to be handled to stay consistent?

TIA,
Marc

I'm talking about sysutils since using unit dos on unix makes little sense imho.

I see, thanks for pointing me there. There *is* a DOS unit on unix, I don't believe it... =:-)

I'll try that first before hacking up the sysutils one.

Thank you!
Marc

Okay,

now I tried it but got no valuable result:

$ ls -lF /usr/X11R6/include/ | grep /
...
drwx------ 104 root wheel 8192 6 Okt 2005 nvu-1.0/ ...

$ ls -lF /usr/X11R6/ | grep include/
drwxr-xr-x  76 root  wheel  10240 14 Mai 00:07 include/

I am allowed to read the last one, but not the first. Your small program gives me:

$ ./doserr /usr/X11R6/include/nvu-1.0/
18
18
18

$ ./doserr /usr/X11R6/include/
18
18
18

Back to the think tank again ...
Marc

If I remember correctly, the DOS FindFirst/Next only handles shortstring-sized path names.

Another "quirk" about using FindNext on Unix is that a symlink to a directory sets the faDirectory flag, and if it happens to be a circular symlink, I don't know of any way tell if it is safe to recurse into it without using fpStat or something like it. FindNext will eventually give up after several loops, but that doesn't seem like a very
elegant way to handle the situation.

- Jeff

Jeff Pohlmeyer wrote

If I remember correctly, the DOS FindFirst/Next only handles shortstring-sized path names.

If possible at all I will not use DOS unit.

This should be no problem in principle, since the unix syscalls detect this state and fail (according to Stevens[1]). But relying on the system working properly is no good policy ...

What I don't understand is why the var "errno" is not set or it's value is destroyed before Findfirst/-Next return. Does this value has to be set explicitely by using fpseterrno()? It returns always -1 in case of permission denied.

Marc

[1] W.Richard Stevens: "Advanced Programming in the Unix Environment" aka "THE Stevens"

IMHO:
On the contrary. If you can't trust the system, then what can you trust ?

What I don't understand is why the var "errno" is not set or it's value is destroyed before Findfirst/-Next return. Does this value has to be set explicitely by using fpseterrno()? It returns always -1 in case of permission denied.

You should not ever rely on SysUtils setting errno. Errno is a variable used in the Unix subsystem.
It is only valid to check errno when directly executing calls from the baseunix,unix calls.

SysUtils calls may or may not destroy this value, but you should not rely on this.

Michael.

Marc Santhoff wrote

Am Mittwoch, den 28.06.2006, 12:17 -0500 schrieb Jeff Pohlmeyer:

.
.

What I don't understand is why the var "errno" is not set or it's value is destroyed before Findfirst/-Next return. Does this value has to be set explicitely by using fpseterrno()? It returns always -1 in case of permission denied.

My opinion is that SysUtils.FindFirst should
return 5 in case of an error; "errno" is not
portable, whereas SysUtils is supposed to support
portable code.

Tomas

Oaky, i can accept this as a guideline for viewing fpc's internals.

My approach to detect failing FindFirst/FindNext would be to do something like this:

<--- rtl/unix/sysutils.pp:549 --->
Function GlobToTSearchRec (Var Info : TSearchRec) : Boolean;
Var SInfo : Stat;
    p     : Pglob;
    GlobSearchRec : PGlobSearchrec;
begin
  GlobSearchRec:=Info.FindHandle;
  P:=GlobSearchRec^.GlobHandle;
  Result:=PNil;
  If Result then
    begin
    GlobSearchRec^.GlobHandle:=P^.Next;
    Result:=Fpstat(GlobSearchRec^.Path+StrPas(p^.name),SInfo)>=0;
+    if NOT Result then
+      begin
+        fpseterrno(??? anything "legal" representing the system error);
+      end;
    Info.PathOnly:=GlobSearchRec^.Path;
    If Result then
</--->

I'd appreciate hints on doing it this way (getting system error number by "fpgetCerrno()"?) or some better technique.

If the errno variants are not reliable it could be done by

1. using an additional out-parameter value
2. changing the result from boolean to interger or longint returning the error number

But both methods would break old code.

Marc

Marc Santhoff wrote

Am Donnerstag, den 29.06.2006, 00:04 +0200 schrieb Michael Van Canneyt:

.
.

If I understand it correctly, GlobToTSearchRec is
an internal function (not available in
interface), and only used in function DoFind.
Changing e.g. its return value shouldn't cause
any harm.

Tomas

(that is not really the problem, OS errorcodes can be hauled with getlastoserror, which is errno on *nix)

Well, it is a problem, because not returning the error code directly in return value violates SysUtils.FindFirst specification and most probably Delphi compatibility too.

Tomas

Depends on what we are talking about. I was replying in the context of the infinite symlink problem, which is a unix specific problem. If you are not, I might have mixed up subthreads.

But in general findfirst should be ok, otoh it should mostly just report success or fail, and not specify too much about the reasons, since that would make findfirst possibily OS specific.

The whole "GLOB" sublevel is not that needed anymore anyway. It was originally meant to avoid a (readdir) syscall per file read, but both freebsd and linux now already buffer at the readdir level (and Darwin, since it uses libc probably too)

.
.

But in general findfirst should be ok, otoh it should mostly just report success or fail, and not specify too much about the reasons, since that would make findfirst possibily OS specific.

Well, I'm still of different opinion. The basic set of error codes should be shared on FindFirst level (BTW as opposed to GetLastError, which clearly returns last OS error and thus is not portable). I consider at least the following to belong to this basic set (in this particular case):

- No more files (18)
- Path not found (3)
- Access denied (5)

I might have forgotten other important ones (writing this from the top of my head on the way from Frankfurt to Bonn).

Tomas

I think so, too. Since nowadays even Windows (2000/XP) has a concept of user permissions, this would be at least nice to have.

(that is not really the problem, OS errorcodes can be hauled with getlastoserror, which is errno on *nix)

I see, I did not know "getlastoserror". But it calls fpGetErrNo in term and this value is not usable:

program unixerr;
uses
 sysutils;
var
 SR: TSearchRec;
begin
 FindFirst (ParamStr ( 1 ), faAnyFile, SR);
 writeln(getlastoserror);
 FindClose (SR);
end.
$ ./unixerr /usr/X11R6/include/
2
$ ./unixerr /usr/X11R6/include/nvu-1.0/
-1
$ ll /usr/X11R6/include/nvu-1.0/
ls: : Permission denied
$ ll -F /usr/X11R6/include/ | grep / | grep nvu
drwx------  104 root  wheel    8192  6 Okt  2005 nvu-1.0/

That brings us back to the start. My only question right now:

How can i detect if sysutils.FindFirst() or FindNext() fails lacking permissions?

TIA,
Marc

Marc Santhoff wrote

How can i detect if sysutils.FindFirst() or FindNext() fails lacking permissions?

As a quick and dirty hack I added an addtitional "stat" on the starting dir for FindFirst and let it bail out early for protecting "ErrNo" ("opendir" gives no reasonable error code on FreeBSD). This in combination with a bug fix in my code using FindFirst solves the problem for me for now. Most important is I'm no longer blocked working on my code.

I'm aware of the fact that there might be problems when hitting an unreadable item while recursing, but for my program I can rule out this case.

This was done on FreeBSD 4.11 with fpc 2.0.2 release.

<diff>
*** sysutils.pp.org	Thu Jun 29 00:06:06 2006
--- sysutils.pp	Fri Jun 30 05:07:22 2006
***************
*** 479,490 ****
--- 479,496 ----
    buffer  : pdirent;
    root,
    current : pglob;
+   SInfo : Stat;
+   res: longint;
+   temp3: string[255];
  begin
  { Get directory }
    temp:=dirname(path);
    if temp='' then
     temp:='.';
+   temp3 := temp+'/.'+#0;
    temp:=temp+#0;
+   res := fpstat(temp3, SInfo);
+   if (res0) then exit(nil);
    thedir:=fpopendir(@temp[1]);
    if thedir=nil then
      exit(nil);
</diff>

Maybe lateron I'll come back on this topic. Did I understand correctly that a new implementation :-

- does not need to defer "readdir" or "stat" calls for being fast enough on modern system implementations

- there could be a check for looping symlinks

- anything else?

Regards,
Marc