Hi all,

I'm trying to write a small utility program to clean up a LOT of files on a network drive. This is using Debian 7.2, and a recent svn version of Lazarus (I actually use fpcup to do the downloads and builds). The information on the 'About' page is version 1.3, date 2014-02-12, fpc version 2.6.3, svn revision 44027, x86_64-linux-gtk 2

This may well be a lack of experience with Linux programming on my part, but it has me baffled. The program I've written needs to do a bunch of chmods and/or deletes on the files on the network drive. Stepping through with the debugger, some, but not all, of the attempted chmods return a status of -1. Some, but not all, of the calls to sysutils.deletefile return false. What is puzzling me is that if, instead of calling fpchmod and sysutils.deletefile I instead call fpsystem and do the operations that way, it works. I can also run up a command prompt and do the failed operations that way with no problem.

The files on the network drive are all owned by me, the chmod is trying to set permissions to 644 rather than 777. The delete is just deleting a whole bunch of log files which got mixed up in there when the data was copied to the drive.

I have no idea why the above should happen, it makes no sense to me, but as I said, I'm still a relative novice as regards coding for Linux. Everything is being run from the same account. Can anybody point me in the right direction?

Thanks,

Brian.

Is it reproducable on the same file?
Can you give an example file name?
What kind of network share?

Special characters, wrong encoding, server lock, faulty network.

Mattias

On the basis of two examples of each, yes.

Can you give an example file name?

Two of the files which failed to delete are unknown.mp4 and video_ts.vob.

Two files where the attempt to chmod them failed are The_Buddhas_Instruction_to_his_son.mp4 and
Yogis_Of_Tibet_Rare_Documentary.mp4

In all cases, the full paths are made up of
['A'..'Z','a'..'z','0'..'9','_'] only

(well, also '/' and '.', obviously!)

What kind of network share?

It's a Western Digital 4TB "My Cloud" drive, with the shared /nfs directory on the drive mounted to a root-level directory on my PC.

Nobody else is using the drive - I've only two PCs on the LAN, and my wife's machine was switched off. The drive is hanging off one of the ports of my four-port Linksys router.

Special characters, wrong encoding, server lock, faulty network.

Definitely not the first. I've written out the full paths to a log file, and they are correct - editing the logs and putting an 'ls ' in front of the file names provides the expected results. The drive is powered down overnight, and I'm the only user, so I can't see how there could be a server lock unless there's something wrong with the drive. I've seen nothing else to indicate a faulty router.

I know this all sounds crazy. I wouldn't have believed it had I not been seeing it for myself. The only thing I could come up with (and the reason for my post here) was that single-stepping under the debugger somehow caused the problem.

I'm not asking anybody to do any amount of research on this - it's trivial for me to tweak the program so that it writes commands to a file rather than tries to run the chmods and deletes, and I can just run the resulting file from the CLI.

Brian.

Does that mean, it only appears when single stepping in the debugger?

Have you checked fpgeterrno?

Mattias

Check permissions/ownership of those files, and ownership/permissions of the directory containing them. Oh, and make sure that they're not actually symlinks to something unexpected, and that nothing's got them open (lsof command).

SAMBA share?
IIRC there is some problem/bug with SMB that can cause problems with fpc.

Bart

On 2/19/2014 8:42 AM, brian wrote:
[...]

have you tried adding at least a 1/4th second up to maybe a 1 second sleep just to try to ensure that it isn't a bottleneck somewhere? it could be the samba stuff... my XP machine used to drop its samba shares on a FreeNAS (bsd) any time there was a huge amount of traffic (copying several hundred megs of randomly sized small files, copying one huge [up to 2G] file, and similar) between the XP box and the NAS... the only way to regain access to the NAS was to reboot the XP box... no other machines exhibited this problem... it was finally solved when the XP box ate itself for lunch one day :/

Bart wrote

SAMBA share? IIRC there is some problem/bug with SMB that can cause problems with fpc.

Yes. AFAIR, it was fixed at least in FPC trunk. I vaguely remember it was about findfirst/findnext not returning all files in the directory, rather than chmod/sysutils.deletefile calls... but who knows...

- Findfirst bug
http://bugs.freepascal.org/view.php?id=23732
- there was another bug IIRC from a guy with a NAS as well, yes found it: http://bugs.freepascal.org/view.php?id=22423
... though it links to the previous issue

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4273 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20140221/e3d35f9e/attachment.bin>

I'm sorry for the delay in replying, folks. Programming is now just a hobby for me, and other things got in the way.

Mattias:

I was single stepping in the debugger just to check I'd not made any mistakes in the code - I didn't have the nerve to run the program in its entirety yet, it would be a lot of work if I'd made a mistake somewhere and had to rebuild all the network drive stuff.

What I did was to write a small test application, all it is is a button on a form (I wanted to stick with the GUI, as that's what the real program is) and the OnClick tries to chmod and delete those two pairs of files which I mentioned above. I compiled this and then ran it outside of the IDE (I can send you the project if you wish, but it really is that simple). The program worked just fine.

Mark:

The directory and file ownerships, permissions etc was the first thing I checked, they're not symlinks, and no, nothing else had them open.

Bart and Reinier:

It's an NFS share, no Samba involved.

Waldo:

I think you've got the answer. I added the suggested delay (0.25 second) before each of the chmod and delete operations, and while the deletions are done, I let the chmods all run again, and got no errors whatsoever. It does begin to look like the drive couldn't keep pace with it all. I don't have enough experience with this sort of configuration to know whether that's something to expect or whether it's a problem with my particular setup, but since I now have three workrounds that work (add a delay, use an fpSystem call instead or write a script) then that's fine by me.

Thanks for the suggestions, all.

Brian.

Thanks for the assistance, folks.