Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2016-06

(Unix) file locking

7 messages · 4 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Mark Morgan Lloyd

I'm approaching a situation where up to four identical processes will be reading and writing the same files. There will be process-level synchronisation which should prevent clashes, but I'd still like to add locking at the file level.

I presume that what I should be looking at is Fcntl(F_SetLk) etc., but what is the appropriate API for reading and writing? All access will be in fixed-size (256 byte) blocks, and blocks will be aligned.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]
Mark Morgan Lloyd
to Jonas Maebe
Jonas Maebe wrote 9 lines
On 20/06/16 11:28, Mark Morgan Lloyd wrote: > I'm approaching a situation where up to four identical processes will be > reading and writing the same files. There will be process-level > synchronisation which should prevent clashes, but I'd still like to add > locking at the file level. If you use sysutils.fileopen, file locking will automatically be used based on fmShare* constants you specify: http://www.freepascal.org/docs-html/current/rtl/sysutils/fileopen.html

I think I should have said that I needed locking at the record level, where each record is 256 bytes. When I referred to "file" I meant that this was the low-level storage layer, with several other layers on top of it; records will be written and read in indeterminate order, the file won't be written by one process and read by the rest.

Each of the four programs (assumed to be running on the same system, i.e. no NFS etc.) will be managing its own advisory locking using shared memory. However for cases where one of them erroneously tries to ignore the custom advisory lock I want to be able to lock individual records in the common file.

The file represents a mainframe disk subsystem, around which up to four systems operated. Each system had two CPUs, with a multitasking OS. There's a lot to go wrong...

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]
Björn Lundin
to Mark Morgan Lloyd: I'm approaching a situation where up to four identical processes will be reading and…
Mark Morgan Lloyd wrote 9 lines
I'm approaching a situation where up to four identical processes will be reading and writing the same files. There will be process-level synchronisation which should prevent clashes, but I'd still like to add locking at the file level. I presume that what I should be looking at is Fcntl(F_SetLk) etc., but what is the appropriate API for reading and writing? All access will be in fixed-size (256 byte) blocks, and blocks will be aligned.

on Linux and AIX I use lockf.
then read() and write() from the os.
However I lock the whole file, but the manpage says it is possible to lock parts of it.

Here some relevant code (in Ada - sorry) but it's the interesting parts

    declare
      C_Name : Chars_Ptr := New_String (To_String(A_Lock.Name));
    begin
      A_Lock.Fd := Posix.Open(C_name, O_WRONLY + O_CREAT , 8#644#);
      --O_NONBLOCK is not needed
      Free(C_Name);
    end;
-- test lock, and lock if unlocked, error if already locked
    Result := Posix.Lockf(A_Lock.Fd, F_TLOCK, 0);

-- the last parameter above says how much to lock; 0 = whole file else seek() and then lock som bytes

    if Result = -1 then
      Log(Me & "Take", "Take lock failed, Errno =" & Errno'Img);
      raise Lock_Error with "Errno =" & Errno'Img ;
    end if;
  -- file is now locked
    --put pid in file
    declare
      use Interfaces.C;
      use Calendar2;
      Str : String := Posix.Getpid'img  & Ascii.LF);
      C_Pid_Str : Chars_Ptr := New_String (Str);
      Size      : Posix.Size_t;
    begin
      Size := Posix.Write(A_Lock.Fd, C_Pid_Str, Str'Length);
      Free(C_Pid_Str);
      if Integer(Size) = -1 then
        Log(Me & "Take", "write pid Errno =" & Errno'Img);
      end if;
      if Integer(Size) /= Str'length then
        Log(Me & "Take", "Size/str'length =" & Size'Img &
                        "/" & Str'Length'Img);
      end if;
    end;

to unlock it later

      Result := Posix.Lockf(A_Lock.Fd, F_ULOCK, 0); --unlock
or just close it
      Result := Posix.Close(A_lock.Fd);
-- Björn
Fabio Luis Girardi
to Björn Lundin

2016-06-20 18:15 GMT-03:00 Björn Lundin :

Show quoted text 9 lines
on Linux and AIX I use lockf. then read() and write() from the os. However I lock the whole file, but the manpage says it is possible to lock parts of it.

This can be used to lock character devices, like /dev/ttyUSB0?

FPC fpFLock does the same than Posix.Lockf?

The best regards, Fabio Luis Girardi PascalSCADA Project http://sourceforge.net/projects/pascalscada http://www.pascalscada.com
Björn Lundin
to Fabio Luis Girardi
Fabio Luis Girardi wrote 16 lines
2016-06-20 18:15 GMT-03:00 Björn Lundin < >: on Linux and AIX I use lockf. then read() and write() from the os. However I lock the whole file, but the manpage says it is possible to lock parts of it. This can be used to lock character devices, like /dev/ttyUSB0? FPC fpFLock does the same than Posix.Lockf?

that I do not know.
from 'man lockf'
DESCRIPTION
Apply, test or remove a POSIX lock on a section of an open file. The file is specified by fd, a file descriptor open for writing, the action by cmd, and the section consists of byte positions pos..pos+len-1 if len is positive, and pos-len..pos-1 if len is negative, where pos is the current file position, and if len is zero,the section extends from the current file position to infinity, encompassing the present and future end-of-file positions. In all cases, the section may extend past current end-of-file.

It does not specify underlaying device - but you need an open file descriptor

Show quoted text 14 lines
-- The best regards, Fabio Luis Girardi PascalSCADA Project http://sourceforge.net/projects/pascalscada http://www.pascalscada.com _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
-- Björn
Mark Morgan Lloyd
to Björn Lundin
Björn Lundin wrote 5 lines
On 2016-06-21 00:37, Fabio Luis Girardi wrote:> > > 2016-06-20 18:15 GMT-03:00 Björn Lundin >:> > > on Linux and AIX I use lockf.> then read() and write() from the os.> However I lock the whole file, but the manpage says it is possible to> lock parts of it.> > > > This can be used to lock character devices, like /dev/ttyUSB0?> > FPC fpFLock does the same than Posix.Lockf? that I do not know.from 'man lockf'DESCRIPTIONApply, test or remove a POSIX lock on a section of an open file. Thefile is specified by fd, a file descriptor open for writing, the actionby cmd, and the section consists of byte positions pos..pos+len-1 iflen is positive, and pos-len..pos-1 if len is negative, where pos isthe current file position, and if len is zero,the section extends fromthe current file position to infinity, encompassing the present andfuture end-of-file positions. In all cases, the section may extendpast current end-of-file. It does not specify underlaying device - but you need an open filedescriptor

The interesting question is what functions that translates to in Pascal (specifically, FPC). I suspect FpFcntl() and FpRead() etc. rather than anything higher-level (e.g. streams).

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]