Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2013-02

Configuration file handling

9 messages · 5 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Mark Morgan Lloyd

If multiple, related, programs running on unix (Linux, Solaris etc.) refer to the same configuration file accessed via a TIniFile, is there any recommended "good practice" to ensure that they don't try to update it simultaneously?

In the case that I'm thinking of, I don't anticipate e.g. multiple logins trying to write setup information at the same time. The more risky case is if there are e.g. multiple daemons or apps trying to save state when a UPS signals imminent shutdown.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email

Andrew Brunner
to Mark Morgan Lloyd

I would not use INI if the systems are writing data. While I've had plenty of success reading INI files in parallel - I seem to recall that the file is just streamed out without regard to which version is on disk. I think the file is completely refreshed.

I recommend using a DBMS system of some sort for read/write ops in parallel.

Mark Morgan Lloyd wrote 10 lines
If multiple, related, programs running on unix (Linux, Solaris etc.) refer to the same configuration file accessed via a TIniFile, is there any recommended "good practice" to ensure that they don't try to update it simultaneously? In the case that I'm thinking of, I don't anticipate e.g. multiple logins trying to write setup information at the same time. The more risky case is if there are e.g. multiple daemons or apps trying to save state when a UPS signals imminent shutdown.
Andrew Brunner Aurawin LLC 15843 Garrison Circle Austin, TX 78717 https://aurawin.com Aurawin is a great new way to store, share, and explore all your content featuring our innovative cloud social computing platform.

Reply by email

Mark Morgan Lloyd
to Andrew Brunner
Andrew Brunner wrote 19 lines
On 02/15/2013 04:54 PM, Mark Morgan Lloyd wrote: > If multiple, related, programs running on unix (Linux, Solaris etc.) > refer to the same configuration file accessed via a TIniFile, is there > any recommended "good practice" to ensure that they don't try to > update it simultaneously? > > In the case that I'm thinking of, I don't anticipate e.g. multiple > logins trying to write setup information at the same time. The more > risky case is if there are e.g. multiple daemons or apps trying to > save state when a UPS signals imminent shutdown. > I would not use INI if the systems are writing data. While I've had plenty of success reading INI files in parallel - I seem to recall that the file is just streamed out without regard to which version is on disk. I think the file is completely refreshed. I recommend using a DBMS system of some sort for read/write ops in parallel.

Not data as such, but e.g. what windows are open. I definitely agree that data being processed should be in a competent database.

Things like database server name should obviously be saved as soon as they're established, i.e. not left until the program shuts down.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email

Sven Barth
to Mark Morgan Lloyd: If multiple, related, programs running on unix (Linux, Solaris etc.) refer to the same…
Mark Morgan Lloyd wrote 10 lines
If multiple, related, programs running on unix (Linux, Solaris etc.) refer to the same configuration file accessed via a TIniFile, is there any recommended "good practice" to ensure that they don't try to update it simultaneously? In the case that I'm thinking of, I don't anticipate e.g. multiple logins trying to write setup information at the same time. The more risky case is if there are e.g. multiple daemons or apps trying to save state when a UPS signals imminent shutdown.

This might be for Windows, but I think this applies here as well: http://blogs.msdn.com/b/oldnewthing/archive/2004/11/22/267890.aspx

Regards,
Sven

Reply by email

Reinier Olislagers
to Sven Barth
Sven Barth wrote 14 lines
On 15.02.2013 23:54, Mark Morgan Lloyd wrote: > If multiple, related, programs running on unix (Linux, Solaris etc.) > refer to the same configuration file accessed via a TIniFile, is there > any recommended "good practice" to ensure that they don't try to update > it simultaneously? > > In the case that I'm thinking of, I don't anticipate e.g. multiple > logins trying to write setup information at the same time. The more > risky case is if there are e.g. multiple daemons or apps trying to save > state when a UPS signals imminent shutdown. > This might be for Windows, but I think this applies here as well: http://blogs.msdn.com/b/oldnewthing/archive/2004/11/22/267890.aspx

Security problems with world-writable files? Yes, that problem applies to *nix as well but Mark has given no indication his config files are world-writable. (Even if the logins Mark speaks of run under different user accounts, there's such a thing as groups on *nix, too :) )

UPS shutdown: no experience with that - I'd expect the process to just get a regular kill signal; perhaps the OS/UPS driver provides dbus (or other messaging layer) notifications.

If the data is not that important, wouldn't a naive approach like a loop that
- repeats say max 3x,
- tries to save the data,
- catching exceptions coming from TIniFile, then waiting a random small amount of time
- giving up after those 3x
help?

Obviously, this problem would be alleviated if you run under multiple different accounts (normal applications) and properly save settings somewhere in the user's ~ directory rather than a systemwide path (which the FPC getconfig... forgot exactly supports).

Reply by email

Marco van de Voort
to Mark Morgan Lloyd: If multiple, related, programs running on unix (Linux, Solaris etc.) refer to the same…

In our previous episode, Mark Morgan Lloyd said:

Show quoted text 7 lines
If multiple, related, programs running on unix (Linux, Solaris etc.) refer to the same configuration file accessed via a TIniFile, is there any recommended "good practice" to ensure that they don't try to update it simultaneously? In the case that I'm thinking of, I don't anticipate e.g. multiple logins trying to write setup information at the same time

That sounds like the domain of of filelocking (fp)Flock.

The more risky case is if there are e.g. multiple daemons or apps trying to save state when a UPS signals imminent shutdown.

Have all systems regularly write local files in something that is not cleared on startup. (e.g. change clearing /tmp to a monthly cronjob instead of startup)

On startup check for crash (local file exists) and if the local file is newer than the server, copy and continue starting up.

Reply by email

Mark Morgan Lloyd
to Reinier Olislagers: Security problems with world-writable files? Yes, that problem applies to *nix as well…
Reinier Olislagers wrote 19 lines
On 16-2-2013 9:09, Sven Barth wrote: > On 15.02.2013 23:54, Mark Morgan Lloyd wrote: >> If multiple, related, programs running on unix (Linux, Solaris etc.) >> refer to the same configuration file accessed via a TIniFile, is there >> any recommended "good practice" to ensure that they don't try to update >> it simultaneously? >> >> In the case that I'm thinking of, I don't anticipate e.g. multiple >> logins trying to write setup information at the same time. The more >> risky case is if there are e.g. multiple daemons or apps trying to save >> state when a UPS signals imminent shutdown. >> > This might be for Windows, but I think this applies here as well: > http://blogs.msdn.com/b/oldnewthing/archive/2004/11/22/267890.aspx Security problems with world-writable files? Yes, that problem applies to *nix as well but Mark has given no indication his config files are world-writable. (Even if the logins Mark speaks of run under different user accounts, there's such a thing as groups on *nix, too :) )

[Checks link] I agree. The specific case I'm thinking of is where a number of related daemons running (after any low-numbered sockets have been grabbed) as the same user. Since they will be referring to the same database servers and probably to common tables it makes sense to use a common configuration file, this also makes it much easier since somebody setting up a new collaborating system can be sent a template for the single .ini required.

I think I'm leaning towards an arrangement where important stuff (database names etc.) is written to the common .ini as soon as it is entered into whichever program is dominant, but less-important stuff (window sizes etc.) which might be updated at termination is kept in its own file. There's still a risk that multiple instances of the same program could interfere with each other, but the worst case result will be messed up screen layout and it will always be fixable by deleting the relevant .ini.

Show quoted text 17 lines
UPS shutdown: no experience with that - I'd expect the process to just get a regular kill signal; perhaps the OS/UPS driver provides dbus (or other messaging layer) notifications. If the data is not that important, wouldn't a naive approach like a loop that - repeats say max 3x, - tries to save the data, - catching exceptions coming from TIniFile, then waiting a random small amount of time - giving up after those 3x help? Obviously, this problem would be alleviated if you run under multiple different accounts (normal applications) and properly save settings somewhere in the user's ~ directory rather than a systemwide path (which the FPC getconfig... forgot exactly supports).

The notable thing about UPS shutdowns is that in at least some cases a system first signals term then after five seconds signals kill. I think that initial term is going to be problematic where shared data is involved, since multiple programs can receive it simultaneously.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email

Mark Morgan Lloyd
to Marco van de Voort: In our previous episode, Mark Morgan Lloyd said: That sounds like the domain of of…
Marco van de Voort wrote 10 lines
In our previous episode, Mark Morgan Lloyd said: > If multiple, related, programs running on unix (Linux, Solaris etc.) > refer to the same configuration file accessed via a TIniFile, is there > any recommended "good practice" to ensure that they don't try to update > it simultaneously? > > In the case that I'm thinking of, I don't anticipate e.g. multiple > logins trying to write setup information at the same time That sounds like the domain of of filelocking (fp)Flock.

Agreed. I've already subclassed TIniFile so I can do things like parsing array- and record-like settings, is there any way of hooking the file open/close to insert locks?

Show quoted text 9 lines
> The more risky case is if there are e.g. multiple daemons or apps trying > to save state when a UPS signals imminent shutdown. Have all systems regularly write local files in something that is not cleared on startup. (e.g. change clearing /tmp to a monthly cronjob instead of startup) On startup check for crash (local file exists) and if the local file is newer than the server, copy and continue starting up.

Provided that it's just unimportant stuff- e.g. the visual appearance of a program controlling a daemon- it's probably not worth updating.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email

Andrew Brunner
to Reinier Olislagers: Security problems with world-writable files? Yes, that problem applies to *nix as well…
Reinier Olislagers wrote 6 lines
Security problems with world-writable files? Yes, that problem applies to *nix as well but Mark has given no indication his config files are world-writable. (Even if the logins Mark speaks of run under different user accounts, there's such a thing as groups on *nix, too :) )

My only concern for using Ini files is invalid data because the entire Ini will stream out and replace the file. If one thread has data that differs from another the order of writing becomes important. If all data refers to the same user, make sure your data is stored in an offshoot of the appropriate sandbox. I think the method getAppConfigDir might prove useful if this is they way you decide to go...

Reply by email