Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2016-01

Random thread-safe

16 messages · 7 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Ondrej Pokorny

Regarding: http://bugs.freepascal.org/view.php?id=29526 (because I cannot answer on closed issues).

> Also: the entire state of the random number generator consists of

regular global variables, so it is not safe to use it from multiple threads in parallel.

Would it make sense to you to make random thread-safe? E.g. with the use of "threadvar" instead of "var" for the global variables in question?

Ondrej

Reply by email

Jonas Maebe
to Ondrej Pokorny

Ondrej Pokorny wrote on Thu, 28 Jan 2016:

Show quoted text 10 lines
Regarding: http://bugs.freepascal.org/view.php?id=29526 (because I cannot answer on closed issues). >> Also: the entire state of the random number generator consists of >> regular global variables, so it is not safe to use it from >> multiple threads in parallel. Would it make sense to you to make random thread-safe? E.g. with the use of "threadvar" instead of "var" for the global variables in question?

No. The Mersenne twister has a lot of state (about 2KB). Duplicating this for every thread, along with associated slowdown, is not worth it. It's better to use a class that encapsulates the state of a random number generator and then instantiate this class for every thread.

Jonas

Reply by email

Unknown author
to Jonas Maebe

Then wouldn't it be possible to make PRNG's plugable. The Mersenne twister is still good as it is but definitely not suitable for every and any case. Marsialigla's might be a better choice nowadays. And the seeding can nowadays often been done from reading hardware random, like on the Raspberry Pi.
e.g.
http://scruss.com/blog/2013/06/07/well-that-was-unexpected-the-raspberry-pis-hardware-random-number-generator/

But that is not an exclusive.

Thaddy

Jonas Maebe wrote 24 lines
Ondrej Pokorny wrote on Thu, 28 Jan 2016: > Regarding: http://bugs.freepascal.org/view.php?id=29526 (because I > cannot answer on closed issues). > >>> Also: the entire state of the random number generator consists of >>> regular global variables, so it is not safe to use it from multiple >>> threads in parallel. > > Would it make sense to you to make random thread-safe? E.g. with the > use of "threadvar" instead of "var" for the global variables in > question? No. The Mersenne twister has a lot of state (about 2KB). Duplicating this for every thread, along with associated slowdown, is not worth it. It's better to use a class that encapsulates the state of a random number generator and then instantiate this class for every thread. Jonas _______________________________________________ fpc-devel maillist - http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Reply by email

Jonas Maebe
to Unknown author: Then wouldn't it be possible to make PRNG's plugable. The Mersenne twister is still good…

thaddy wrote on Thu, 28 Jan 2016:

Show quoted text 7 lines
Then wouldn't it be possible to make PRNG's plugable. The Mersenne twister is still good as it is but definitely not suitable for every and any case. Marsialigla's might be a better choice nowadays. And the seeding can nowadays often been done from reading hardware random, like on the Raspberry Pi. e.g. http://scruss.com/blog/2013/06/07/well-that-was-unexpected-the-raspberry-pis-hardware-random-number-generator/

The random number generator is just there as a general purpose tool, for TP/Delphi functional compatibility (not implementation compatibility, as we use a different PRNG). Unlike functionality such as the memory manager and code page conversions, it is not used by other parts of the language or RTL. As a result, there is no need to complicate the system unit by implementing a complete plugin infrastructure for this.

Jonas

Reply by email

Michael Van Canneyt
to Jonas Maebe
Jonas Maebe wrote 17 lines
thaddy wrote on Thu, 28 Jan 2016: > Then wouldn't it be possible to make PRNG's plugable. The Mersenne twister > is still good as it is but definitely not suitable for every and any case. > Marsialigla's might be a better choice nowadays. > And the seeding can nowadays often been done from reading hardware random, > like on the Raspberry Pi. > e.g. > http://scruss.com/blog/2013/06/07/well-that-was-unexpected-the-raspberry-pis-hardware-random-number-generator/ The random number generator is just there as a general purpose tool, for TP/Delphi functional compatibility (not implementation compatibility, as we use a different PRNG). Unlike functionality such as the memory manager and code page conversions, it is not used by other parts of the language or RTL. As a result, there is no need to complicate the system unit by implementing a complete plugin infrastructure for this.

Indeed.
You can just write and use your own random number mechanism if this is so important. read /dev/urandom or so...

Michael.

Reply by email

Unknown author
to Michael Van Canneyt

The point is partially more or less at the end of the link I included. The reference to the Rdrand
<http://software.intel.com/en-us/articles/user-manual-for-the-rdrand-library-linux-version> Which means software should take advantage of hardware when possible. But it is about more than that.

And with due regards: the choice of the Mersenne twister over a Delphi equivalent one was made by about the same arguments I have now: it was better.

This is not that I disagree with your initial answers (also from Jonas) but to put things in an historical perspective.
You were both involved at the time the Mersenne twister was chosen , I believe. Correct me if I am wrong.

PRNG's are more and more important. They deserve extra effort even if it is not used for the compiler itself.
(It may very well be: when all software needs signing on all major platforms, which is tomorrow in IT standards)

Regards,

Thaddy

Michael Van Canneyt wrote 31 lines
On Thu, 28 Jan 2016, Jonas Maebe wrote: > > thaddy wrote on Thu, 28 Jan 2016: > >> Then wouldn't it be possible to make PRNG's plugable. The Mersenne >> twister is still good as it is but definitely not suitable for every >> and any case. Marsialigla's might be a better choice nowadays. >> And the seeding can nowadays often been done from reading hardware >> random, like on the Raspberry Pi. >> e.g. >> http://scruss.com/blog/2013/06/07/well-that-was-unexpected-the-raspberry-pis-hardware-random-number-generator/ > > The random number generator is just there as a general purpose tool, > for TP/Delphi functional compatibility (not implementation > compatibility, as we use a different PRNG). Unlike functionality such > as the memory manager and code page conversions, it is not used by > other parts of the language or RTL. As a result, there is no need to > complicate the system unit by implementing a complete plugin > infrastructure for this. Indeed. You can just write and use your own random number mechanism if this is so important. read /dev/urandom or so... Michael. _______________________________________________ fpc-devel maillist - http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Reply by email

Jonas Maebe
to Unknown author

thaddy wrote on Thu, 28 Jan 2016:

Show quoted text 9 lines
The point is partially more or less at the end of the link I included. The reference to the Rdrand <http://software.intel.com/en-us/articles/user-manual-for-the-rdrand-library-linux-version> Which means software should take advantage of hardware when possible. But it is about more than that. And with due regards: the choice of the Mersenne twister over a Delphi equivalent one was made by about the same arguments I have now: it was better.

There are different kinds of "better". E.g. the current one is the same on all platforms, which means that by setting a particular randseed, you can get the same sequence on all platforms (e.g. for simulations). If you start using platform-specific functionality, that is no longer the case. Depending on the implementation, they may not even support getting the same sequence of numbers again on the same platform by setting randseed to a particular value, which is a basic feature of the TP/Delphi PRNG that many programs depend on.

PRNG's are more and more important. They deserve extra effort even if it is not used for the compiler itself. (It may very well be: when all software needs signing on all major platforms, which is tomorrow in IT standards)

If you're using a PRNG for security purposes, you're doing security wrong. RNGs suitable for security sensitive applications belong in their own unit or set of units, along with the encryption algorithms.

Jonas

Reply by email

Sven Barth
to Unknown author: The point is partially more or less at the end of the link I included. The reference to…
"" wrote
PRNG's are more and more important. They deserve extra effort even if it

is not used for the compiler itself.

(It may very well be: when all software needs signing on all major

platforms, which is tomorrow in IT standards)

That's not the usecase that Random() is designed to cover. If one wants a more sophisticated random number generation then this should be in a new unit that could be in the FCL... Contributions for this are welcome, but Random() will stay as it is.

Regards,
Sven

Reply by email

Mark Morgan Lloyd
to Jonas Maebe: Ondrej Pokorny wrote on Thu, 28 Jan 2016: No. The Mersenne twister has a lot of state…
Jonas Maebe wrote 11 lines
>>> Also: the entire state of the random number generator consists of >>> regular global variables, so it is not safe to use it from multiple >>> threads in parallel. > > Would it make sense to you to make random thread-safe? E.g. with the > use of "threadvar" instead of "var" for the global variables in question? No. The Mersenne twister has a lot of state (about 2KB). Duplicating this for every thread, along with associated slowdown, is not worth it. It's better to use a class that encapsulates the state of a random number generator and then instantiate this class for every thread.

Could I ask for clarification of this please. Are you saying that Random() will crash if called simultaneously by multiple threads, or that it will return suboptimal results?

If, as an example, I have two threads doing network or comms jobs and I want to introduce jitter in the retries, does that mean that I have to put Random() into a critical section?

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email

Jonas Maebe
to Mark Morgan Lloyd
Mark Morgan Lloyd wrote
Could I ask for clarification of this please. Are you saying that Random() will crash if called simultaneously by multiple threads, or that it will return suboptimal results?

It's undefined. The current implementation won't crash, but the results will indeed be suboptimal.

If, as an example, I have two threads doing network or comms jobs and I want to introduce jitter in the retries, does that mean that I have to put Random() into a critical section?

Yes.

Jonas

Reply by email

Mark Morgan Lloyd
to Jonas Maebe
Jonas Maebe wrote 13 lines
Mark Morgan Lloyd wrote: > Could I ask for clarification of this please. Are you saying that > Random() will crash if called simultaneously by multiple threads, or > that it will return suboptimal results? It's undefined. The current implementation won't crash, but the results will indeed be suboptimal. > If, as an example, I have two threads doing network or comms jobs and I > want to introduce jitter in the retries, does that mean that I have to > put Random() into a critical section? Yes.

Thanks Jonas, noted.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email

Jonas Maebe
to Ondrej Pokorny

Ondrej Pokorny wrote on Fri, 29 Jan 2016:

Show quoted text 5 lines
On 28.01.2016 13:30, Jonas Maebe wrote: > Mersenne twister Should I be worried about the GPL-2 license of the Mersenne Twister that is cited in system.inc?

It's LGPL, not GPL.

Jonas

Reply by email

Jonas Maebe
to Jonas Maebe: Ondrej Pokorny wrote on Fri, 29 Jan 2016: It's LGPL, not GPL. Jonas

Jonas Maebe wrote on Fri, 29 Jan 2016:

Show quoted text 9 lines
Ondrej Pokorny wrote on Fri, 29 Jan 2016: > On 28.01.2016 13:30, Jonas Maebe wrote: >> Mersenne twister > > Should I be worried about the GPL-2 license of the Mersenne Twister > that is cited in system.inc? It's LGPL, not GPL.

There is however the problem that it doesn't seem to be licensed with the static linking exception, which I apparently missed back in the day. I'll replace it with a Pascal translation of the public domain implementation available at https://github.com/dajobe/libmtwist

Jonas

Reply by email