Hi,

A Russian user raised the issue in the fpGUI newsgroups... fpGUI uses UTF-8 as the internal string encoding. He noticed that the File Dialog which displays the file sizes with thousand separators were totally blank. On further investigation he noticed that it was FormatFloat() that caused the issue. FormatFloat uses the ThousandSeparator locale variable.

In FPC the ThousandSeparator is of type Char which can only hold one byte. Yet the Russian locale uses the non-breaking space character as expressed in UTF-8 as 'C2 A0' (bytes) and takes up 2 bytes. So how do we assign the Russian ThousandSeparator (U+00A0) in FPC to the ThousandSeparator variable?

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

In fpGUI the rsThousandSeparator contains the locale specific character in UTF-8 encoding.

Also ThousandSeparator := Utf8toAnsi(rsThousandSeparator) is not compilable because Utf8ToAnsi() returns a string, not a Char. And simply casting the result to a Char might not be a good idea in all cases.

This issue obviously affects more that just the FormatFloat() function. It affects any functions using FPC locale variables defined as Char and the UTF8toAnsi() result is more that one byte.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Op Tue, 29 Jul 2008, schreef Graeme Geldenhuys:

As a workaround, it can be converted into a normal breaking space. There is no proper solution, MBCS requires it to be a string rather than a char, but compatibility requires it to be a char. Which means you are limited to SBCS compatible thousand separators.

Daniël

Daniël Mantione wrote

is no proper solution, MBCS requires it to be a string rather than a char, but compatibility requires it to be a char. Which means you are

Isn't a string backward compatible with a Char?

Micha

Op Tue, 29 Jul 2008, schreef Micha Nelissen:

No. A char can be automatically typeconverted to a string, but they are not compatible types.

Daniël

Daniël Mantione wrote

As a workaround, it can be converted into a normal breaking space. There is no proper solution, MBCS requires it to be a string rather than a char, but compatibility requires it to be a char. Which means you are limited to SBCS compatible thousand separators.

This is what the Russian user had to revert to, using a normal $20 (space) character. But seeing that Delphi is now going to be fully Unicode compliant, surely we need to attend to these issues as well in FPC - being fully Unicode compliant.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

No. You can't typecast or ORD() it anymore, or subtract other chars from it.

Micha Nelissen wrote

Isn't a string backward compatible with a Char?

I don't understand your question? Ansi Char is always 1 bytes. A UTF-8 character can be anything from 1-4 bytes. The non-breaking spaces is such a case, being 2 bytes and the ThousandSeparator variable only accepting 1 byte.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

And just like Delphi, I think it is better to wait to a major version with proper unicode string support, and do it (mostly) right in one go, instead of little ad-hoc changes to support workarounds.

Op Tue, 29 Jul 2008, schreef Graeme Geldenhuys:

Delphi will do char=widechar, which will solve this. It is likely FPC will follow, but that doesn't change anything for the situation which is that we have a sysutils unit that has not been designed for UTF-8 usage.

Daniël

I did not suggest ad-hoc changes, I meant FPC should fully support unicode strings as Delphi is doing in the next release. Unicode is not something new, it's just Delphi that is way behind with their implementation compared to the demand, and FPC now seems to be in the same boat. I don't see any point in waiting for Delphi, after all, we may NOT look at their implementation anyway!

Is there any FPC discussions as to how this is going to be tackled and what implications there are etc..? Anything I can join? Any timeline for developers to expect unicode support? Anything we can help test or write unit tests for?

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Op Tue, 29 Jul 2008, schreef Graeme Geldenhuys:

It depends on implementation of new widechar based string types, which will be supported.

The developers haven't talked about it yet, but I can imagine we will have some target platforms where sizeof(char)=1, which would provide for 100% compatibility with old code and some platforms where sizeof(char)=2, which will provide the unicode support for the future.

For the time being you can prepare your own code by adding:

type  char=widechar;
       string=widestring;

... at the top of your files

By the way, I do believe we need to be Delphi compatible here and thus know their implementation. Code exchange between Delphi & FPC otherwise becomes too hard for people.

Daniël

No, but we can try to keep compability.

Is there any FPC discussions as to how this is going to be tackled and what implications there are etc..?

There have been, but they are now postponed till we have real Tiburon usage data.

Anything I can join? Any timeline for developers to expect unicode support?

When it is ready (tm). Probably it will also depend on the plans for the 2.3.x branch, and how intrusive the new string type will be. If it is very intrusive it might be post 2.4.

I don't think it is realistic to expect anything releaseworthy in the next year.

Anything we can help test or write unit tests for?

Yes. You can make tests using widestrings. When it is working, translate all literals to utf-8. Do so for preferably all sysutils,dateutils and strutils string routines, but start with the more important ones. Probably you can see if there are already routines testing this for ansistring in the testsuite and build on that.

It will help me/us to start making UTF-8,16 versions of the RTL routines. That work is parallelizable with the compiler work, and these will have to be done anyway sooner or later, and fairly independant of the exact outcome of the string types.

Daniël Mantione wrote

Delphi will do char=widechar, which will solve this. It is likely FPC will follow, but that doesn't change anything for the situation which is that we have a sysutils unit that has not been designed for UTF-8 usage.

So is anybody working on unicode support in the RTL or sysutils unit? I don't see why we need to be waiting for Delphi, we may not look at their implementation anyway. And in the mean time FPC developers like myself need to keep hacking their code to be unicode compatible, when unicode support should be in FPC directly.

fpGUI, Lazarus, MSEGUI etc all have their own unicode helper functions for string manipulation, file access handling etc.... All this should actually be in FPC itself, not duplicated over and over in user applications and frameworks.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

MSEgui has a widestring version of the FormatFloat function (lib/common/kernel/mseformatstr.pas formatfloatmse). There were no bug reports from Russian users, so it seems to work, although I did not test the U+00A0 ThousandSeparator...

Martin

So FPC plans to always be worse off than Delphi. :-( I really think playing 'catchup' with somebody like Delphi is not a good thing. They have different goals as far as I can see, plus their future doesn't look bright (for a very long time now). Delphi tries to compete with Microsoft using Microsoft's own tool (.NET). We all know how that turns out when anybody tries to compete with Microsoft. Because of that, Delphi language features are very behind compared to the developer demand. So now FPC wants to be even more behind, because we need to wait for Delphi to one day get there act together. :-(

As far as I heard we are already incompatible with Delphi regarding Generics (I don't know generics, I just heard of them). So even though FPC has Generics for some time, nobody can use it, because it's incompatible with Delphi.

Unicode is another issue. Delphi dictates there design decisions based on Windows only. FPC targets multiple platforms, so we should target our implementations based on OUR requirements.

An open source project trying to be compatible with a commercial product is simply a pipe dream. That's my thought on the subject, but that irrelevant I guess because I have no say in the FPC core team and there direction. I'm also getting a bit off topic here..

I'll have a look at the current testsuite to see what's there...

It will help me/us to start making UTF-8,16 versions of the RTL routines. That work is parallelizable with the compiler work, and these will have to be done anyway sooner or later, and fairly independant of the exact outcome of the string types.

Exactly my point! And if you want some code to look at (which you may legally do, not like Delphi code), have a look at MSEIDE, Lazarus and fpGUI for implementation examples of some functions.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Op Tue, 29 Jul 2008, schreef Graeme Geldenhuys:

FPC behind? What are you talking of man? :)

Waiting until we know more about the Delphi implementation is in the interrest of everyone here. Borland, oops, Codegear, whoops, Embarcadero, might have strange plans, but Delphi users remain important for FPC. Not everyone has completely switches to FPC like you and even you benefit from being able to use more external code. Code exchange is extremely important.

Note we need a major revision anyway to introduce these changes.

As far as I heard we are already incompatible with Delphi regarding Generics (I don't know generics, I just heard of them). So even though FPC has Generics for some time, nobody can use it, because it's incompatible with Delphi.

People who need their code compilable with Delphi cannot use generics indeed. So what, they can use both Delphi and FPC if they keep their code clean of compiler specific features. But, doing strings incompatible with Delphi means Delphi users cannot use FPC. Because any program uses strings.

Unicode is another issue. Delphi dictates there design decisions based on Windows only. FPC targets multiple platforms, so we should target our implementations based on OUR requirements.

Agree, but note that one of our requirements is code exchange with Delphi.

Good. MSEIDE is quiet a bit ahead because it made the switch to widechar/strings from the start.

Daniël

Daniël Mantione wrote

The developers haven't talked about it yet, but I can imagine we will have some target platforms where sizeof(char)=1, which would provide for 100% compatibility with old code and some platforms where sizeof(char)=2, which will provide the unicode support for the future.

Sorry, but a unicode character can be anything from 1-4 bytes. 2 bytes will hardly cover the full unicode character range.

For the time being you can prepare your own code by adding: type char=widechar; string=widestring;

I already use something similar to fpGUI...

  TfpgString      = type string;
  TfpgChar        = type string[4];

TfpgChar being 4 bytes to cover the whole Unicode range.

And some UTF-8 compliant string functions which are used often...

function  UTF8Copy(const s: string; StartCharIndex, CharCount: integer): string;
function  UTF8Length(const s: string): integer;
function  UTF8Pos(const SearchForText, SearchInText: string): integer;
procedure UTF8Delete(var S: string; Index, Size: integer);
procedure UTF8Insert(const Source: string; var S: string; Index: integer);
// short form (alias or convenience) functions for the UTF8 ones above
function  Copy8(const s: string; StartCharIndex, CharCount: integer): string;
function  Length8(const s: string): integer;
function  Pos8(const SearchForText, SearchInText: string): integer;
procedure Delete8(var S: string; Index, Size: integer);
procedure Insert8(const Source: string; var S: string; Index: integer);

And some UTF-8 file access functions....

// RTL wrapper filesystem functions with platform independant encoding
// These functions are common for all platforms and rely on
fpgXXXPlatformEncoding
function  fpgFindFirst(const Path: TfpgString; Attr: Longint; out
Rslt: TSearchRec): Longint;
function  fpgFindNext(var Rslt: TSearchRec): Longint;
function  fpgGetCurrentDir: TfpgString;
function  fpgSetCurrentDir(const NewDir: TfpgString): Boolean;
function  fpgExpandFileName(const FileName: TfpgString): TfpgString;
function  fpgFileExists(const FileName: TfpgString): Boolean;

And this is my point..... Many developers have such implementations in there own code for some time now. All these string encoding functions should actually be in FPC (a long time ago).

By the way, I do believe we need to be Delphi compatible here and thus know their implementation. Code exchange between Delphi & FPC otherwise becomes too hard for people.

A pipe dream, like I said before... :-) I don't see the point why developers want to switch between compilers, using the same code base. Simply pick a compiler that can do it all, FPC!! ;-) That's what our company did. FPC fills our needs perfectly - cross platform, 32 bit and 64 bit support, an awesome FCL, loads of header translations etc... We don't need Delphi!

Now the next item I would like to add to that awesome feature list of FPC, is full Unicode support.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Op Tue, 29 Jul 2008, schreef Graeme Geldenhuys:

Please read http://unicode.org/notes/tn12/ why using 2 byte strings internally is the best choice.

Daniël

My case in point. We keep rewriting the sysutils unit functions, when all these things should be in FPC itself!

Could you do a test with the U+00A0 ThousandSeparator character? If it works, I guess I'll be using another custom written sysutils function in fpGUI. :-(

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

I'm not sure how many people are trying to do that on the ThousandSeparator variable, but in theory you are right.

Micha

Well, of course. We have more requirements!? Multi-platform and compability to self and Delphi. But that we pull that off is also one of our main (if not THE) strengths.

And in the past it has proven that it pays to be compatible, and to be not rash in major course decisions because of pressure to "immediately" do something.

I really think playing 'catchup' with somebody like Delphi is not a good thing. They have different goals as far as I can see, plus their future doesn't look bright (for a very long time now).

We have Delphi compatibility as a major point, and it pays off. We are the only smaller development project that is not stewarded by some firm that has commercial components written for it.

Also, the clear direction of compability limits the "design by committee" problem a bit, where you get a feature laden, but unusuable product.

Delphi tries to compete with Microsoft using Microsoft's own tool (.NET).

Tiburon comes without .NET.

We all know how that turns out when anybody tries to compete with Microsoft.

We also have a Windows compiler.

Because of that, Delphi language features are very behind compared to the developer demand. So now FPC wants to be even more behind, because we need to wait for Delphi to one day get there act together. :-(

We are not a commercial company, and even a commercial company doesn't automatically only cater to the herd. There are specialized commercial companies too.

You say it yourself, you can't compete with Microsoft, and we need an edge over the zillion of useless Open Source development projects somehow.

A clear direction, our relative independance and the fact that we dare to take impopular choices is our main edge there. That, and the influx from a major external commercial codebase.

Let's not ruin that.

As far as I heard we are already incompatible with Delphi regarding Generics (I don't know generics, I just heard of them). So even though FPC has Generics for some time, nobody can use it, because it's incompatible with Delphi.

We will see how that pans out in time. Maybe we'll support Delphi's syntax too in time. Wouldn't be the first time.

Unicode is another issue. Delphi dictates there design decisions based on Windows only. FPC targets multiple platforms, so we should target our implementations based on OUR requirements.

Yes. But while we don't have the exact requirements that Delphi does, we do
have Delphi compability as major feature. Period.

All the major open source codebases (Jedi, Indy, VST, the remobjects stuff, whatever is still active of the TurboPower stuff etc) will go Tiburon sooner or later. If you want to be the messenger that tells them that they have to manually insert a conversion in each procedure, or fork the codebase for FPC, I can give them your email address.

The same ackward position is created for more FPC oriented packages that also support Delphi (like e.g. TDBF).

A lot of former deviations from Delphi compatability have been changed back to compatible in time (e.g. the procvar stuff, the required @ which is still in objfpc) except for some of the most lowlevel details, for exactly such reasons.

An open source project trying to be compatible with a commercial product is simply a pipe dream.

Well first, I don't think so, but also what are you suggesting? Becoming the a small "last stand" for Pascal ?

Yes, we can stuff the rough routines in a few temporary utitlity routines even (sysutilsunicode and strutilsunicode or so), till the rest is ready to go native. Or postfix them with encoding for time being and stuff them in sysutils/strutils.

But first we need a set of tests, then make the routines, and then we'll see where to stuff them.

(while true, IMHO that changes nothing for us, since the OS developers mostly made that choice for us)

(it can be larger, since a printable char might be more than one codepoints. (Thai iirc uses up to 4). This is due to the fact that interpunction in a lot of languages is more or less combined into the last char.

A pipe dream, like I said before... :-) I don't see the point why developers want to switch between compilers, using the same code base. Simply pick a compiler that can do it all, FPC!!

(It happens. However sharing libraries are more important for sharing then actual end-developer projects. Specially the open source ones. And why not? A few defines extra and your audience is larger)

Op Tue, 29 Jul 2008, schreef Micha Nelissen:

charvar:=ThousandSeparator;

... would fail as fail.

Basically anything fails except where the char is converted to string. I don't the compat issues are theoretic. There for sure exists code that assigns thousandseparator to a char or passes it to a var parameter.

Daniël

Daniël Mantione wrote

> developer demand. So now FPC wants to be even more behind, because we > need to wait for Delphi to one day get there act together. :-( FPC behind? What are you talking of man? :)

I knew that statement would get some attention. :)

Waiting until we know more about the Delphi implementation is in the interrest of everyone here. Borland, oops, Codegear, whoops, Embarcadero,

Ya, my point. And nobody knows what Embarcadero is going to be like.

might have strange plans, but Delphi users remain important for FPC. Not everyone has completely switches to FPC like you and even you benefit from being able to use more external code. Code exchange is extremely important.

Code exchange - the pipe dream (tm). :) I have done quite a bit of code porting. Trust me, FPC and Delphi is not as compatible as everybody would hope. The reason is not because FPC is inferior, it's because FPC targets more platforms, so some things need to be different.
I'm pretty sure there are NO reasonably sized projects that are code compatible without a need of some changes.

Plus if FPC frees itself from that requirement, it has free reins to implement things perfectly based on OUR requirements. Look at Chrome (or whatever they are called now). Some awesome language features have been knocked out in a relatively short period, because they don't bother with Delphi compatibility. Instead they concentrate on improving the language.
Delphi on the other hand are more interested in building a Visual Studio clone and .NET support. When last did Borland, CodeGear, Emb.... add language features that was not specifically due to .NET requirements.... D6 or D7 maybe? That's like a lifetime ago.

People who need their code compilable with Delphi cannot use generics indeed. So what, they can use both Delphi and FPC if they keep their code clean of compiler specific features.

And loose all the benefits Generics can give them!

Good. MSEIDE is quiet a bit ahead because it made the switch to widechar/strings from the start.

Pity FPC could do such a bold move. ;-) Imagine how much less work Martin would have had to do.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Op Tue, 29 Jul 2008, schreef Graeme Geldenhuys: interrest of everyone here. Borland, oops, Codegear, whoops, Embarcadero,

You forgot Inprise and Devco :-) And codegear is still good, since the Delphi oriented division retains that name.

Op Tue, 29 Jul 2008, schreef Graeme Geldenhuys:

True. Because of the influence of Lazarus, the FPC team has spent much more time satisying the needs of ansistring based LCL than satisfying the needs of widestring based MSEGUI. But Martin also choose consiously to do work outside the FPC team rather than work with it to integrate his needs into FPC.

Daniël

Because of that, Delphi language features are very behind compared to the developer demand.

In my personal view, there are very few points that are shared by Delphi and FP and that are "very behind compared to the developer demand".

The main of these is multi-threading support.

While both Delphi and FP do have the very workable TThread, same was planned with an application in mind that runs on a single processor engine and is mainly intended to free up the GUI thread when long winded calculations are to be done, so that the GUI does not freeze.

.NET introduced some very interesting new concepts on parallel programming that are easy to use and are planned with the application running on SMP machines in mind.

RemObject's "Oxygen" implemented these concepts in Object Pascal "Delphi Language".

As theses concepts, while being "invented" by .NET, can decently be implemented in the RTL without using (or even thinking of) a .NET runtime, IMHO it would take FP a great step ahead (of Delphi :) ) if it would provide these concepts.

The language constructs "*future *variables" and "*async* expressions" allow to easily "spinning off" an instruction sequence from the running code into the background (and thus allow it to be calculated by another processor). See: http://wiki.remobjects.com/wiki/Future_%28keyword%29 I feel that this can be (quite easily :) ) be converted to using (a special kind of) TThreads by the RTL.

Maybe there are some more .NET goodies that can be implemented without actually using .NET. http://wiki.remobjects.com/wiki/Oxygene_Glossary might provide some hints on how these can be translated into Object Pascal enhancements.

-Michael

So back to my original question.... :)

Due to ThousandSeparator being a Char type, is using a normal space ($20) the only available option for Russian users, using the current RTL implementation? Though this might cause issues in text wrapping routines which can now not distinguish between breaking spaces and non-breaking spaces.

The only other alternative is writing my own string format functions like fpgFormatFloat() and hope the users of fpGUI use the custom written functions instead of the RTL ones. This also means I need to implement my own locale variables to be Unicode compatible. What a job, for something that seemed so small an issue in the beginning. :)

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Op Tue, 29 Jul 2008, schreef Graeme Geldenhuys:

A possible hack could be to abuse an ASCII control character for the non breaking space, for example ThousandSeparator=#0 means a non breaking space, so it is later converted back to the right utf-8 representation.

Well, this seems exactly what Martin has done, so perhaps you can reuse some of his work.

Daniël

Yes, this is your only option.

Let's not jump to conclusions too quickly.

Obviously, the FPC team will have to do something for Unicode support; Tiburon compatibility plays a big role in that. Once this is done, there should be no problem. For the time being, replacing the non- breaking space with a breaking space is the best you can do. The chances that you really need a non-breaking space somewhere in an amount are infinitesimally small.

So, no need to make the issue bigger than it is, and start coding duplicates of existing routines because of such a small issue. We'll fix it eventually, this is the main message.

Michael.

That will be realy great!

Yup. If it's really the case then I'm sorry to say that FPC has such a "loser mentality". FPC has chances to become the leader of object pascal native compiler since Delphi was starting to die after Delphi 7. But, instead of taking the lead, FPC let itself and the users down in the name of "compatibility". What a shame! :-P

Then why FPC implemented generics before Delphi in the first place? I never heard compatibility issue when we discussed about this feature. Delphi didn't have this feature when FPC implemented its own syntax on generics. I was proud to know that FPC really implemented generics before Delphi did. It made me confident to completely using FPC and forget Delphi. I saw bright future with FPC. But now... :(

Unicode is another issue. Delphi dictates there design decisions based on Windows only. FPC targets multiple platforms, so we should target our implementations based on OUR requirements.

+1

Compatibility is good, no doubt about it. But if it obstacles FPC to have feature(s) that most users need, FPC should put compatibility issue aside and get users need on higher priority. Be the first, be the leader, be a winner. If Delphi then implements it (if ever) in different way, then we can start to discuss about compatibility. It's absurd talking compatibility to something that even doesn't exist!

An open source project trying to be compatible with a commercial product is simply a pipe dream. That's my thought on the subject, but that irrelevant I guess because I have no say in the FPC core team and there direction. I'm also getting a bit off topic here..

FPC could lead the object pascal "standard" and make Borland /CodeGear /Embarcadero /whatever follows what FPC had done. Not the other way around. How can FPC become a better compiler than Delphi if FPC doesn't have the gut to be the best?! :(

Please look at other open source projects that have dignity to win the competition e.g. Linux, Apache, Firefox, etc. They dare to be different on the beginning but people begin to follow them in the end. They never want to be a "clone" or "copycat" or "shadow" or "tail" to other successful (commercial) products. They have winner mentality, that's why they succeed.

Sorry to be a little bit out of topic here. I just couldn't hold myself to express my opinion on this.

-Bee-

The main of these is multi-threading support.

I also demand this support. :)

I believe FPC team could provide this feature. They are genius. But, I'm afraid, they don't want to provide it simply because Delphi doesn't have it (yet). They don't want FPC being incompatible with Delphi. :-P

I don't understand why FPC has DELPHI MODE directive in the first place if FPC don't want to be different with Delphi. Maybe FPC should eliminate this directive and make it as the default mode. :-P

-Bee-

Op Tue, 29 Jul 2008, schreef Bee:

Could you show me the advantage of having an incompatible string implementation in FPC 2.4, which will be out after highlander? What is the point to "leading" here?

Just to be clear once and forever: We don't want a Netscape vs. Internet Explorer HTML extensions like war. It is our aim to unite rather than to divide the Pascal world more.

Sorry to be a little bit out of topic here. I just couldn't hold myself to express my opinion on this.

Good. Now it's time show practical proposals or stop the discussion.

Daniël

Adding some keywords would it not make incompatible. To stay compatible you just don't use them :).

I don't understand why FPC has DELPHI MODE directive in the first place if FPC don't want to be different with Delphi. Maybe FPC should eliminate this directive and make it as the default mode. :-P

Maybe a compiler switch can be added that disallows the new keywords and that is set by default in Delphi mode (but can be switched off by the user if he wants to take advantage of the new features.

-Michael

Bee wrote

I don't understand why FPC has DELPHI MODE directive in the first place if FPC don't want to be different with Delphi. Maybe FPC should eliminate this directive and make it as the default mode. :-P

This is exactly the reason. Strings and API touches everything, while generics are a language feature and thus under control of {$mode X}

Micha

-------- Original-Nachricht --------

Well, I second that. Especially because the "Delphi" implementation seems to be so Win32-centric, that copying it would be close to useless on platforms other than Windows. As far as I understand there is no such thing as a "codepage number" on Unices.

I doubt that the FPC team would want such a specific implementation, so despite all arguments, FPC would have to find its own way in either case.

Vinzent.

FPC has chances to become the leader of object pascal native compiler since Delphi was starting to die after Delphi 7. But, instead of taking the lead, FPC let itself and the users down in the name of "compatibility". What a shame! :-P

...

FPC could lead the object pascal "standard" and make Borland /CodeGear /Embarcadero /whatever follows what FPC had done. Not the other way around. How can FPC become a better compiler than Delphi if FPC doesn't have the gut to be the best?! :(

IMHO, Oxygen is the "leader of object pascal compiler" right now, but they don't provide support for native code, so you are right.

-Michael

Why, according to you, is Oxygen Object Pascal at all? Aside from their advertizements? Just if you compare the base subset ?

Op Tue, 29 Jul 2008, schreef "Vinzent Höfler":

Disagree. Nobody maintains codepage registries except two companies: Microsoft and IBM.

While I have my doubts about Windows API constants, there are only a few identifications that make sense:

* Microsoft code page number
* IBM code page number
* IANA mib-enum
* IANA encoding name

It doesn't matter which one you choose, you should choose one and use that over all platforms.

I doubt that the FPC team would want such a specific implementation, so despite all arguments, FPC would have to find its own way in either case.

I can live with using numbers from the Microsoft code page registry. Numbers are far more comfortable to use in portable code than using strings, which is the common practise on Unix. So, while there is some bad Windows taste involved, it's not that bad from a portability point of view.

Daniël

Why, according to you, is Oxygen Object Pascal at all? Aside from their advertizements? Just if you compare the base subset ?

Is there some independent definition of the term "Object Pascal" ? I don't suppose so. So they are right to claim that they are compatible to Object pascal :) .

But If you call their Oxygen code "Delphi Language" they in fact will get angry :) :) :) .

Of course Oxygen is a lot less compatible to Delphi than FP is. They are greatly CIL centric - though happily not really .NET centric as they explicitly do support Mono and Linux. That is why the strict create...free mechanism is not used there (as the CIL framework performs the garbage control for the application) and thus the code can be a lot different. OTOH they claim that "Delphi for .NET" is not a decent way to write CIL

Well, there is the actual Object Pascal standard draft. Delphi deviates quite some from that though.

So they are right to claim that they are compatible to Object pascal :) .

"compatible" is nonsense, since they are not compatible to any of the roughly three preexisting ones. Descendant could be said, but I don't even see much evidence for that. There is a superficial resemblance in the parser model and that is about it.

They are about as Pascal as Perl is C because they both have curly braces and some similar operator names.

Of course Oxygen is a lot less compatible to Delphi than FP is. They are greatly CIL centric

"less compatible"?!?!? Can Oxygen actually compile and execute any preexisting code in any Pascal dialect ?

OTOH they claim that "Delphi for .NET" is not a decent way to write CIL that doesn't make them "Object Pascal".

I do recognize that Rem Objects needs some language to package and promote their frameworks (the thing they are IMHO good in), but the featurelist is a bunch of C# me too's.

Of course they will greatly stress the improved readability and the like, but I would too if I had to sell it :-)

For more information on what Vinzent is talking about, please see the following website.

http://blogs.codegear.com/abauer/2008/07/16/38864/

examples:

type
  MyString = type AnsiString(<1..65534>);

were 1..65534 is the Windows codepage number.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

http://blogs.codegear.com/abauer/2008/07/16/38864/

This is all known and processed on July 17th.

type MyString = type AnsiString(<1..65534>); were 1..65534 is the Windows codepage number.

Elegant is different, but a reason incompatability? Also note Daniel's remark about codepage classification.

Could you show me the advantage of having an incompatible string implementation in FPC 2.4, which will be out after highlander?

Boost the usage of Unicode in FPC that would boost the usage of FPC itself. Unicode is one of the most demanded features (beside cross platform, 64bit support, etc) in Delphi since Delphi 7 (2001?). Yet, CodeGear never fulfill it until Tiburon.

What is the point to "leading" here?

Be the first object pascal compiler that provide newest technologies needed by users. Object pascal users are not as much as Java or C/C++ users. But, they are not in very small numbers either. In capitalism term, they're asset. By providing pascal compiler that always capable to provide new technologies would prevent pascal from its death. That would also save millions lines of pascal code being trashed.

Just to be clear once and forever: We don't want a Netscape vs. Internet Explorer HTML extensions like war. It is our aim to unite rather than to divide the Pascal world more.

You may say that. But look at the reality. You want it or not, users would create competition atmosphere by themselves by comparing features. There are many faith Delphi developers who still think that FPC is just a Delphi follower and Lazarus just another Delphi-like wannabe project because FPC/Laz doesn't really provide something that they really need. No wonder if those Delphi developers move to other languages (Java, Ruby, Python, etc) instead of using FPC. The biggest advantage of FPC/Laz over Delphi, IMO, is only the cross platform ability which is also available on other languages.

Unite doesn't mean to be a follower. Being different also doesn't mean against union. Look at the big picture. There are many pascal developers out there that need new technologies being implemented in pascal. No matter how much they love the language, if the language doesn't provide what they need, sooner or later they would leave pascal for other languages.

By providing DELPHI MODE (and other pascal dialects directive), IMO, FPC is already in the correct way of being different but still united with other pascal compilers.

Good. Now it's time show practical proposals or stop the discussion.

Well, I'm not a compiler expert. Is it possible to keep the "old" string in Delphi mode and implement "new" string in FPC mode? This way, FPC could keep (backward) compatibility in Delphi mode and provide the new (Unicode) implementation in FPC mode. FPC can modify Delphi mode string later after Delphi shows its mechanism and follow it to get Delphi compatibility.

However, I think it's too late to use this scenario since Unicode support in Tiburon is just a few months away. I think FPC should have done it since v.2.0. :( Perhaps FPC could think about multiprocessor support using this scenario. ;)

-Bee-

"compatible" is nonsense, since they are not compatible to any of the roughly three preexisting ones. Descendant could be said, but I don't even see much evidence for that. There is a superficial resemblance in the parser model and that is about it.

At least, they're trying to answer what the users need in .Net world which CodeGear couldn't do. They have guts to be different instead of being follower.

They are about as Pascal as Perl is C because they both have curly braces and some similar operator names.

But, users have an option to convert their old Delphi codes and get the new technology as the pay-off.

"less compatible"?!?!? Can Oxygen actually compile and execute any preexisting code in any Pascal dialect ?

Neither FPC. My old TP codes is hardly can be compiled using FPC due the old DOS nature. In the sake of being cross platform, incompatibility is inevitable. Converting to less compatible but similar syntax is easier than rewriting the whole codes in totally different syntax.

I do recognize that Rem Objects needs some language to package and promote their frameworks (the thing they are IMHO good in), but the featurelist is a bunch of C# me too's.

It's acceptable as they provide Oxygene only for .Net platform. Nothing wrong of being "me too" if it offers benefits of new technologies. It's wrong, IMO, of being stagnant and not creative in the name of "compatibility". Technologies are always improving and changing. We can't force ourselves to stick with old technologies just for compatibility sake. Compatibility is preserved only if it's possible to be done.

-Bee-

Op Tue, 29 Jul 2008, schreef Bee:

Why would it boost FPC usage?

Okay, here is the deal: From now I'm going to stick my fingers in my ears until someone says "You should be incompatible because....".

Until now I have heard these valid arguments:
- Delphi's implementation is ugly/non portable.

By providing DELPHI MODE (and other pascal dialects directive), IMO, FPC is already in the correct way of being different but still united with other pascal compilers.

The intention of the FPC mode is threefold:
- Allow FPC to be compatible by itself
- Fix the worst quirks in the Delphi dialect
- Disable some FPC features that interfere with compatibility

The FPC modes were never intended to design our own dialect from scratch, or give the finger to existing code. Any code should compile in FPC mode with just a few fixes, at least that is how it is intended.

Further, you lack complete understanding of the impact of incompatible strings. Delphi mode is feasible because in one mode, you can call code in compiled in other modes.

I don't consider this a practical solution for Graeme's issue with the numeric separators, he would still have to wait for 2.4, and we could implement it compatible anyway in 2.4.

However, I think it's too late to use this scenario since Unicode support in Tiburon is just a few months away. I think FPC should have done it since v.2.0. :( Perhaps FPC could think about multiprocessor support using this scenario. ;)

Agree for the part that an proprietary unicode string would have made sense then and not now. I would not have done things differently in hindsight. Postponing it for even more features would have given extra stress supporting 1.0 and made users impatient waiting for 2.0.

Daniël

Wouldn't do, since it is a totally different market. A new language, incompatible with everything. While Codegear caters to the needs in its own usergroup, not in (a very vaguely defined) .NET world.

They have guts to be different instead of being follower.

Funny that you say that about the Oxygene language. To me the language concept and marketing screams ".NET me too wannabe".

> They are about as Pascal as Perl is C because they both have curly braces > and some similar operator names. But, users have an option to convert their old Delphi codes

Where? There is no migration path. It is scratch from new for all non-trivial code.

and get the new technology as the pay-off.

Which "new" technology?

> "less compatible"?!?!? Can Oxygen actually compile and execute any preexisting > code in any Pascal dialect ? Neither FPC.

Note that I say _any_ not _all_.

There is nothing new there. It is all C# and Java rehash.

It's wrong, IMO, of being stagnant and not creative in the name of "compatibility".

Well, then where are the major new features of Oxygene? Where, what?

Technologies are always improving and changing. We can't force ourselves to stick with old technologies just for compatibility sake.

Programmers are not exempt from normal business practices. And that means constant revenue, dealing with installed base and long term investment plans.

Compatibility is preserved only if it's possible to be done.

And all this from a codepage number in one line of source.

Funny that you say that about the Oxygene language. To me the language concept and marketing screams ".NET me too wannabe".

Remobj is one of the few companies that bring the cross-platform features, and thus the promising future of CIL (aka ".NET" in Microsoft speak) up to front. They, too, do use the term ".NET" instead of "CIL", because otherwise 90% of the readers would not know what they are talking about :( . But they do advertise that Oxygen can compile for multiple OSes (using Mono bindings) and multiple processors (e.g. using portable .NET bindings). If the user code is done in a way that all bindings are possible (this can be checked with the SDK), the resulting Assembly will run "everywhere". (At least this is what I understood.)

IMHO, CIL (".NET") is a really promising concept for the future of computing. I just read about an "embedded" CIL Framework, done for a deeply embedded CPU that is to be configured ("programmed") into an FPGA (IMHO this is the future of deeply embedded Computing) and this Framework does not even need an OS to run (but can take advantage of any co-existing OS, says the article).

Provided that it's a lot easier to create a compiler for CIL than creating it for multiple CPUs and OSes, IMHO, this is the way to go on the long run. (But of course native code will widely be in use for several years to come :) ).

-Michael

Provided that it's a lot easier to create a compiler for CIL than creating it for multiple CPUs and OSes, IMHO, this is the way to go on the long run. (But of course native code will widely be in use for several years to come :) ).

If this was true, Java would have taken that market already. There is nothing new to that aspect of CIL, and specially with only one minor vendor supporting it.

Well, then where are the major new features of Oxygene? Where, what?

I already did rant about "future" and "async".

If there would be much more I would already have bought it :) .

-Michael

Like Bee said... The time frame has come and gone! If you read the delphi.non-technical newsgroup you would notice that quite a few Delphi developers use FPC for backend features. Linux services etc... They also talked about FPC having 64bit support. If FPC didn't worry to much about compatibility and instead implemented Unicode support long ago like it could have, it might just have boosted FPC usage, beating Borland to the punch by about 7 years. FPC could have attracted and converted more Delphi developers...

Yes it's always easy to say that with the benefit of hindsight, but it's not far from the truth. Our company is a case in point. We saw features in FPC that we would like and Borland was going in the wrong direction for us. At that point in time (2-3 years ago) there was no mention of when Delphi would ever support those features. FPC seemed the obvious choice to us, so we moved - an still with no regrets. :-) Yes it costed use some time and effort to port our code and required tools, but it's still been worth it. We can now target different platforms, 32/64bit systems.... Our next evolution for our products are non-English speaking countries, hence the Unicode requirements.

But the time frame to beat Delphi with Unicode support has passed... :-(

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

If this was true, Java would have taken that market already. There is nothing new to that aspect of CIL, and specially with only one minor vendor supporting it.

Basically you are right, but
- In fact Java is very widely in use (even though there are lots of shortcomings regarding Java <e.g. performance> )
- AFAIK, CIL seems to improve some of the Java shortcomings
- CIL defines concepts for multi-treading and multi-processing
- every year the processing performance and available memory resources improve and thus creating "economical" object code is less critical
- While (AFAIK) there are no (or only very few and rarely used) languages besides Java that can create byte code for the just-in-time compiling Java Framework, There are a lot languages with compilers form different brands usable with CIL (several C# compilers, C++, Pascal, (Oxygen and Delphi), Visual Basic, Iron-Python, .., <is there a Java->CIL compiler ?> )

- Michael

Zitat von Graeme Geldenhuys :

I wonder, what the next 'boost' feature of fpc is, that Delphi does not have yet: Automatic parallelization, language extensions for parallelization, fpc as scripting language, fpc as browser plugin, packages like cpan, jit compiling?

Mattias

Well, if we could clone the FPC team a couple of times, we probably would have had it a long time ago. And a host of other we are few, we have day jobs, and we're not being paid for any of this.

Volunteers have also been very scarce. Talks of foundations and whatnot have popped up at various times, but always dwindled away, no-one doing anything was the result time and again.

And that is the whole explanation for some features not being there: it's open source, and a hobby project for the core teams.

If users badly need some features, it's time they organized themselves and get something going to support FPC development *actively*.

If this doesn't happen, then I conclude that these features aren't really needed, but are in the domain of 'nice to have'.

Michael.

Zitat von Michael Schnell :

Which is mostly due to the garbage collector. There are already java coprocessors boosting java to native speed.

- AFAIK, CIL seems to improve some of the Java shortcomings

I'm curious. Has someone evidence for that?

- CIL defines concepts for multi-treading and multi-processing - every year the processing performance and available memory resources improve and thus creating "economical" object code is less critical

Same for java.
I have some doubts about the increases of processing performance. The speed ups of the last years were mainly due to multiple cores. The increase of speed of a single core decreased. In fact many recent computers like the eeepc are pretty slow to get longer run times. And writing multithreaded code is economically more expensive than single threaded. So I see a strong need for native compilers like FPC.

Yes, but I didn't test it.

Mattias

We spent this time frame with making FPC multiplatform. Knowing what's important doesn't help. You need the time to implement it. I (and probably a lot of other people) know that really good multithreading support would be currently a killer feature, especially with language support. I don't have time to implement it. In three years some smart guy says, the time frame for good multithreading has come and gone. Great.

If you read the delphi.non-technical newsgroup you would notice that quite a few

delphi.non-technical is yellow press made by users for users.

Delphi developers use FPC for backend features. Linux services etc... They also talked about FPC having 64bit support. If FPC didn't worry to much about compatibility and instead implemented Unicode support

FPC has unicode support for years. Remember what a widestring is?

long ago like it could have, it might just have boosted FPC usage,

Unlikely. Anyways, who is interested in usage? The important part is contribution.

beating Borland to the punch by about 7 years. FPC could have attracted and converted more Delphi developers...

Compatibility is the only way to attract Delphi developers. Just consider the kcl/fpgui vs. lcl case: kcl/fpgui started before the lcl and was much more advanced regarding the design than e.g. the vcl. Nobody except Sebastian and years later you were interested in it. People used the lcl. The MSE is fully unicode. How much people use it and contribute? Without Martin, MSE would be dead.

The CIL bytecode removes the typing needed only for interpreting, and thus might be a bit more JIT friendly, and in general the requirement to interpret is not there, and never has been. I don't know what recent Java's change there though. I'm no VM technician, so I can't comment on how useful this really is.

Microsoft was also more vocal from the start about supporting it to other languages, including from other classes of languages (e.g functional languages) from the start. While Sun tried to keep control, and keep JVM about Java. However that afaik has changed on the Java side too since. I think this has all been more intention and marketing that superiority on the CIL part.

Just like they have been very vocal about possibilities for portability too, something they have never supported. Sometimes I think .NET is more about loosening Microsofts ties to PC and Intel, to allow them to pull off architecture migration stunts a la Apple, than it is about programming.

Michael Van Canneyt wrote

But we are few, we have day jobs, and we're not being paid for any of this.

I'm in the same boat. And I did offer to extend or start writing tests cases to help implementation. I'm not a compiler developer (I don't understand any of it's internals), but I can help implement Unicode enabled string functions. We have a few implementations floating around already.

And that is the whole explanation for some features not being there: it's open source, and a hobby project for the core teams.

I did not demand the unicode features... I simply stated a shortcoming when I started this thread and asked how I could proceed with a work-around.

Also, I did not see an issue in highlighting my thoughts on cloning other projects. I thought we are free to express our opinions, seeing that this is an open-source community project. I also do not consider myself one of those users that simply bitch from the sidelines and not really participate. I might not have been as active as other users in FPC or Lazarus development, but I have submitted patches for things I could fix or implement. Plus, if I feel strongly about (against) something (like my issues regarding LCL), I do something about it, like I'm doing with fpGUI.

Anyway, I have received a answer to my original question and I thank those that participated in the conversation. Always a pleasure chatting to you guys. ;-)

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Agree fully. Also note that a lot of the delphi projects out there support LCL, not MSEGUI (or if so, it is way behind).

There is a huge difference between finding something intellectually interesting (e.g. many people are instinctly attracted to from scratch purposes), and committing any really major time to it. Because then the real world comes crashing in.

As for threading support, people (also in non-tech) drone on about this like chickens without heads, fed by multicore marketing offensive. However there is next to zero original thought. Users can't even seem to agree if the result should be focused on ease of use or maximum performance. Some people even drag in CUDA support etc.

Read this and the reactions, and weep:

http://groups.google.com/group/borland.public.delphi.non-technical/browse_frm/thread/db61d19063a2f948/289008199451755a?lnk=gst&q=voort+multicore#289008199451755a

Florian Klaempfl wrote

We spent this time frame with making FPC multiplatform. Knowing what's

And I thank you again for it. :)

delphi.non-technical is yellow press made by users for users.

What better place to find out what users think of a product. After all, they are the people using it. I don't want to hear from the Borland/CodeGear/Embarcadero sales and marketing team, I want to hear from the actual people using the product! That newsgroup does just that.

FPC has unicode support for years. Remember what a widestring is?

So I'm just imagining the locale issue with U+00A0 character? Plus the knock-on affect to FormatFloat, FormatDateTime etc..

Does FPC have any any functions that work correctly with surrogate pair used by UTF-16? If so, it would same me a lot of time implementing them myself. What would Pos, Length, Copy etc return? BTW: If it comes down to it that I have implement UTF-16 string functions myself, you are more that welcome to use the code in FPC. :)

> long ago like it could have, it might just have boosted FPC usage, Unlikely. Anyways, who is interested in usage? The important part is contribution.

With usage comes contributions! Nobody can deny that.

The MSE is fully unicode. How much people use it and contribute? Without Martin, MSE would be dead.

Nope, MSE only supports UCS2, not full UTF-16. And off the top of my head I know of myself, Felipe and Michael van C. that had a look at MSE. It's just to hard to read Martin's code and be able to contribute. He has a very different coding style. It works for him, but unfortunately not for others.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Read this and the reactions, and weep: http://groups.google.com/group/borland.public.delphi.non-technical/browse_frm/thread/db61d19063a2f948/289008199451755a?lnk=gst&q=voort+multicore#289008199451755a

I don't agree on the point that good mt support is a matter of the framework. _Really_ good multithreading support is a matter and must be a matter of the language as well and in several years and must be as common as while or for loops. Currently, multithreaded programming is like programming spaghetti basic. A good framework is comparable to at least the try to program structured with line numbered basic but it is not forced by the language. The compiler must know about parallism.

Mattias Gärtner wrote

I wonder, what the next 'boost' feature of fpc is, that Delphi does not have yet: Automatic parallelization, language extensions for parallelization, fpc as scripting language, fpc as browser plugin, packages like cpan, jit compiling?

Lets wait 7 years, then bitch about it not being supported. ;-)

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Note that I already hinted on the duality. Performance vs ease of use. You more or less specify the performance angle, for newly written code for people with heavy calculation that generally know what they are doing. I think that is the rarer case in practice.

But the other side is not about perfecting lineair algorithm with a bit of multithreading. I'm talking about what to do from the perspective of the avg Delphi/FPC user. And most of them don't have really intensive calculating loops, outside maybe some minor encryption/compression code that they never wrote themselves in the first place.

It is not a simple linear algorum that needs updating with a bit of manually instrucmented threading.

The problem of "ease" is just that multithreading from scratch is too hard or too time consuming as a programming model for most. Specially with a single threading GUI (and maybe here and there GUI api), and the ad-hoc way they throw together programs.

And they desperately turn to the tools makers (CG and us) to "fix" that, because Intel promised them heaps of extra performance due to the multicore revolution. So preferably they want us toe wave a magic wand and that their old crusty VCL code ridden with timers, COM objects, external libs and the like is suddenly fully multithreading and scales with number of cores.

Without a totally new design paradigm that is harder, and without a careful tuning to avoid the threaded loop being slower than the single core one.

And that is IMHO not realistic. So the cure you are suggesting is for a different problem, and for a happy few that also can handle manual threading fine.

And then I think better frameworks is a better solution then optimizing loops OpenMP style, because you won't even notice the speed up of that in the avg app.

Currently, multithreaded programming is like programming spaghetti basic. A good framework is comparable to at least the try to program structured with line numbered basic but it is not forced by the language. The compiler must know about parallism.

Personally I hope we get it tomorrow. For me it would mostly work, since at work I really do dataprocessing in a fairly linear way. But that isn't the avg users case as far as I can see.

I fully support this. Regarding Object pascal I am dreaming about "thread events" since many years. (This would need no new keyword and only a very small addition to the compiler. You might find posts on this in the backlog of this forum.) I did provide a concept, but had not the spare time to try to implement it.

The said "future" and "asnc" concept that Oxygen adopted from .NET seems very promising, and I'm sure that it can be implemented in the RTL in native code without thinking about a .NET framework.

-Michael

This is so, and is good, but compiler support for a new string type is of a different order...

I am not targeting anyone in particular, definitely not you...

But the huge discussion about features in FPC is a bit gratuitous, because as always, many people participate in the discussion, but there are too few that actually help out.

Ideas and plans are not the problem.

Manpower and time are usually the problem in FPC. Compared to the GCC team or Mono, we're seriously understaffed.

Michael.

Michael Van Canneyt wrote

This is so, and is good, but compiler support for a new string type is of a different order...

That I can believe... and like somebody said before, it affects everything.

Manpower and time are usually the problem in FPC. Compared to the GCC team or Mono, we're seriously understaffed.

As soon as I win the local Lottery, things for FPC will change. :-)

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

In that case, to speed things up, I suggest you consult a voodoo doctor for the winning numbers :-)

Michael.

Op woensdag 30-07-2008 om 11:33 uur [tijdzone +0200], schreef Florian Klaempfl:

I'm not really into paralel-computing. But it does interest me.

Just to test some ideas/opinions. Could something like this be usefull?

Function DoSomething(const astring : string) : boolean; parallel;
begin
..
end

So that the 'parallel' keywords means that if you call this procedure, it's started in a separate thread. (maybe just a compiler hint, like in inline. So that the compiler can decide)
If you actually use the result somewhat further in you program, the compiler detects this and waits for the other thread to finish, before it continous.

Ofcourse, of someone uses some globa-vars in an 'parallel' procedure, he could be doomed, if he don't know what he does. But maybe the compiler can even forbid this.

Would this be usefull at all? Doable?

Joost

Hi Joost,

One option is each assembly instruction in the function to be actually executed in a separated thread. This is one of the proposals of Intel. Of course the compiler will have to packs the instructions the same way the compilers for DSP chips, Transmeta Crusoe, and Itanium processors do that. They group the assembly instructions in groups to be executed as a single parallel instruction. Intel proposes the same model, except the instructions will be sent to separated cores. This way your function will be executed N times faster.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

That's exactly what Florian meant with the 'asynchronous' keyword and what RemObjects did with their compiler (I believe).

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Zitat von Joost van der Sluis :

http://groups.google.com/group/borland.public.delphi.non-technical/browse_frm/thread/db61d19063a2f948/289008199451755a?lnk=gst&q=voort+multicore#289008199451755a

There is a wiki about that discussion:
http://wiki.freepascal.org/OpenMP_support

Mattias

Why not stick to the "future" / "asyc" paradigm ? The fact that it (supposedly) is originated from .NET does not make it a bad thing. And if anybody already knows this paradigm from other language and thus does not need to learn how to use it, he will be happy, which is not a bad thing either.

-Michael

This is only usefull if you have really, really, much cores. As Intel indeed propagates? And how do those threads handle othe resources? (memory?)

I think this is the far-away future. Intel is only pressing this because they are scared that in a few years they can not improve PC speed as fast as they could before. They have to do/scream something. But time will tell?

Joost.

Op woensdag 30-07-2008 om 08:05 uur [tijdzone -0700], schreef Boian Mitov:

Op woensdag 30-07-2008 om 17:08 uur [tijdzone +0200], schreef Mattias Gärtner:

There is a wiki about that discussion: http://wiki.freepascal.org/OpenMP_support

Thanks for the link, very usefull. And I do see that (Graeme) that my proposal is the same af that from Florian. I just hadn't read his mail yet.

Joost.

Hi Joost,

Actually the trend started probably ~10-15 years ago with the DSP processors. Then along came Transmeta, and the Itanium, then there ware the GPUs including from NVidia, and there is PlayStation 3. They all use this type of approach. The first massive multicore I am aware of is the new Sparc http://en.wikipedia.org/wiki/UltraSPARC_T1 and
http://en.wikipedia.org/wiki/UltraSPARC_T2 from SUN. Intel actually is playing a bit of a catch-up game. The shared memory however has worked perfectly for the DSPs for many years, so it has for the Itanium, Crusoe, the GPUs, and PlayStation. The future is not likely to be in faster systems, but in more cores. This seems to be the consensus lately among the processor architects. Intel already demonstrated 100+ core processor last year. This year we expect the first 16 core processors to hit the market ( 8 HT cores), and the direction is very clear. Any compiler vendor, or developer, should at least be paying attention ;-) .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Hmm, isn't that about execution units not cores?

Yes. DSP, Crusoe and Itanium do it with execution units. The new wave however is to do the same with cores. The idea is that a core can be thread either as a stand alone core, or as execution unit, and to be better utilized. This way you should in theory (According to Intel) get the best of both worlds ;-) . PlayStation as the best of my knowledge also utilizes this approach.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

As an alternate to whether we go along with open mp or whatever in the current discussion, why don't we take notice of the previous email discussion about fpc deciding things for itself & just do the very very simplest (even noddy) thing that'll give people who aren't gurus the ability to use the processing on a multi core system that are becoming so cheap & popular - I think joost suggested something like this here 1st but maybe we should make it even simpler.

Eg only allow the parallel or async keyword (I personally do not care which) on an otherwise normal procedure and then another procedure/keyword isfinished(x)? which checks whether the 'parallel' procedure x has finished. With minimum (ie no) checking for whether the procedure uses global data. It wouldn't be foolproof but would be easily usable, wouldn't it.

Because it's so simple we could then announce that we support parallelism eg in 2.2.4 or 2.4? Of course if a de facto standard did come along eg in Delphi or some other of the languages mentioned, we could take a view as to whether we'd support it.

John

This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.

Hi John,

This is just a discussion. There is no real need for something like this at the moment yet, and probably you should really just keep an eye on what CodeGear and the other vendors are doing and follow their lead ;-) . It is very likely we will start to see more and more of this stuff in the coming years, but we will see.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

How many of these cases do you already have? Because it is nothing more than a threadstart and a thread.waitfor (or some more involved), something like

procedure something.buttonclick;
var thr :TSyncthread;
begin
  thr:=poolmgr.getblockedthread; // get a thread from the pool.
  ;
  thr.run
  <do other stuff>
  thr.waitforsignal; // also returns to pool.
end;

You could define a helper even for the first three lines.

Now unless you have something like this under every event already, it is not worth the effort. It's a shortcut for something that is already possible, and if you don't use it now, you apparantly don't need it that much?

Boian Mitov wrote

among the processor architects. Intel already demonstrated 100+ core processor last year. This year we expect the first 16 core processors to hit the market ( 8 HT cores), and the direction is very clear. Any compiler vendor, or developer, should at least be paying attention ;-) .

I think it's the world upside down. Intel should be engineering stuff we want, not the other way around. If "we" compiler/application developers don't develop multithreaded programs then they can't sell their expensive processors anymore. Seems wrong to me ;-).

Micha

Hmm... I thought the end users ware the ones always wanting faster stuff ;-) . Like real time photo realistic 3D rendering ;-) . This means we the developers press both the Processor vendors and the compiler developers to get us the tools for that.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Actually if the compiler vendors don't do that it help us a bit. Our libraries already have been redesigned for the 4.0 release to utilize vast number of cores, and if the compilers don't have the multicore support, it simply will give us competitive advantage ;-) . We are on the winnig side in this case :-D .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Op Wed, 30 Jul 2008, schreef Graeme Geldenhuys:

Does FPC have any any functions that work correctly with surrogate pair used by UTF-16?

Likely, because I doubt FPC has routines, other than UTF-8 <-> UTF-16 conversion, that need to bother with surrogate pairs. I.e. the following code is fully correct in an UTF-16 environment:

procedure upcase(var s:widestring);
var i:longint;
begin
   for i:=1 to length(s) do
    s[i]:=upcase(s[i]);
end;

...because all characters that need upper/lower casing are in the BMP. Likewise, all possible thousand separators are in the BMP, so you don't need to bother with that either.

If so, it would same me a lot of time implementing them myself. What would Pos, Length, Copy etc return?

This would be the same behaviour as with widestrings. RTL routines are dump, smarter ones should go in sysutils.

Daniël

Actually if I am not mistaken it was the MIPS that first shifted the pipelining ordering from the processor into the compiler ;-) . You can't even have a normal branch instruction in that bugger without another instruction after it executing. It really gives me the itch when I have to look into MIPS or Sparc code :-D .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

[...]

Because it's so simple we could then announce that we support parallelism eg in 2.2.4 or 2.4? Of course if a de facto standard did come along eg in Delphi or some other of the languages mentioned, we could take a view as to whether we'd support it.

Sorry, but adding parallelism into the language without providing proper synchronization primitives sounds kind of stupid to me.

But to add some more food to the discussion: For "asynchronous" or something similar I would require it to be scoped:

procedure RunningMultiCore;
   procedure Task1; asynchronous; begin ... end;
   procedure Task2; asynchronous; begin ... end;
var
   i : Integer;
begin
   begin // Task scope
      Task1;
      for i := 1 to 1000 do
         Task2;
   end; // If we get here, Task1 and 1000 Task2's are finished.
   // Evaluate results...
end;

Alas, there's no way to declare sort of block-local variables... but this at least allows for finer grained parallelism without to many hassles (when and where to decide of some routine has finished, was it started at all before and such things). Coarse grain parallelism could be handled the usual way via TThread class & Co.

Of course, there's the problem with core-utilization (how many subroutines are allowed to run parallel? etc.pp.), but as those are mostly runtime decisions, there should either be some API to control the behaviour or the compiler/OS should decide as they seem fit. And the main problem: To evaluate any results, you need some memory being accessed by the parallel tasks, which brings us back to the initial objection: There's no sense in doing that without also providing proper synchronization primitives. "volatile" or "atomic" keyword, anyone?

Wasn't suggesting no sync - just trying to say let's keep it very simple so we can do it quickly, but not, in words of mr E too simple! I yield to the gurus re how simple it can be w/o it not being usable. J

[...]

Because it's so simple we could then announce that we support parallelism eg in 2.2.4 or 2.4? Of course if a de facto standard did come along eg in Delphi or some other of the languages mentioned, we could take a view as to whether we'd support it.

Sorry, but adding parallelism into the language without providing proper synchronization primitives sounds kind of stupid to me.

But to add some more food to the discussion: For "asynchronous" or something similar I would require it to be scoped:

procedure RunningMultiCore;
   procedure Task1; asynchronous; begin ... end;
   procedure Task2; asynchronous; begin ... end;
var
   i : Integer;
begin
   begin // Task scope
      Task1;
      for i := 1 to 1000 do
         Task2;
   end; // If we get here, Task1 and 1000 Task2's are finished.
   // Evaluate results...
end;

Alas, there's no way to declare sort of block-local variables... but this at least allows for finer grained parallelism without to many hassles (when and where to decide of some routine has finished, was it started at all before and such things). Coarse grain parallelism could be handled the usual way via TThread class & Co.

Of course, there's the problem with core-utilization (how many subroutines are allowed to run parallel? etc.pp.), but as those are mostly runtime decisions, there should either be some API to control the behaviour or the compiler/OS should decide as they seem fit. And the main problem: To evaluate any results, you need some memory being accessed by the parallel tasks, which brings us back to the initial objection: There's no sense in doing that without also providing proper synchronization primitives. "volatile" or "atomic" keyword, anyone?

This is wrong. Asynchronous means that the procedure call is put into a queue and not immediatly handled if no thread is in the pool. Further, your code doesn't handle efficient parameter passing. Especially this needs compiler support.

asynchronous is of course only a part of possible support.

I know, but that is only a minor variation on the above scheme.

Further, your code doesn't handle efficient parameter passing. Especially this needs compiler support.

True. It doesn't have state associated (that is returned afterwards) too. And then we are closer to continuations afaik.

But this is not about what you could do to embelish it, the question if it is needed.

I don't have a real opinion yet. As I said, I can see the need of such shortcuts if some framework really bases on such a fundament (async procedures), and shortcuts improve handling, and one is really sure the shortcut saves a lot of time and avoids certain common typo mistakes.

Just more or less like dyn arrays aren't the nicest thing alive, but such dynamically sized arrays are simply used really a lot, and having a somewhat safer syntax really saves times and mistakes.

But I don't see why the framework could do a 1.0 without such features, and then inventorize and pick a few, most worthwhile extensions.

That is IMHO better than implementing a handful of extensions, and hope that something usable will automatically pop around it.

Op Wed, 30 Jul 2008, schreef Boian Mitov:

It is extremely important to note the number of cores cannot scale endlessly. The limits of SMP based systems were discovered in the HPC world more than 10 years ago. 64 or 128 cores seems pretty much the limit, few machines with more cores have been produced.

The reason is that the cache coherency algorithms don't scale. You get too much traffic between processors. This is already visible with eight socket Opteron systems today, the performance is disappointing, despite the very good NUMA-style design of the Opteron processor.

In fact, we are getting close to the limits. With AMD's announcement of the 12 core Magny Cours processor, we would have 96 cores in an 8 socket machine. I have to see this happen and I am sceptical it will perform. Quad socket may well limit for this processor.

The HPC world did move from SMP systems to clusters, distributed memory systems. Multi-threading was replaced by message passing, and there we are today, Roadrunner has been built, the first 1 petaflops computer.

The end of message passing has now been announced, it for sure we won't be able to do exaflops with passing. Still, this is not very relevant for us desktop users.

What would be interresting for us, is what will happen on the desktop. What will happen if we can afford 100+ cores in our desktops? Will our desktops become clusters with an interconnect network?

Now, of course, many people are still coding single threaded applications. Parallelizing is a need for the future. But know that the multi-core race won't last as long as the megahertz race, whatever cpu manufacturers tell you about 100+ core processors. We may already be about halfway at the moment.

Daniël

You are even more wrong: you should not make any assumptions about queues and concurrency here. You are just describing an algoritm (not the solution) to solve a certain problem. Just one of many. Including jumping the queue stuff. But you know that. :)

Boian Mitov wrote

The future is not likely to be in faster systems, but in more cores. This seems to be the consensus lately among the processor architects.

It's the consensus between the marketing departments of various processor manufacturers, because that's what they're delivering now and in the short term future. As shown by Amdahl's law, just adding more and more cores stops getting you significant performance gains fairly soon (even if 95% of your code is perfectly parallelisable).

See e.g. http://en.wikipedia.org/wiki/Amdahl's_law

Jonas

Well, try http://en.wikipedia.org/wiki/Gustafson's_Law for a comparison.

And don't forget, despite the name, those are not laws, not even theories. See, Moore's Law turned out to be wrong after all. ;)

Of course, you're right in the general case. Personally, I seriously doubt that we will see a desktop system with hundreds of processors in the next few years or so. Practical benchmarks these days have shown, that a Quad-Core gains almost nothing compared to a cheaper Dual-Core in most practical circumstances. Of course, that's what the processor architects are trying to change here. But considering the majority of desktop users, what would an e-mail program or word-processor gain from 64 cores? 99% of the time it's idling anyway...

Vinzent.

The good news for us, is that our customers do DSP, video and audio processing, and our libraries are designed to utilize both single and multicore systems, so we win in any case ;-) .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

The reason is that the cache coherency algorithms don't scale.

This again is a software problem.

If the task to do allows it the software needs to be written in a way that the cache coherency is not disturbed.

BTW all "super-computers" feature thousands of CPUs, but of course this is not simple SMP.

-Michael

Very well said ;-) .
Then on the other said to play the "Diablo's advocate" and support what our opponents have said in the "super-computers" the tasks are handled by the operating system.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Hmm... correction, they probably are also using ADA for that ;-) .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

-------- Original-Nachricht --------

Hmm... correction, they probably are also using ADA for that ;-) .

No, Fortran (I was told, it ain't FORTRAN anymore, modern compilers can handle lower case).

Vinzent.

P.S. The American Dentist's Association has nothing to do with that... did you mean "Ada (Byron Lovelace)" as in "(Blaise) Pascal"? ;)

Op Thu, 31 Jul 2008, schreef Michael Schnell:

> The reason is that the cache coherency algorithms don't scale. This again is a software problem.

Cache coherency algorithms are implemented in hardware.

If the task to do allows it the software needs to be written in a way that the cache coherency is not disturbed.

Easier said than done, because threading always involves multiple cores operating in the same address space, which means the cores need to communicate which core has which page in cache.

BTW all "super-computers" feature thousands of CPUs, but of course this is not simple SMP.

Like I said, the HPC world did move to distributed memory with message passing years ago and is not using threading. Threading simply didn't scale enough.

Daniël

Stupid me, to 20 years ago when I studied Fortran 77, I thought it was obsolete :-)
I hope they at least have loops now, and who knows, it may even support Hmm.... Hmm... recursion :-D .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Daniël Mantione wrote

Easier said than done, because threading always involves multiple cores operating in the same address space, which means the cores need to communicate which core has which page in cache.

They don't care about pages. They only care when writing to shared memory (to keep other caches consistent). This is on cache line level, e.g. 32 bytes for PPC usually, 128 bytes on Intel.

Micha

Op Thu, 31 Jul 2008, schreef Boian Mitov:

Stupid me, to 20 years ago when I studied Fortran 77, I thought it was obsolete :-) I hope they at least have loops now, and who knows, it may even support Hmm.... Hmm... recursion :-D .

No, Fortran is still in big use, and yes it is obsolete. However, it has some features people like, like local arrays that are dimensioned at procedure call time. I'm playing with the idea of implementing such a feature in FPC, like:

procedure x(const s:integer);
var a:array[0..s] of byte;

... or something.

Daniël

This discussion is fun, but we probably should leave this to Intel, Sony, AMD and Sun to fight. Personally I am surprised that we are on the verge of having systems with 32 cores, be it that some are half cores (The Hyper Threading ones). Having 32 cores even if you never get more, already starts to make a statement in support for the need of multithreaded applications, and probably compilers better utilizing the cores. Then again maybe it is just a hype, but if so it has been continuing for few years already, and it only seems to grow in strength.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

It had some feature I liked including direct Complex number support. Variable stack arrays are also available in C99 but not in the C++ standard, and as the best of my knowledge there are no plans to add them in the 200X ANSI revision :-( . So far Delphi has been doing fine with the classical variable arrays and SetLength ;-) .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Stupid me, to 20 years ago when I studied Fortran 77, I thought it was obsolete :-) I hope they at least have loops now, and who knows, it may even support Hmm.... Hmm... recursion :-D .

As far as I know it does. Lemme check ... http://en.wikipedia.org/wiki/Fortran ... yes, since Fortran90, it seems.

But we're getting dangerously off-topic now...

Vinzent.

Op Thu, 31 Jul 2008, schreef "Vinzent Höfler":

With C you have exactly the same compiler. True, it is a reason against Pascal, but does not explain the popularity of Fortran.

Daniël

Ada and C99 have that. Scripting languages anyway... I think the main feature of Fortran is the heavily optimzing compiler. ;)

Hmm.

|var a : array of byte;
|
|SetLength (a, s);

should do?

Vinzent.

Thank you!
I agree.
To summaries my opinion:

1. Massive multicores are slowly becoming a reality.
2. The modern compilers do little to support them aside from some library support.
3. The need for such support is not urgent, and probably will not be for at least 1-2 more years, but the demand will start to rise.
4. The best approach is to keep an eye on what the other compiler developers are doing, and try to follow the proven path ;-) .
5. In the mean time any good developer an utilize the existing tools and design true multithreading architectures. We have enough tools for that (Well maybe with the exception of a fast "Multi Read Single Write" implementation, but the life is never easy ;-) )

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Daniël Mantione wrote

...because all characters that need upper/lower casing are in the BMP. Likewise, all possible thousand separators are in the BMP, so you don't need to bother with that either.

Ah, that's so true. :-) I'm starting to like the idea of UTF-16 more and more compared to UTF-8. UTF-8 is a lot more work.

OK thanks. I'll start creating unit tests for some of these basic functions so I can solidify my understanding and expected behaviour of widestrings and UTF-16.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Eg only allow the parallel or async keyword (I personally do not care which) on an otherwise normal procedure and then another procedure/keyword isfinished(x)?

In fact Oxygen does also support the "parallel" keyword/paradigm that works very similar to what you suggest:

http://wiki.remobjects.com/wiki/Parallel_Loops

They only seem to allow the parallel keyword for loops:

for parallel i: Integer := 0 to 10 do ...

But why not (additionally) do

parallel procedure x(a: integer);
begin
...
end;

Michael

To be honest, I used to wonder why people still use Pascal when they have C++ ;-) . I still wonder, but now I am one of those people :-D .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Boian Mitov wrote / napísal(a):

I didn't want to add fire to this discussion, but I think one thing needs to be mentioned, and that is graphics card usage from Threads and possible APIs and their limitations.

I was at #opengl channel a few times recently due to work and personal hobby too, and once I asked about doing basic texture loading in threads. It came down to it that the openGL API is completely thread allergic, however after a short discussion it was concluded that with today's cards (and the trend here isn't changing AFAIK), the pipeline would serialize all the calls in the end anyhow.

In other words, having a thread-safe API for gfx calls would bring only overhead (e.g: DX 11 or original concept of GL 3.0)

Just thought I'd mention this little bottleneck of the future (for games at least.. with 32+ cores, having only 1 doing the graphics I think could be fatal, especially since data throughput probably won't get much better on the main bus, e.g: RAM/Disk/VRAM transfers).

As a tidbit, I *heard* (rumor level truthfulness) that the "objective" GL 3.0 threadsafe concept was thrown away in the winter, and that GL 3.0 will be a continuation of GL 2.0, not a rewrite. Take with a pinch of salt, news on this front will arrive soon :)

Ales

Boian Mitov wrote / napísal(a):

Probably because C++ OOP model sucks so much, it could outsuck microsoft in vacuum cleaners :)

Ales

Well... This is probably the reason we develop our own libraries ;-) . Our architecture was multithreaded since it was originally envisioned 7 years ago. We still use other libraries but we call them from separated threads, and we split the tasks upfront. In our latest design each video filter will optionally perform in a separated thread thus distributing the video or audio processing. We even plan to add distribution for large video frames where each processor will be assigned only portion of the frame thus utilizing multicore even in a single filter.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Nobody said that mt support is limited to smp systems.

To be honest I hate the Pascal OOP implementation even more, but the lack of Singleton support, good RTTI and AfterConstructor/BeforeDestructor . ;-) . The OOP model is something we as developers implement. We are not talking MFC, or VCL, but the language. There are a lot of fantastic OOP models in C++, and C++ supports conceptual programming (Well kind of), where Pascal does not.
To be honest both languages have a lot of issues, but we take them for what they are and use them and love them :-) . I love both C++ and Object Pascal, well maybe C++ a bit more, but is still frustrates me when I can't create a simple singleton :-D .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

"Vinzent Höfler" wrote

a desktop system with hundreds of processors in the next few years or so. Practical benchmarks these days have shown, that a Quad-Core gains almost nothing compared to a cheaper Dual-Core in most practical circumstances.

That's probably due to the currently available software all being single threaded. I remember back in the day when I used OS/2. Even though it ran on a single core system, it felt a lot more responsive that todays OS's, because most apps and the OS + WPS used multiple threads extensively. Quicker visual feedback played wonders to the illusion that it was fast. Now with todays multi-core systems it will not simply be an illusion.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

I meant to say I hate the lack of Singleton support, good RTTI and AfterConstructor/BeforeDestructor in C++ .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

-------- Original-Nachricht --------

Well, there's a subtle difference between "parallel" and "async": whilst the for-loop may run in parallel, once you got to the statement after it, all iterations are finished. The same is not true for an async(hronous) procedure.

Vinzent.

Boian Mitov wrote / napísal(a):

I can't like an OOP model in which you:
1. can't call virtual methods from constructors
2. can't safely do stuff in constructors without using exceptions
3. can't limit the constructors being called (e.g: call order a, b, c)
4. no metaclasses or RTTI

These are all language ad-hoc design limitations of C++, I'm sure there are more, but I personaly hit these. Perhaps I'm just spoiled by good OOP tho :P

I'm curious however, what do you mean by "conceptual programming"? Can you provide an example?

Ales

then another procedure/keyword isfinished(x)? which checks whether the 'parallel' procedure x has finished.

This is what they do with with the "future" keyword, which I find very appropriate here.
-> http://wiki.remobjects.com/wiki/Futures

If we would restrict an implementation to parallelizing of complete procedures the syntax would be

var
  f: future Integer:
  n: Integer;
function x(i: Integer);
begin
...
end;
begin
...
  f := x(3)
...
  n := f
..
end.

Thus a "parallel" procedure would always be a function that returns a value that implicitly shows its execution state.

Note that the "parallel" keyword is not even necessary here, as the "thredenizing" of the procedure is done when calling it (f := x(3) ). the "parallel function" keyword _might_ be useful to denote which functions can be used with "future" results, but I don't see that the function compiling itself needs to know.

The execution waits for the thread to be ready when reading the value of f. At this point there are three possibilities:
- thread already done: just use the value
- thread not yet started: the procedure is just called in place.
- thread still running: wait until ready

BTW there might be a huge difference between a future array of values and an array of future values: last would allow for a big number of threads to be spawned.

-Michael

Before we discuss endless about useless stuff, I'll make a proposal for a first addition: support of the synchronized keyword. It does no more than protecting a procedure automatically by a critical section so only one thread can enter it. But I want to extend it. First, it can be applied also to classes and object:

tmyclass = class(...) synchronized

This means that all classes synchronize against one critical section.

Further, it allows an additional symbol to be given

procedure p;synchronized mycriticialsection;

it must be of the type System.TRTLCriticalSection and the procedure will be synchronized against this critical section.

Besides saving the typing of endless try ... finally statements and the initialiation of the unnamed critical sections, it creates also slightly better code because the LeaveCriticalSection call can be inside the implicit exception frame which is created anyways.

Any suggestions about doing it better?

BTW: I'll ignore comments like "nobody needs this", I need it :)

Can you explain this statement ?

All methods of this class are protected by the same critical section ? So in fact only one method of such a class is executing at the same time ?

Michael.

Well, there's a subtle difference between "parallel" and "async": whilst the for-loop may run in parallel, once you got to the statement after it, all iterations are finished. The same is not true for an async(hronous) procedure.

Very good point !!! !

That is why the "parallel" keyword should be reserved for loops (that internally run in parallel) and not be used for stuff that runs asynchronous (in parallel to something external to that scope).

This is why the "async" and/or "future" keywords are much more appropriate for procedures to run in the background. (Which might be the easiest paradigm to implement.)

OTHO this also points out that "parallel loops" are by far the easiest to used, as you don't need to care of any synchronizing outside of the loop.

-Michael

the "parallel function" keyword _might_ be useful ...

in fact "asyc function" if at all ...

-Michael

procedure x(const s:integer); var a:array[0..s] of byte;

I never understood why this is not possible in Pascal.

OTOH with dynamic arrays this is not really necessary any more, but non-dynamic arrays of course can be faster than dynamic arrays.

-Michael

Do you mean one global crit sect per class (and descendants) or one global crit sect for all classes?

Will this also allowed for methods?

I like it.

Mattias

direct Complex number support.

Do we still lack this .... ;)

-Michael

Op Thu, 31 Jul 2008, schreef Michael Schnell:

This is indeed the point, stack allocations are magnitudes faster than heap allocations.

Daniël

Yes.

So in fact only one method of such a class is executing at the same time ?

Yes. Acutally only one thread can enter the class.

Per class. The question is if it's possible to do this per class instance too.

Yes.

I agree with your points, and I listed some of them. Pascal has its own long list of issues, such as lack of functional and block enclosement (Especially true in Lazarus and free Pascal). Conceptual programming is what STL does as example. You declare a concept and then you do implementations. C++ conceptual support is not very user friendly however, but still works. They may add some more support for conceptual programming in the next ANSI standard, but I have not checked it out yet.
In conceptual programming you can create concepts that will work on anything. The concepts will work on Integers or objects as well as on any type of container. There are number of books on STL and conceptual programming.
Personally I think C++ is just not very good in it, but Pascal simply can't do it.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Just to make it clear. C++ is very good in conceptual implementation, but lacks any support for concept declarations as example, and there is no error checking :-( .
I have done some STL compliant containers in the past and validating the concept was a nightmare :-( .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

4. The best approach is to keep an eye on what the other compiler developers are doing, and try to follow the proven path ;-) .

That is why I suggest to take a look at .NET (and the pascal implementation of same (not Codegear's :) ), even if implementing the paradigms without a framework (which in fact on that behalf is something nothing but an external RTL).

-Michael

Michael,

CodeGear has not yet started working on the new features. I am not allowed however to mention what they plan to do, even if I know ;-) .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

-------- Original-Nachricht --------

You mean:

1) All classes of the type (and descendants)?
2) Only the methods additionally marked with a "synchronized" statement?

3) Corollary: A critical section is either class wide or instance wide? Meaning: you can't possibly use both approaches at the same time? For instance, having some methods that are synchronized between all instances existing (class wide) _and_ another set of methods only synchronized on the instance? If so, maybe the syntax below could be extended to specify how the synchronization should be done without the explicit use of another RTLCriticalSection? (Maybe "synchronized class" for the class-wide one, a simple "synchronized" for the instance-wide?)

OTOH, it's looks about the same as in Java and even Java now has explicit Locks, because "synchronized" simply isn't efficient nor flexible enough... ;)

Besides saving the typing of endless try ... finally statements and the initialiation of the unnamed critical sections, it creates also slightly better code because the LeaveCriticalSection call can be inside the implicit exception frame which is created anyways.

Hmm. Depends on the code. ;)

Any suggestions about doing it better?

Warnings, if you use potentially blocking functions inside a synchronized method? ;)

Like:

|procedure My_Thread.Do_Work; synchronized;
|begin
|   ...
|   Suspend;
|end;

Well, I guess the compiler doesn't know which operations are potentially blocking, does it?

Vinzent.

Yes. This was my idea so for.

2) Only the methods additionally marked with a "synchronized" statement?

No.

Yes, this is something to think about.
- class wide
- instance wide
- method/procedure wide
One overrides the other, i.e. if there is a class wide synchronized, the compiler will throw an error if there is a method one.

OTOH, it's looks about the same as in Java and even Java now has explicit Locks, because "synchronized" simply isn't efficient nor flexible enough... ;)

Since I don't use java, what's the difference of locks?

No.

This was added recently:
See language reference 'generics' (chapter 8 in pdf).

Mattias

How to avoid deadlocks? If a synchronized class is calling another synchronized class ... :-S.

Micha

Generics can't implement conceptual programming. As example for conceptual programming you can perform the same algorithm on simple arrays such as "char Array[ 1000 ]" as you can on any STL compliant container. The integrator concept is implemented different way but since it is concept as Sort algorithm will work. Generics do not allow that. With generics you can do Generics based programming but not concept based programming.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

I mean the "iterator" concept... Gee... a long night ;-)

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Yes. Acutally only one thread can enter the class.

Hmmm.

In my mind, a "class" is meant tor managing code and variables, while a thread is for managing execution contexts.

These two concepts don't have anything to do with each other: a class and an instance of same can be "associated with" or "executed by" several threads, a thread can execute several instances of a class.

E.g. Even the standard "TThread" defines a single thread (this is how it's code is implemented) , but the some functions and variables of same are/can be used by the thread, by another thread or by both.

Now you suggest that all procedures an instance of this kind of class are protected and thus thread-safe by definition.

This might be a really good thing, but needs some decent explanation as the bundling of code/variable administration and thread context administration is not a "natural" self-explaining concept.

-Michael

Per class. The question is if it's possible to do this per class instance too.

To me "Per class" does not seem appropriate at all. "Per instance" in fact seems to make sense.

-Michael

OTOH, it's looks about the same as in Java and even Java now has explicit Locks, because "synchronized" simply isn't efficient

I suppose if using Futex (in Linux) or an implementation of a user space Mutex in the RTL (using a bus locked read-modify write instruction and do a system call only when the resource is taken by another thread) needs close to no CPU time.

-Michael

Now you suggest that all procedures an instance of this kind of class are protected and thus thread-safe by definition.

They aren't but it makes things easier. Just look at the linux kernel: it's smp support started also with BKL: the Big Kernel Lock

Indeed, I'am open to any suggestions :) Just an idea: maybe a global call tree analysis with a call tree attributed with critical section information can find potential deadlocks?

If a synchronized class is calling another synchronized class ... :-S.

That's not so good example since now it's almost gone. That suggests this synchronized keyword would be deprecated in a few years as well ;-)

Micha

IMHO per class may create some problems? but maybe I am just overlooking something?

TBar =class( TFoo ) end synchronized class;

Let's say TFoo was declared in another unit, and TFoo was not declared Synchronized. Then TFoo code can be entered by anyone at any time. Even if a thread is in TBar. How would one make sure, that a base class, knows about all classes that inherit from it?

Neither can you say the baseclass must also be "synchronous class", because ultimately there always is TObject. and TObject does not want to be Synchronous class, or all Create/destroy would be in critical sections?

It my be possible to work around, this, but it would probably create a ig overhead, as every method one very class would have to perform checks at runtime.
Similar for instance wide protection.

Regards
Martin

Just look at the linux kernel: it's smp support started also with BKL: the Big Kernel Lock

And since that time they are working to get rid of it.

-Michael

Why does it need to know? Base classes can't access the fields of inherited classes anyway.

Micha

Come one, it's not that hard to understand :): synchronized might allow the same way of working: first, you work with big locks, later to add more fine granulated ones.

Funny thing, we went trough the same evolution with our libraries over the years :-D . We used to have a single lock for each connection chain. Now we have individual locks for each component and each OpenWire pin, and it works much better and faster.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Per instance would mean to need an algorithm to automatically define a new Mutex when creating an instance.

So maybe the TSynchronizedObject that might be the parent of all class(...) synchronized will define a TSynchronizedObject.Create that creates a Class-common or a system wide Mutex, while the create in a child class can modify this behavior to the use of an instance variable as the Mutex to have a per-instance protection.

-Michael

-------- Original-Nachricht --------

More control over where, when and how parts of the code must be locked. So you can do finer grained, and thus usually more efficient, locking. Just like the classic approach. ;)

A good example might be the multiplereader-singlewriter locks. Doing that with synchronized can be really inefficient if a protected data structure gets written seldom, but read often.

That's BTW why I like the Ada approach of the protected object much more:

protected Buffer is
entry Put (I : in Integer); -- Potentially blocking (if internal queue is full) entry Get (I : out Integer); -- Potentially blocking (if internal queue is empty)

   procedure Write_Something;  -- May write to the object, so can only be entered once, but never waits.
   function Read_Something;  -- Only reads, so this subroutine can be entered several times. Never waits.
end Buffer;

The basic idea is: entries may block (on guards = "condition variables"), procedures can only be entered once at a time, because they may change the object's state and functions can be entered concurrently. Of course, if a write operation currently occurs (procedure call), no read operation (function call) is allowed during that time. In that way, the whole data structure and operations on it are relatively well defined even in temporal terms.

Vinzent.

My personal experience so far has confirmed this. Reducing the scope and the duration of the locks, and allocating individual locks to individual resources increases the performance and oddly reduces the danger of deadlocks. I used to spend days tracking deadlock conditions. Hardly nowadays after allocating individual locks per critical resource, and carefully defining the scope of each lock. Having single centralized lock is hardly a good approach.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

-------- Original-Nachricht --------

Yes, but often the question is not how long the *locking* operation takes, but rather how long the *locked* operation takes.

Vinzent.

Yes, but often the question is not how long the *locking* operation takes, but rather how long the *locked* operation takes.

Of course you are right, but this is not a compiler / Library thing but depends on the design of the user program.

-Michael

-------- Original-Nachricht --------

Of course, but sometimes it's simply not feasible to split the algorithm into several "synchronized" subroutines. Especially when multiple locks are involved.

Vinzent.

Hello,

In practice imo not too much, synchronized is just a convenience functionality, exactly like the one you proposed.

In short the difference:

Java always locks on monitors i.e. guards which are available for any object; Pre 1.5 the only way to lock was using some syntax additions. The syntax allows per method (class or instance) or per block scope, i.e.

synchronized void doSomething() { ... } // method level: monitor object is either "this" or this.class object

void doSomething() {
...
synchronized (monitor) { // on block level; monitor object is "this" if not specified otherwise the given object }
...
}

With 1.5 they added a "Lock" interface which provides the usual lock ()/trylock()/unlock() methods which should be self-explaining. This for example allows interlocked locks if required

lock1.lock();
...
lock2.lock();
...
lock1.unlock();
...
lock2.unlock();

Hth,
Thomas

This is an idea using automatic cleanup of interfaces:

procedure foo; <- lock
begin
  synchronize(cs);
  [...]
end; <- unlock

or

procedure foo; <- lock
begin
   with synchronize(cs) do <- lock
   begin
     [...]
   end; <- unlock
end;

{ TLock }

type
  TLock = class(TInterfacedObject)
  private
    FCriticalSection: TCriticalSection;
  public
    constructor Create(cs: TCriticalSection); virtual;
    destructor Destroy; override;
  end;
constructor TLock.Create(cs: TCriticalSection);
begin
  FCriticalSection := cs;
  cs.Acquire;
end;
destructor TLock.Destroy;
begin
  FCriticalSection.Release;
  inherited;
end;
function synchronize(cs: TCriticalSection): IUnknown;
begin
  Result := TLock.Create(cs);
end;

2008/7/31 tom_at_work

Hmm... it looks almost one to one copy from our code in Version 4.0 of our libraries ;-) . Are you one of our customers, or you have simply come with the same idea as us?

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

BTW: As it was discussed a while ago this does not work so well with Lazarus due to some "optimizations" they do with when a reference is decremented. With some tricks however it seems that we are on our way to get it to work under Lazarus as well.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

sorry, it is also my idea :D
I recently written a new light db interface for firebird, to simplify queries
and auto commit transactions (context)

  with pool.GetConnection.newContext do <- start transaction
    Execute(newCommand('DELETE FROM COUNTRY WHERE COUNTRY = ?'), [1]);
  <- commit/rollback transaction

I also remember MS DirectShow SDK (to write codecs) use an object allocated in the stack to syncronyse critical sections, it look like our trick with the interface

2008/7/31 Boian Mitov

In C++ this is a classical block enclosement. I have used for few years in Delphi with interfaces. I was badly surprised that there are some issues when you try to use it in Lazarus, but with some code changes I think I have worked it around, and so far it seems to work in Lazarus too, just with some limitations ( Destructors as example ). In Delphi however there is a different problem. In C++ the order of destruction will be exactly opposite of the order of creation. There is no such guarantee in Delphi, so having 2 locks declared leads to problems due to wrong order of destruction. This is yet another example where C++ simply shines ;-) .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Boian Mitov wrote / napísal(a):

I also thought you ment generics.

Please provide (for example C++) code example, I still don't get what you mean.

Ales

Henri Gourvest wrote

This is an idea using automatic cleanup of interfaces:

That is so cleaver! :)

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Sort for example:

It can work with C type array:

int a[7] = {23, 1, 33, -20, 6, 6, 9};
sort(a, a+7);

or it can work with a container such as linked list:

list<Something> v1;
sort(v1.begin(), v1.end());
  With best regards,
    Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

This is what all C++ people do all the time ;-) .
In Pascal we have to use the Interface trick.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

I actually posted fragments of our code couple of moths ago during the interface decrement discussion.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Boian Mitov wrote

Hmm... it looks almost one to one copy from our code in Version 4.0 of our libraries ;-) . Are you one of our customers, or you have simply come with the same idea as us?

I'm not Henri, but we did similar setting and auto-restoring mouse cursors in our Delphi applications. I can't imagine why I never thought of using the same idea for critical sections. :-)

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

We have been using this for the last 4 years, although we switched from MuliRead locks to lightweight CriticalSections in V 4.0 . We simply increased the number of locks and didn't need the multiread functionality any more so we reduced the time spent locking. The interface functional enclosement however is today the same as it was 4 years ago ;-) . I first used this trick when I developed the OpenWire initially - www.openwire.org .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Boian Mitov wrote

I actually posted fragments of our code couple of moths ago during the interface decrement discussion.

I'll go have a look. :) Thanks to Gmail, I have about 2-3 years of archived mail. I don't use interfaces much (and don't know them very well), but starting to like them more and more. Some solutions are so much more elegant when using interfaces.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

You can download the 3.1 version of OpenWire from www.openwire.org . It still uses the MultiReads. I have a new version that compiles under Lazarus, although the editors are not fully functional under GTK in Linux yet. The Checkboxes don't show :-( . I will have to do some more work, but it will be after Delphi 2009 is out. Our main focus is the D2009 release, and Lazarus is on hold for the next 2-3 weeks.
I have also included OS independent versions of Mutex, Semaphore Event etc. in the new version. If you want me I can send you the latest sources. They also include the workarounds for the Lazarus reference counting "optimization", that used to break the old version.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

I would make one comment. This type of keyworded critical sections are very nice. To make it really powerfull consider the following declaration:

// main function which performs a task, parts of it in parallel
procedure BigFunction(ATaskCount: Integer);
var
   fStartEvent: TEvent;
   // Local function which does parallel work:
   procedure MyFunction(X: Integer); parallel;
   begin
     // typically wait before the main function sets up whatever needs to
be set up
     // - global vars (for BigFunction), etc.
     fStartEvent.WaitFor;
     // do something
   end;
   var
     exec_parallel: array[1..ATaskCount] of MyFunction;
   // The instantiation of ATaskCount parallel functions is done at the
declaration of the array
   // They all hold until the fStartEvent is set in the main functions.
   // there is a problem - TEvent is a pointer to a class which is not
instantiated at
   // declaration time
begin
   // Set up global data for parallel tasks
   fStartEvent.SetEvent;
   //
end;

This code is loosely borrowed from the way parallel code is done in ADA. It has been a while, but as far as I remember the parallel features in ADA were really elegant and well thought-out. So I would suggest to study carefully the parallel syntax that is intrinsic to ADA. The key thing there were semaphores (critical sections) and arrays of functions which are run in parallel;

In order to make the above code work, semaphores and events have to become internal types to Pascal, so they can be instantiated at declaration time. One can also extend the procedure/function syntax by including a WaitFor feature:

...
function MyFunc(x: integer): TObject;
begin
end

....
//the following statement would call the function and then wait until it terminates.

  case WaitFor(MyFunc(20), time_out) of
    csSignaled: ...
    csTimeout: ...
  end;

Best regards to all.

Peter Popov

You can download the 3.1 version of OpenWire from www.openwire.org . It still uses the MultiReads. I have a new version that compiles under Lazarus, although the editors are not fully functional under GTK in Linux yet. The Checkboxes don't show :-( .

Have you reported the bug?

Note: Free Pascal Lazarus. This is a fpc list so please use the correct name.

Mattias

----- Original Message -----

Boian Mitov wrote / napísal(a):

This you mean?

template <class RandomAccessIterator>
  void sort ( RandomAccessIterator first, RandomAccessIterator last );

You can do this with generics. Just use sizeof() to get to next element and overloaded comparator functions for base types. I don't really see much new here, it's a bit non-typical generics usage.

Here's a pascal example (OOP, true but that's a current limitation, afaik "generic function" should be possible):

program Project1;
{$mode objfpc}{$H+}
type
  { TSorter }
  generic TSorter<T> = class
   public
    procedure Sort(var First, Last: T);
  end;

{ TSorter }

procedure TSorter.Sort(var First, Last: T);
var
  x, y: ^T;
  z: T;
begin
  x := @First;
  y := @First;
  while x^  Last do begin
    while y^  Last do begin
      Inc(y);
      if x^ > y^ then begin
        z := x^;
        x^ := y^;
        y^ := z;
      end;
    end;
    Inc(x);
    y := x;
  end;
end;
type
  TIntSorter = specialize TSorter<Integer>;
var
  s: TIntSorter;
  a: array[1..5] of Integer = (5, 4, 3, 2, 1);
  i: Integer;
begin
  s := TIntSorter.Create;
  s.Sort(a[1], a[5]);
  for i := 1 to 5 do
    Writeln(a[i]);
  s.Free;
end.

Ales

And the same code will work both with normal array and with object implementing iterators ?
I am not sure it will.
How I can call your sort for a linked list container instead on array? Am I missing something or your code works with arrays only. The STL sort will work with arrays or with STL containers or with any class that implements the concept (I.E. defines iretator and reference types, and has iterator generating function etc.)
As the best of my knowledge this is not doable with generics. C++ is doing it, but there is no validation mechanism aside from just plain old testing all possible combinations ;-) .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Boian Mitov wrote / napísal(a):

Was just a case example.

I looked at their code, and it's full of hacks. Practically, they do what I did for basic types (e.g: arrays) with overloads. For classes they do a runtime detection wether it is a container implementing the interface and then procede that way.

I'd be doable in pascal too, but I'm still dizzy from looking at their code :)

Ales

Note that I'm not trying to do a "who can piss further" thing here. Just saying that IMHO the way C++ does this isn't exactly great.

The idea behind it is nice, however apart from pure OOP approach I don't see much choice of how to do this nicely.

Ales

I agree 100%.
They are not doing hacks however. I have read some of the books written by them, and they have followed a very strict scientific approach. It really made my head spin at the time ;-) . I have done some STL compliant containers, and yes I hate it, but I was able to do it. There are better languages for conceptual programming, from what I have heard but we seem to be stuck with C++ and OPascal for the moment ;-) . We will do our best with them.
Hopefully the next C++ standard will help a bit in this area and will allow concept declarations and validations.
The problem is that conceptual programming actually is a totally different concept from the OOP and one of the requirements is that the concept must be hierarchy impendent, so you can make any existing class concept compliant at any level and point.
I made existing library STL compliant without touching the hierarchy which I was not allowed to do any way and it worked.
All this simply does not seem to be doable with generics.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

I think, this sounds like a misuse of interfaces.
What you really need is an auto referenced class, that guarantees order and short life time.

Mattias

I agree, but we don't have anything like this in Delphi, now do we?

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

one thread can enter it. But I want to extend it. First, it can be applied also to classes and object: tmyclass = class(...) synchronized

A class protected by a sync object is a monitor. Several other languages have this as a first class citizen.

TMyClass = Monitor (xxx)

This means that all classes synchronize against one critical section.

For the straight case this is ok, but what about virtual methods ?

For methods or classes? I think single classes with an own syncobj are the rare case?

I'd say 'another use for refcounted interfaces'.

Joao Morais

Well, one of the points with OS/2 is that the OS handles thread (and process) switching rather efficiently (without burning too many resources just on doing this); something not necessarily true for other operating systems including their most recent versions.

Tomas

Boian Mitov wrote

BTW: As it was discussed a while ago this does not work so well with Lazarus due to some "optimizations" they do with when a reference is decremented.

No, as explained roughly ten times by now it's due to some optimisations which we do *not* do, and that assumption is wrong in Delphi as well because this particular optimisation is simply unsafe in certain cases. So please, finally, stop propagating and endlessly repeating this misinformation.

See http://www.hu.freepascal.org/lists/fpc-devel/2008-June/014005.html for the full explanation.

Jonas

Jonas,

Thank you!
I disagree with your definition of what is optimisation, but I accept the FPC as it is. Being aware of the way it is implemented in FPC is very important to the other developers, and I was just making them aware of the danger. I am not saying you are doing it wrong (At least not any more), I am warning them that the problem exists, and they have to take some measures not get burned. I have taken such measures already, and the code seems to work with the workarounds I have implemented.
I don't understand why you are reacting as if I am offending you when I am simply pointing that the specific implementation of the refference counting leads to unexpected side effects, and certain measures should be taken especially if you use the interfaces for thread locking. If there is nothing wrong with that why you are behaving so defensively? Nobody is attacking you.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

In order to have peace and to make Jonas happy, please all read "optimizations" as "unoptimizations" in all my messages to this point. Please accept my apologies for using the apparently wrong term when describing the problem.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Jonas, Thank you! I disagree with your definition of what is optimisation,

Well, and this is exactly the point were you are wrong (as proved by several code samples you ignored) and that's why Jonas reacted the way he did.

Please peace!
I will not change my opinion but I will restrain from mentioning the word "optimization" in this context.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Boian Mitov wrote

I disagree with your definition of what is optimisation

Optimisation = do not use a temp, but directly assign a function result to its eventual destination. FPC's non-optimisation is that it does use intermediate temps in some cases where Delphi doesn't.

I don't understand why you are reacting as if I am offending you

Because we have already gone through several cycles now of a) you mentioning this alleged optimisation that FPC does which supposedly breaks interfaces behaviour compared to Delphi b) several people explaining that it is not an FPC optimisation and that it is wrong to count on it Delphi too (since it will go wrong there as well under various circumstances)
c) you saying thank you for the explanation
d) you again mentioning this alleged optimisation that FPC does which supposedly breaks interfaces behaviour compared to Delphi e) ...

Since you're not stupid, the only other explanation I can come up with is that you are simply saying "thank you" to shut people up, but apart from that are completely ignoring what they actually say. And I find that indeed offensive.

Jonas

You are making a good point. I may be not that smart after all. I have been enormously frustrated by this issue, but I spent 2 weeks rewriting our code and I seem to have implemented a reasonable workaround. It has coasted me a lot, and I still feel the pain from it, so I am probably still venting "unreasonably", but believe me not with bad intent, it just hurts a lot. Hopefully it is over, I am just scared of what other surprises are lurking there for me ;-) .
If I have dedicated 2 weeks of work, to make sure 4 Mln lines of code are properly going to work with FPC and Lazarus, it means I am passionate enough toward the product, and I am committed to put the extra effort to support it. Passion is a double edged sword however, and comes with its dangers ;-). I am a bit too passionate I guess, and in this case it works against me and seems to hut other people, when hearting was not the intention. Please accept my apology, and understand that this is not attack against you, just display of frustration with a certain implementation that has driven me nuts for long time.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Boian Mitov wrote

Please accept my apology

Apology accepted. And good luck with your review process!

Jonas

I not only owe the haplology, but I also must say how impressed I am by the work of both FPC and Lazarus teams. I have been able to roughly port the libraries within mater of days, and some of the stuff is already working. The Interfaces issue was by far the biggest problem I have encountered, almost everything else went straight forward with minimal efforts. The teams have achieved astonishing and unbelievable quality of the project, and deserve congratulation, and gratitude for their work. I don't know if any other Delphi vendors have started supporting Lazarus yet, but I am sure we will have it supported within few months. If more vendors start supporting it, the project popularity will be ensured ( I hope :-) ).

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Just some additional testing info and things to think about.

*) Performance of non granular locks is poor.
*) Creating 50K+ of lock instances makes the OS behave strange, so to fine granualer does not help either :-)

Simplest working Deadlock Avoidance strategys is:

Lock all resources at the same time in the same order.

Thread x
LOCK A
LOCK B
LOCK C
... Work ....
UNLOCK C
UNLOCK B
UNLOCK A

Thread x
LOCK B
LOCK C
... Work ...
UNLOCK C
UNLOCK B

This way deadlocks are impossible.

As this is sometimes not possible - we use deadlock detection in our software.
(as classical database servers are doing it :-))

I describe it brief:

Everytime we lock in a thread we post a event to an non-blocking queue (Michael&Scott),
similar on unlocking. The MS Queue is fast and does not block. At the other side we
Detect the deadlocks by time (like a asynchrounous logger). All our Locked operations are designed to take a small amount of time (say some milliseconds). When we detect a nonpairing timedifference of say one second, we found the deadlock and have to correct the code.

This is a debugging tool, wich does not save design time. But saves huge amount of time
searching for deadlocks.

IMHO a full graph-theoretic cyclic redundancy check would be to much waste of time - inefficient.

Another aproach would be STM. (software transactional memory).

For theory about locks:

Hector Garcia Molina, j.Ullman, J. Widom / DATABASE Systems the complete Book.

Or hardcore: http://www.cs.rochester.edu/~scott/

Greets,

helmut

Hi Helmut,

This is great for relatively simple system with number of shared objects. In our case we have a graph of up to hundreds of objects and not all of them are connected (i.e. there are multiple independent graphs). The connections can be changed while the graphs are working, and all components can send data in any direction. and we need the components to do their job in separated threads. I have experimented with multiple approaches over the last 5 years, all the way up to this year we used a relatively small number of locks based on the chains in the graph. It never worked well until we finally switched to full granularity 3 months ago. For first time we see an very high throughput, and no deadlocks :-) . Simple strategies are great, and what you describe is what we usually do in relatively simple applications with limited threads and shared resources, but it is not universal solution, and unfortunately is not always applicable. Other solutions need to exist, to solve different scenarios.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Here is example of just simple linear chain:

A -> B -> C ->D

In this case there are no even any splits or merges, and yet the problems become obvious.

A can start sending data to B
and D can start sending let say clock events to C

Each component has to run in a separated thread as A may be capturing data from a device or receiving it from the web and unzipping it. B may be filtering the dada, C maybe performing 2D or 3D graphic transformation, and D maybe is compressing and storing the data in a file. Obviously you need them to run on separated cores if possible, and be locked the minimal amount of time.

Our granularity mechanism is a to lock the A component when changing data in it, then unlock it and lock the connection A->B send the data to B then unlock the connection and lock B to deliver the data to it and unlock it immediately and it continues.
This way each thread own only one lock at the time and eliminates the deadlock danger.
Otherwise deadlocks are inevitable :-( .

Our previous design was to have a master lock we ware obtaining then tracing and locking the chain we ware to deliver to, then unlocking the master lock and delivering trough the chain and then unlocking it in left to right fashion. Obviously a sick design born from a sick mind (i.e. mine ;-) ), hopefully at some point I figured I am insane, and obvious improvement :-D .

As you can see there are obvious scenarios where the granularity approach seems to be the only reasonable one.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Here are 2 examples of relatively simple graphs. The synchronization problems are obvious:

http://www.mitov.com/VLDemo_Combine_Diagram.jpg
http://www.mitov.com/SLDemo_FFT_Diagram.jpg

Those are graphs from just our demos for the customers. Obviously a real case scenario can be much more complex.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Boian Mitov wrote

As you can see there are obvious scenarios where the granularity approach seems to be the only reasonable one.

I think you misunderstood me:
I wanted to say:

*) Maximum granularity gives maximum performance, but as always there is a tradeoff.
*) Problems come in if you have DIFFERENT resources to lock and are not able to maintain the right
ORDER of locking, in the timeschedule.

Locks are well researched but yesterdays technology, and there is no need to reinvent the wheel,
because there are exising published DEADLOCK Avoidance and DEADLOCK Detection Algorithms available
for decades ... (Start here http://www.acm.org/sigs) (So IMHO best would be to use existing and mathematical profen strategies)

Please take a look at the papers at: http://www.cs.rochester.edu/~scott/

New directions are STM and Non Blocking Algorithms.

I want not to criticise, understand your design, nor i want to make it better :-)
Just some state of the art research info - read it or ignore it - what ever makes you happy ...

There are much smarter people then we are maybe, at least for me this holds,
inventing new wheels ...

Greets,

helmut

Hi Helmut,

Thank you!
I agree with you.
I was just showing a case when the approach you ware describing is not applicable. I did not think you ware criticizing me or something ;-) . We both ware just showing approaches we use, and they both are very much classical one. I fell into the same pitfall as what they have. Unfortunately although we don't have any deadlocks at the moment, we are also thinking for some sort of automated deadlock resolution just in case. In our systems the developers can put the components in all kinds of configurations and expect them to work 24/7 (If they are in the Military or a TV broadcasting or in a Robot), and we are always afraid we may have overlooked some configuration where they will run into a deadlock. I agree however that while automatic resolution is a good thing to have if possible you must not rely on it, and mush have lock design that is not intended to produce deadlocks. The automatic resolution however is just a nice additional protection just in case things go wrong somehow.
Once again your points are 100% on the mark! I was just probably adding some small aspects to them ;-) .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Hi Folks,

I am trying to test linking GCC++ code to FPC code in Linux. It seems to work so far, but I don't know where the TObject is implemented. What library I have to link to in order to get the TObject ?

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

I am trying to test linking GCC++ code to FPC code in Linux. It seems to work so far, but I don't know where the TObject is implemented. What library I have to link to in order to get the TObject ?

Probably objpas.

Hi Marco,

Thank you!
I can't find a library with this name, only directory with sources. Do you know what is the name of the library it compiles to ? "rtl" something ?

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Boian Mitov wrote

I can't find a library with this name, only directory with sources. Do you know what is the name of the library it compiles to ? "rtl" something ?

FPC Makefiles by default do not generate any libraries, only separate object files. objpas is indeed part of the rtl, and the file you are looking for is called objpas.o

Jonas

Hi Jonas,

Thank you!
I found it, but I am stull getting the same unresolved references plus a bunch more ;-) .

Here are the references I am missing:

mytest1.o: In function `TMYCLASS__CREATE':
mytest1.pas:34: undefined reference to `FPC_PUSHEXCEPTADDR' mytest1.pas:34: undefined reference to `FPC_SETJMP' mytest1.pas:35: undefined reference to `SYSTEM_TOBJECT_$__CREATE$$TOBJECT' mytest1.pas:35: undefined reference to `FPC_POPADDRSTACK' mytest1.pas:36: undefined reference to `FPC_RERAISE' mytest1.pas:34: undefined reference to `VMT_SYSTEM_TOBJECT' mytest1.pas:34: undefined reference to `SYSTEM_TOBJECT_$__DESTROY' mytest1.pas:34: undefined reference to
`SYSTEM_TOBJECT_$__NEWINSTANCE$$TOBJECT'
mytest1.pas:34: undefined reference to `SYSTEM_TOBJECT_$__FREEINSTANCE' mytest1.pas:34: undefined reference to
`SYSTEM_TOBJECT_$__SAFECALLEXCEPTION$TOBJECT$POINTER$$LONGINT' mytest1.pas:34: undefined reference to
`SYSTEM_TOBJECT_$__DEFAULTHANDLER$formal'
mytest1.pas:34: undefined reference to `SYSTEM_TOBJECT_$__AFTERCONSTRUCTION' mytest1.pas:35: undefined reference to `SYSTEM_TOBJECT_$__BEFOREDESTRUCTION' mytest1.pas:35: undefined reference to
`SYSTEM_TOBJECT_$__DEFAULTHANDLERSTR$formal'
mytest1.o: In function `TMYCLASS__TEST11':
mytest1.pas:40: undefined reference to `VMT_SYSTEM_TOBJECT'

They don't seem to be in the objpas.o .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

I got that part. system.o , but now I have some more unresolved:

/usr/lib/fpc/2.2.2/units/i386-linux/rtl/system.o: In function `SYSTEM_FINDRESOURCE$LONGWORD$PCHAR$PCHAR$$LONGWORD': system.pp:(.text+0x136): undefined reference to `FPC_RESLOCATION' system.pp:(.text+0x1ac): undefined reference to `FPC_RESLOCATION' system.pp:(.text+0x1ee): undefined reference to `FPC_RESLOCATION' /usr/lib/fpc/2.2.2/units/i386-linux/rtl/system.o: In function `SYSTEM_LOADRESOURCE$LONGWORD$LONGWORD$$LONGWORD': system.pp:(.text+0x231): undefined reference to `FPC_RESLOCATION' system.pp:(.text+0x242): undefined reference to `FPC_RESLOCATION' /usr/lib/fpc/2.2.2/units/i386-linux/rtl/system.o:system.pp:(.text+0x24e): more undefined references to `FPC_RESLOCATION' follow /usr/lib/fpc/2.2.2/units/i386-linux/rtl/system.o: In function `SYSTEM_INITSYSCALLINTF':
system.pp:(.text+0x33b): undefined reference to
`operatingsystem_parameter_envp'
/usr/lib/fpc/2.2.2/units/i386-linux/rtl/system.o: In function `SYSTEM_SETUP_FASTMOVE':
system.pp:(.text+0x15f5): undefined reference to `__fpc_valgrind' /usr/lib/fpc/2.2.2/units/i386-linux/rtl/system.o: In function `SYSTEM_FPC_CPUCODEINIT':
system.pp:(.text+0x1db2): undefined reference to `__fpc_valgrind' /usr/lib/fpc/2.2.2/units/i386-linux/rtl/system.o: In function `FPC_INITIALIZEUNITS':
system.pp:(.text+0x103a1): undefined reference to `INITFINAL' system.pp:(.text+0x103b4): undefined reference to `INITFINAL' system.pp:(.text+0x103be): undefined reference to `INITFINAL' system.pp:(.text+0x103c6): undefined reference to `INITFINAL' /usr/lib/fpc/2.2.2/units/i386-linux/rtl/system.o: In function `FPC_FINALIZEUNITS':
system.pp:(.text+0x103f6): undefined reference to `INITFINAL' /usr/lib/fpc/2.2.2/units/i386-linux/rtl/system.o:system.pp:(.text+0x103fb): more undefined references to `INITFINAL' follow
/usr/lib/fpc/2.2.2/units/i386-linux/rtl/system.o: In function `SYSTEM_INIT_ALL_UNIT_THREADVARS':
system.pp:(.text+0x12f1d): undefined reference to `FPC_THREADVARTABLES' system.pp:(.text+0x12f34): undefined reference to `FPC_THREADVARTABLES' /usr/lib/fpc/2.2.2/units/i386-linux/rtl/system.o: In function `SYSTEM_COPY_ALL_UNIT_THREADVARS':
system.pp:(.text+0x12fad): undefined reference to `FPC_THREADVARTABLES' system.pp:(.text+0x12fc4): undefined reference to `FPC_THREADVARTABLES' /usr/lib/fpc/2.2.2/units/i386-linux/rtl/system.o: In function `SYSTEM_SYSTEM_EXIT':
system.pp:(.text+0x16ed0): undefined reference to `_haltproc' /usr/lib/fpc/2.2.2/units/i386-linux/rtl/system.o: In function `SYSTEM_PARAMCOUNT$$LONGINT':
system.pp:(.text+0x16ee1): undefined reference to
`operatingsystem_parameter_argc'
/usr/lib/fpc/2.2.2/units/i386-linux/rtl/system.o: In function `SYSTEM_PARAMSTR$LONGINT$$SHORTSTRING':
system.pp:(.text+0x16f21): undefined reference to
`operatingsystem_parameter_argc'
system.pp:(.text+0x16f2a): undefined reference to
`operatingsystem_parameter_argv'
/usr/lib/fpc/2.2.2/units/i386-linux/rtl/system.o: In function `SYSTEM_SETUPCMDLINE':
system.pp:(.text+0x16f90): undefined reference to
`operatingsystem_parameter_argc'
system.pp:(.text+0x16fc6): undefined reference to
`operatingsystem_parameter_argv'
system.pp:(.text+0x16ff6): undefined reference to
`operatingsystem_parameter_argv'
system.pp:(.text+0x17039): undefined reference to
`operatingsystem_parameter_argv'
system.pp:(.text+0x1706c): undefined reference to
`operatingsystem_parameter_argc'
system.pp:(.text+0x17092): undefined reference to
`operatingsystem_parameter_argc'
/usr/lib/fpc/2.2.2/units/i386-linux/rtl/system.o: In function `SYSTEM_init': system.pp:(.text+0x17516): undefined reference to `__stklen' system.pp:(.text+0x17556): undefined reference to `__stkptr'

Any idea?

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Boian Mitov wrote

I got that part. system.o , but now I have some more unresolved:

Some of these are defined in the startup code (in case of Linux/i386: si_prc.o for pure Pascal program, si_c21.o for programs linked against libc). Others are symbols generated by the compiler in the object file of programs (e.g. INITFINAL is a table containing pointers to the intialisation and finalisation code of all used units).

I think it would be easier if you just compiled the Pascal code you want to use into a dynamic library and use that, rather than trying to manually replicate the whole main program compilation and linking process.

Jonas

I am trying to see if I can create C++ project that uses some LCL components, and what will be involved. I will also try the shared library approach, but I want to see if the direct linking into a single executable is possible.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Is there a tool that will allow me to find object or library file containing a specific entry in Linux?
I was experimenting with shared libraries but realized that it will not work for me.
I plan to make LCL based libraries but they can link only statically. The components are in hierarchies so if somebody needs to use 2 of the libraries they will not be able to work and link properly together as the same entries will be present in both shared files. Unfortunately Lazarus does not support dynamic LCL packages as far as I understand, so static linking seems to be the only available option.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

I think I got it, something along these lines :

for file in `ls -1 *.o`
do
echo "In file : $file"
nm $file
done

I am a bit rusty on the Unix side, but brushing up ;-) .

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Hi Folks,

I am doing a translation of the hrtimer functions in Linux as I will need them for my libraries. I was looking into the "libc" unit. I am wondering how you guys are keeping it in sync with any changes done in the libraries by the C++ folks? I suspect this being such a core library is probably kept highly backward compatible, but I am just curious. Are you doing it by manual changes/additions or you use some sort of automated header -> cpp converter?

After I get the hrtimer header converted if you want it I can send it to you for review and inclusion in the "libc" *.inc files.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Hi Michael,

Thank you!
I meant the "C" people, when I said C++. Sorry.
I meant maintaining compatibility with the "C" header files. For now all the code I have is Intel only, so libc should do it for me. Thank you for the information. I am very new on Linux and still trying to learn the ropes. I am figuring out the Linux equivalents of the Windows APIs I am using, and I am trying to add the necessary changes to our libraries. Multimedia Timers is one of those APIs and I discovered that the hrtimer seems to be the correct Linux API for that, and I am working to port it to Lazarus and test how it will work.

I will send the patch when I get it done.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

We're not tracking what C++ does; I doubt that the C++ people have anything to do with Libc, since it's a plain C library ?

The initial conversion was done using h2pas, with manual additions for Kylix compatibility. Maintenance is done manually. We accept patches if things have changed or they added some calls, but we're not actively tracking that.

After I get the hrtimer header converted if you want it I can send it to you for review and inclusion in the "libc" *.inc files.

Feel free to do so; Like I said, we accept patches.

Note that libc use is limited to i386-linux.

Michael.

Looking forward to it.

Michael.

Op Sun, 10 Aug 2008, schreef Michael Van Canneyt:

This is not the right approach. Libc is a Kylix compatibility unit, anything that Kylix doesn't have does not have to be provided by FPC (after all this is, and i386 assumptions by Borland is why it is 386 only).

If you want to use high resulotion timers, please integrate them with FPC's Unix access units.

Now, if you have Kylix code and absolutely need to use high resultion timers in your code, I am not principally against adding them to the libc unit. But in principle, libc should not be used for new code.

Daniël

Hi Daniel,

Thank you!
As I mentioned I am very new, and still don't know the exact rules of the purpose of specific units. I am not sure if the timers are available on other systems than Linux yet. I am sure there is some form of equivalent. I actually don't have any Kylix code. I am porting Windows code, actually making it cross platform as we support Delphi 5-2007, and plan to support Lazarus for Windows and Linux for now, and hopefully for other platforms in the future. Porting huge code from Windows is slow and error prone process, so I am doing it in stages. For now I have made everything to recompile under Lazarus in Linux, and now I am filling the items I had to cut from the code during the conversion. In Windows I use the Media Timers, and the only equivalent thing I have found in Linux is the High Resolution timers. I have been studding the FPC and Lazarus code, and I discovered the libc, and it seemed as the logical place such timers to be implemented. If they belong to a different location, this is fine. I can also keep them in my code, but I thought other people can benefit from the translation.

As I said I am days fresh on Linux, and have been playing with Lazarus and FPC under Windows for only a month or so, and I have a lot to learn ;-) . I was able however to get the libraries to work nearly 80% already, and I think in few months can have them ready for deployment. I still will have to figure out how to package them in Linux, and how to deploy them compiled against different visual frameworks, but that is for the future. For now I am focused on getting the high resolution timers to work, so I can move to the next issue in the list ;-) .

With best regards,
Boian Mitov

Op Sun, 10 Aug 2008, schreef Boian Mitov:

In Free Pascal we have spread the functionality over multiple units:
- "baseunix" contains Unix system calls that are portable accross Unixes.
- "unix" contains generally available Unix routines that are portable accross Unixes:
- "linux" contains Linux specific system calls.

If the HR timers are not part of any standard, implementing them in "linux" seems the best approach to me.

Timing on Unix is often done using alarm/setitimer system calls, gettimeofday, etc. I don't know wether these satisfy your needs.

Good to hear that you are making progress!

Daniël

I think it is necessary to explain certain things a bit more:

FPC tries to be independent of the C library as much as possible.

We do provide an interface to the Unix system which is very much like the C library (that is, adheres roughly to certain Posix standards), so you will find most LibC functionality in the BaseUnix, Unix, UnixTypes units. These units are cross platform (for unix like systems), and, where possible, interface to the unix kernel. For some platforms they simply fall back to the C library of the system.

On the other hand, Kylix based itself on GNU Libc entirely. For compatibility reasons, we have created a similar unit as Kylix had. This unit can be used, but only on i386 and on systems that use GNU libc (Linux, in casu), because the record and type definitions it contains are i386 and GNU Libc specific.

You have therefor 2 options:
- Continue with using Libc, and provide patches that keep it up to date.

- Implement the functionality that you need in a Posix like way, and make sure it works accros processor architectures and operating systems.

Naturally, we prefer the latter - as explained by Daniel - but if you want to do the former (which is most likely less work): no problems, we accept patches.

Michael.

Thank you guys for the detailed explanations. I think it will help me a lot to understand the infrastructure. I am doing in on stages. First I will implement it in my code and test it and then will extract it and put it in suitable unit for you. I will study the units to see a good fit for it. What I need is 1 nS units repeating timer. If I am not mistaken this can be achieved only with the hrtimers, but I may be wrong. If I can come up with a better more platform independent Unix approach, it will be even better.

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------

Hi,
this thread has been really interesting. It has filled in my understanding of the tactics behind fpc a bit more.

Cheers,
Mark.

Note the problem was less that Kylix based itself on libc, but more the fact that they stuffed anything they could find into unit libc without any discrimination:

- linuxisms
- glibcisms
- Libraries that are in fact external, but linked into the glibc library (but not part of the actualy glibc distribution). Here I mean stuff like iconv.
- kernel version dependant stuff
- x86 specific issues
- other system libraries like libdl.

You have therefor 2 options: - Continue with using Libc, and provide patches that keep it up to date.

No. Patches to unit libc should be limited to fixes to the Kylix compability of unit libc, and fixes needed to keep Kylix using code running with modern kernels.

If there we are serious in creating own versions of libc functionality as pure headers in the RTL, we could have clinux and cunix for those.

Not for unit libc. New functionality has no place there. In 2.3.1 there is a formal "deprecated" in that unit even, and then you shouldn't add new functionality in it. Create a new package, clinux whatever (if it is linux specific), but _NOT_ unit libc.

Maybe other platforms have other means. You can write an abstract wrapper around them, avoiding the linux gore to spill out.

If I can come up with a better more platform independent Unix approach, it will be even better.

Study the open group site, and test occasionally on the website of FreeBSD if a certail call exist (nearly all unices have some place where you can view manpages on the web).

Also note that there is a difference between "linux" and "linux/x86", and if hrtimers wrap to some hardware feature, this could be a problem.

I am also interested in learning more about the rules of how new functionality should be contributed to the project. Although I intend to use FPC and Lazarus for my developments, I also intend to do my best to contribute to the projects. Maybe some of the information posted in this thread could be copied to the Wiki site for future references. I am sure there are a lot of people willing to help with contributions, but similar to me a not able to find the best possible approach. The information from this thread has been extremely valuable for me!

With best regards,
Boian Mitov

-------------------------------------------------------------------- Mitov Software
http://www.mitov.com
--------------------------------------------------------------------