Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2002-11

Re: unicode

9 messages · 5 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Florian Klaempfl
to Rimgaudas Laucius
you wrote
Hi. Is it posible to make fpc to support unicode source (principle)?

Of course it is, and it shouldn't be that hard.

But would be the use of it? The only thing I could imagine are comments or constant strings.
But I guess people who need this use MBCS instead of unicode, no?

Furthermore, the 1.1 compiler contains codepage support for the input source, so you can specify how constant characters are converted to unicode when doing a wide string conversion for a constant string. Of course, this is only useful for languages with <=256 characters, it doesn't help users with very large character sets. Furthermore, the current implementation doesn't support dynamic loading of code pages and only the common (for me common :) )
code pages (850,437 and iso 8xxx-1) are compiled in.

Reply by email

Rimgaudas Laucius
to Florian Klaempfl
Show quoted text 20 lines
At 14:06 07.11.2002, you wrote: >Hi. > >Is it posible to make fpc to support unicode source (principle)? Of course it is, and it shouldn't be that hard. But would be the use of it? The only thing I could imagine are comments or constant strings. But I guess people who need this use MBCS instead of unicode, no? Furthermore, the 1.1 compiler contains codepage support for the input source, so you can specify how constant characters are converted to unicode when doing a wide string conversion for a constant string. Of course, this is only useful for languages with <=256 characters, it doesn't help users with very large character sets. Furthermore, the current implementation doesn't support dynamic loading of code pages and only the common (for me common :) ) code pages (850,437 and iso 8xxx-1) are compiled in.

Yes it is true. But i am interesting about support of other than basic latin simbols in idetificators (a.e. variables) also and about problems related with it (a.e. one of problems windows dlls do not support unicode identificators of exported routines).

Reply by email

Peter Vreman
to Rimgaudas Laucius
you wrote 24 lines
> At 14:06 07.11.2002, you wrote: > >Hi. > > > >Is it posible to make fpc to support unicode source (principle)? > > Of course it is, and it shouldn't be that hard. > > But would be the use of it? The only thing I could > imagine are comments or constant strings. > But I guess people who need this use MBCS instead of unicode, no? > > Furthermore, the 1.1 compiler contains codepage support for the input > source, so you can specify how constant characters are converted to > unicode when doing a wide string conversion for a constant string. Of > course, this is only useful for languages with <=256 characters, > it doesn't help users with very large character sets. Furthermore, the > current implementation doesn't support dynamic loading of code pages > and only the common (for me common :) ) > code pages (850,437 and iso 8xxx-1) are compiled in. > Yes it is true. But i am interesting about support of other than basic latin simbols in idetificators (a.e. variables) also and about problems related with it (a.e. one of problems windows dlls do not support unicode identificators of exported routines).

The problem is the linker and assembler that only support normal ascii symbols.

Peter

Reply by email

Florian Klaempfl
to Peter Vreman
you wrote 28 lines
At 16:25 11-11-2002, you wrote: > > At 14:06 07.11.2002, you wrote: > > >Hi. > > > > > >Is it posible to make fpc to support unicode source (principle)? > > > > Of course it is, and it shouldn't be that hard. > > > > But would be the use of it? The only thing I could > > imagine are comments or constant strings. > > But I guess people who need this use MBCS instead of unicode, no? > > > > Furthermore, the 1.1 compiler contains codepage support for the input > > source, so you can specify how constant characters are converted to > > unicode when doing a wide string conversion for a constant string. Of > > course, this is only useful for languages with <=256 characters, > > it doesn't help users with very large character sets. Furthermore, the > > current implementation doesn't support dynamic loading of code pages > > and only the common (for me common :) ) > > code pages (850,437 and iso 8xxx-1) are compiled in. > > >Yes it is true. But i am interesting about support of other than basic latin >simbols in idetificators (a.e. variables) also and about problems related >with it (a.e. one of problems windows dlls do not support unicode >identificators of exported routines). The problem is the linker and assembler that only support normal ascii symbols.

Are there any windows dll which use really chars > 127?

Reply by email

Rimgaudas Laucius
to Florian Klaempfl
Show quoted text 6 lines
At 00:12 14.11.2002, you wrote: >At 16:25 11-11-2002, you wrote: >> > At 14:06 07.11.2002, you wrote: >> > >Hi. >> > > >>Yes it is true. But i am interesting about support of other than basic

latin

>>simbols in idetificators (a.e. variables) also and about problems

related

Show quoted text 5 lines
>>with it (a.e. one of problems windows dlls do not support unicode >>identificators of exported routines). > >The problem is the linker and assembler that only support normal ascii >symbols.

I do not know what you mean saying "normal ascii simbols". Ascii is standart for 7 bit characters but many of people use it talking about 8 bit standarts.
The linker and assembler supports 8 bit characters (i tayed it under windows).

Are there any windows dll which use really chars > 127?

I do not know. But it is posible to use them. I tryed. It is useless if dll is written for public use. But it is usable for closed projects.

Why i am speeking about internationalization of fpc? The most important reason is that we use Pascal translator for teaching algorithms in the schools. Programs written in the native language are more comprehensible and readable for pupils.

Reply by email

Olle Raab
to Rimgaudas Laucius

02-11-15 08.39, skrev Rimgaudas Laucius följande:

Show quoted text 21 lines
> At 00:12 14.11.2002, you wrote: >> At 16:25 11-11-2002, you wrote: >>>> At 14:06 07.11.2002, you wrote: >>>>> Hi. >>>>> >>> Yes it is true. But i am interesting about support of other than basic latin >>> simbols in idetificators (a.e. variables) also and about problems related >>> with it (a.e. one of problems windows dlls do not support unicode >>> identificators of exported routines). >> >> The problem is the linker and assembler that only support normal ascii >> symbols. I do not know what you mean saying "normal ascii simbols". Ascii is standart for 7 bit characters but many of people use it talking about 8 bit standarts. The linker and assembler supports 8 bit characters (i tayed it under windows).

As I see it, they do only support a subset of 7-bit ascii, namely those constituting valid identifiers. At least in the general case, but of course it differs between different assemblers and linkers. E g PPCAsm for MacOS supports only symbols which is consided valid identifier (as in C or Pascal). That is they must be in the range _,A..Z,a..z,0..9 and not start with a digit.

To make FreePascal true cross-platform and to avoid problems, identifiers should be valid pascal of C identifiers, IMO.

But, as I can see, the problem could be solved if unicode names is encoded to names with chars in the above range. When symbols are parsed by the compiler, they could be converted, and stored as such internally. Error messages bounced back to the user is translated back to Unicode.

I dont know which encoding scheme could be used. Can UTF-8 or UTF-7 be used, do they translate a valid unicode identifier to a valid ascii identifier. ?

Show quoted text 6 lines
Why i am speeking about internationalization of fpc? The most important reason is that we use Pascal translator for teaching algorithms in the schools. Programs written in the native language are more comprehensible and readable for pupils.
//Olle Raab

Reply by email

Rimgaudas Laucius
to Olle Raab

----- Original Message -----

Earlier message 51 lines
From: "Olle Raab" To: Sent: Wednesday, November 20, 2002 12:30 AM Subject: Re: [fpc-devel]unicode > >>>> Yes it is true. But i am interesting about support of other than basic > > latin > >>>> simbols in idetificators (a.e. variables) also and about problems > > related > >>>> with it (a.e. one of problems windows dlls do not support unicode > >>>> identificators of exported routines). > >>> > >>> The problem is the linker and assembler that only support normal ascii > >>> symbols. > > > > I do not know what you mean saying "normal ascii simbols". Ascii is standart > > for 7 bit characters but many of people use it talking about 8 bit > > standarts. > > The linker and assembler supports 8 bit characters (i tayed it under > > windows). > > > > As I see it, they do only support a subset of 7-bit ascii, namely those > constituting valid identifiers. At least in the general case, but of course > it differs between different assemblers and linkers. E g PPCAsm for MacOS > supports only symbols which is consided valid identifier (as in C or > Pascal). That is they must be in the range _,A..Z,a..z,0..9 and not start > with a digit. > > To make FreePascal true cross-platform and to avoid problems, identifiers > should be valid pascal of C identifiers, IMO. > > But, as I can see, the problem could be solved if unicode names is encoded > to names with chars in the above range. When symbols are parsed by the > compiler, they could be converted, and stored as such internally. Error > messages bounced back to the user is translated back to Unicode. > > I dont know which encoding scheme could be used. Can UTF-8 or UTF-7 be used, > do they translate a valid unicode identifier to a valid ascii identifier. ? I think it is good idea to encode unicote to utf. I think it is the best to encode to UTF-8. It is not difficult to adapt fpc to it, but it needs some changes for tools used with fpc. As i tested asw and ldw supports 8 bit symbols, but libgdb does not. Is it difficult to adapt it too?

Reply by email