I am not sure how this is occurring, however, my socket daemon on Linux - is launched by hand ./program is able to send to the socket:

Socket.Write(#218+#196+#191);

And the terminal (any) will display the single highbit characters.

If systemctl start myprogram... that same call sends '???'. Same binary, not recompiled or anything. Then I do systemctl stop myprogram, and ./program connect to it, and I get the single highbit characters again.

I have googled everything I could think of for FPC and SYSTEMCTL, to see if either have documented this - without success. Since you guys know what the compiler is producing, and all these tricks you guys do for UTF8, CP437, etc... I decided to give up and ask you guys - any idea what is going on?

* The other challenge is, this just started a week ago. the myprogram.service file has not been modified, nothing for the OS had been modified, I just noticed one day - the code I use for UTF8 auto detection was showing ???? ... and then ran my 8bit test, and noticed ever character 0x80+ displays as '?'.

Thanks,
Ozz

Op 29/01/2020 om 14:23 schreef Ozz Nixon via fpc-devel:

LANG or LOCALE environment variables ?

Would/Could, LANG/LOCALE affect socket output?

* I personally do not touch environment variables - so I am not sure what to ever try. The "client" (Telnet) I have tried Terminal.App, iTerm2, Putty.exe, Telnet.exe, xTerm-256, fTelnet, etc. all clients on all 3 OSes render '?' from the daemon if systemctl starts it, again, if I just start the fpc binary from a shell'd session, ./program - it works fine on all 3 OSes that previously show '?'. I have tcpdump'd the socket, it sends '?' when ran under systemctl, however, sends the 8bit character when ran manually on the command line.

Thanks,
Ozz

On Wed, Jan 29, 2020 at 7:39 AM Marco van de Voort <

Ozz Nixon wrote

Would/Could, LANG/LOCALE affect socket output?

wouldn't they affect what the server decides to transmit based on the selected "character set"?

Op 29/01/2020 om 14:54 schreef Ozz Nixon via fpc-devel:

Afaik the unicode manager (cwstring)initializes using those environmentvariables

Note the best way to check is to use a binary client that dumps data as hex, or wireshark.

1. My code does not directly interact with any environment variables.
2. I am using version 2.6.4 to compile the daemon.

my .service file for systemctl looks like:

[Unit]
Description=The Modern Pascal Code Running Socket Server After=network.target

[Service]
Type=simple
PIDFile=/run/coderunner2.pid
EnvironmentFile=-/etc/coderunner2.conf
ExecStartPre=/usr/bin/rm -f /run/coderunner2.pid
ExecStart=/usr/local/bin/coderunner2
ExecReload=/bin/kill -s HUP $MAINPID
KillSignal=SIGQUIT
TimeoutStopSec=5
KillMode=process
PrivateTmp=true
Restart=on-failure
RestartSec=42s
StandardOutput=null

[Install]
WantedBy=multi-user.target

3. I do not see anything in that file that would override socket output.
4. And again, if I to /usr/local/bin/coderunner2 from the shell, the client see's the 8bit code.
5. per wireshark, tcpdump (if the same) - it shows '?' is being sent when the daemon is managed by systemctl, and the expected 8bit is sent when ran at the shell prompt.

6. So, the thought of LANG/LOCALE affecting it, from the shell: LEGACYX]# echo $LANG
en_US
LEGACYX]# echo $LOCALE

6b. Not sure how to do that except have my daemon display those two at connect time...
7. CentOS 7, dropped using init.d for systemctl - on my CentOS 6 machine, it works perfectly...

8... does socket.pp get affected at RUN-TIME, by environment variables? (did not notice my first pass through)...

Op 29/01/2020 om 16:07 schreef Ozz Nixon via fpc-devel:

1. My code does not directly interact with any environment variables.

Unit cwstring and clocale might do on startup. This might influence e.g. widestring/unicodestring<-> ansistring conversions and back.

2. I am using version 2.6.4 to compile the daemon.

That makes encoding debugging quite more difficult, as 2.6.4 behaves differently from 3.0.x in that regard.

That proves the problem is unrelated to sockets. Is is related to

string/encoding handling.

Now that makes a lot more sense than the socket doing goofy magic. The string layer in FPC doing goofy magic!

I am trying your .sh script idea now...

On Wed, Jan 29, 2020 at 9:42 AM Nikolai Zhubr via fpc-devel <

per units, coderunner uses:

uses
{$IFDEF UNIX}
cthreads,
baseunix,
unix,
{$ENDIF}
{ huge list of units I developed }
variants,
keyboard,
PdfDocs,
PdfTypes,
PdfFonts,
SQLDb,
odbcconn,
sqlite3conn, <-- soon to remove, it's unstable in our testing jansql, <-- soon to remove could not get it to work spellcheck;

Those are the unit I run that I did not code.

Back to testing .sh idea.

PS. Worsecase, I can scrap 2.6.4 for 3.0.x - just been waiting for a stable LLVM build for all 3 OSes I support.

On Wed, Jan 29, 2020 at 9:45 AM Marco van de Voort <

Hi!

29.01.2020 18:07, Ozz Nixon via fpc-devel:

1. My code does not directly interact with any environment variables.

Ok.

ExecStart=/usr/local/bin/coderunner2

I'd suggest, as a quick and horrible temporary hack/test, replace this direct binary call with a shell wrapper, lets name it e.g. coderunner2.sh, containing something like:

#!/bin/sh
echo "LANG=$LANG" >>/tmp/test.txt
echo "LOCALE=$LOCALE" >>/tmp/test.txt
export LC_ALL=C
/usr/local/bin/coderunner2

(Put it into /usr/local/bin/ as otherwise systemd might refuse it; I suppose the script is simple enough to be self-explanatory)

3. I do not see anything in that file that would override socket output.

Sure.

5. per wireshark, tcpdump (if the same) - it shows '?' is being sent when the daemon is managed by systemctl, and the expected 8bit is sent when ran at the shell prompt.

That proves the problem is unrelated to sockets. Is is related to string/encoding handling.

HTH...

Regards,
Nikolai

using the .sh concept:
# systemctl status coderunner
* coderunner.service - The Modern Pascal Code Running Socket Server

   Loaded: loaded (/usr/lib/systemd/system/coderunner.service; enabled;

vendor preset: disabled)
Active: active (running) since Wed 2020-01-29 10:55:30 EST; 8s ago Process: 18065 ExecStartPre=/usr/bin/rm -f /run/coderunner2.pid (code=exited, status=0/SUCCESS)
Main PID: 18066 (coderunner2.sh)
CGroup: /system.slice/coderunner.service
|-18066 /bin/sh /usr/local/bin/coderunner2.sh
`-18069 /usr/local/bin/coderunner2

Jan 29 10:55:30 www.modernpascal.com systemd[1]: Starting The Modern Pascal Code Running Socket Server...
Jan 29 10:55:30 www.modernpascal.com systemd[1]: Started The Modern Pascal Code Running Socket Server.

So it starts - telnet 127.0.0.1 - and I get the ???? stuff still. cat /etc/test.txt
<null>
ls -alrt /etc/test.txt
-rw-r--r-- 1 root root 0 Jan 29 10:54 /tmp/test.txt

chmod 777 just incase...
do not make a different, I even tried, bash, and even added starting... #!/bin/bash
echo "Starting">>/tmp/test.txt
echo "LANG=$LANG" >>/tmp/test.txt
echo "LOCALE=$LOCALE" >>/tmp/test.txt
export LC_ALL=C
/usr/local/bin/coderunner2

Nothing writes to the file when systemctl starts it. However, if I ./coderunner2.sh I get:
Starting
LANG=en_US
LOCALE=

So, I guess try FPC 3.0.x???

On Wed, Jan 29, 2020 at 9:42 AM Nikolai Zhubr via fpc-devel <

Ozz Nixon via fpc-devel schrieb am Mi.,
29. Jan. 2020, 16:51:

PS. Worsecase, I can scrap 2.6.4 for 3.0.x - just been waiting for a stable LLVM build for all 3 OSes I support.

Then you're in for long wait as the LLVM backend is fully supported only on trunk (at least for macOS and Linux on x86_64). We're currently preparing the 3.2 release and that means you'll have to wait around two years at least.

Also 3.0.x and 3.2 provide better code generation compared to 2.6.4 as well. So there is probably no need to require the LLVM backend.

Regards,
Sven

I have waited this long now :-)

I only run 2.6.4 as something I developed didn't work correctly when compiled with 3.0.0, 3.0.2...

Ozz

On Wed, Jan 29, 2020 at 10:11 AM Sven Barth via fpc-devel <

Hi!

29.01.2020 19:04, Ozz Nixon via fpc-devel:

cat /etc/test.txt <null> ls -alrt /etc/test.txt -rw-r--r-- 1 root root 0 Jan 29 10:54 /tmp/test.txt

Make sure there is no naming confusion (such as e.g. /tmp/test.txt and /etc/test.txt unintentionally intermixed)

Another point is that a binary running from systemd context might have somewhat different access rights as compared to running from an interactive shell, so that might lead to some files being unreadable and that might potentially also lead to changed behaviour. Actually systemd allows you to set a user and group you want for the service, like e.g.:

[Service]
User=root
Group=root

A don't remember what it sets by default.
(And certaily, root/root is generally not a safe practice, but just for a test...)

Regards,
Nikolai

So far in testing, looks like setting $TERM to xterm-256color resolved my issue. (not 100% sure if it was specifically that setting, but, after 20+ hours for a couple days in a row, it started working with that being defined in the .sh idea (thanks!)).

path typo was just my exhaustion... it is set to /tmp/test.txt ...

per user pid, I am old school (risk taker), I shell in as root, and code/run as root. Have since 1997 when I started running Linux as my main server line...so not a permissions issue. ;-) off to bed...

Thanks everyone for your input and ideas where/what to try! Monday should be a good day :-)

On Wed, Jan 29, 2020 at 11:28 AM Nikolai Zhubr via fpc-devel <