Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2016-11

DOM and UTF-8 widestring conversion

2 messages · 1 person ·

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Seth Grover

I'm hoping somebody could point me to the specific change (mantis bug #, revision #, user changes wiki page link, whatever) between 2.6.4 and 3.0.0 that fixes this problem I was running into.

Given this sample program:

=======================================
program Project1;
{$mode objfpc}{$H+}

uses
cthreads,
cwstring,
Classes,
SysUtils,
XMLWrite,

  dom;
var
  s : ansistring;
  s2 : ansistring;
  dstDoc : TXMLDocument;
  rootNode : TDomNode;
  childNode : TDomNode;
begin
  s := #$E3#$82#$A6#$E3#$82#$A3#$E3#$83#$AB#$E3#$82#$B9;
  writeln('[', s, ']');
  dstDoc := TXMLDocument.Create;
  try
    rootNode := dstDoc.CreateElement('root');
    childNode := dstDoc.CreateElement('contents');
    TDOMElement(childNode).SetAttribute('value', widestring(s));
    s2 := ansistring(TDOMElement(childNode).GetAttribute('value'));
    rootNode.AppendChild(childNode);
    dstDoc.AppendChild(rootNode);
  finally
    WriteXMLFile(dstDoc, 'export.xml');
    FreeAndNil(dstDoc);
  end;
  writeln('[', s2, ']');
end.
=======================================

With FPC 2.6.4 I end up with "????????????" for the output (both to the console and written to the xml file) as the output. With FPC 3.0 I get the correct representation of the japanese characters.

So my question is:

1. What changed between FPC 2.6.4 and 3.0 to cause the change in behavior?

and

2. (apart from upgrading to FPC 3.0) Is there any workaround in 2.6.4 to get the correct behavior?

I'd prefer just to be using 3.0 everywhere but on some legacy versions of our product upgrading the build environment is not an option right now.

Thanks much,

-SG

Seth Grover Be kind to all of your neighbors Because they're just like you. And you're nothing special Unless they are too.
Seth Grover

After playing around some more with my example, I was able to get it to work in 2.6.4 (without using the cwstring unit) by using the lazutf8 unit's UTF8ToUTF16
routine on the SetAttribute and UTF16ToUTF8 on the GetAttribute:

    TDOMElement(childNode).SetAttribute('value', UTF8ToUTF16(s));
    s2 := UTF16ToUTF8(TDOMElement(childNode).GetAttribute('value'));

So for now I have a workaround until we're able to upgrade to the 3.0 compiler and use cwstring and the correct implicit conversions. I'd still welcome any input to help me understand the difference between the two versions though.

Thanks,

-SG

Seth Grover Be kind to all of your neighbors Because they're just like you. And you're nothing special Unless they are too.