Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2009-08

Re: XML canonical

7 messages · 3 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Sergei Gorelkin
to Dariusz Mazur
Dariusz Mazur wrote 9 lines
Hi this is my first approach to implement canonical transform from <ds:CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"/> its used in XADES sign. can somebody review it, if its interesting

First of all, it ignores the fact that DOM is based on WideStrings (UTF-16 encoding). Plain assignments of DOM properties to string (or UTF8String) will cause conversion to system locale encoding. It will happen to work if system locale is actually utf-8, but won't work correctly e.g. in Windows.
Next, a number of crude errors can be seen just by reading the source. For example, a processing instruction is delimited by '<?' and '?>', not by '<!' and '>'. It lacks escaping '<' by '&lt;' and '&' by '&amp' in both text and attributes. Attribute value normalization must not only replace #13, #9 and #10 by space, but also trim leading and trailing whitespace and replace all consequtive whitespaces by single space (and it is already done by DOM for non-CDATA attributes).

I'd suggest you to find a test suite (some examples are contained directly in the text of w3.org specification, others may be ripped from opensource projects that support canonicalization, like libxml2) and verify your unit against it.

Sergei

Reply by email

Dariusz Mazur
to Sergei Gorelkin

Sergei Gorelkin pisze:

Show quoted text 15 lines
Dariusz Mazur wrote: > Hi > this is my first approach to implement canonical transform > from <ds:CanonicalizationMethod > Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"/> > > its used in XADES sign. > > can somebody review it, if its interesting > First of all, it ignores the fact that DOM is based on WideStrings (UTF-16 encoding). Plain assignments of DOM properties to string (or UTF8String) will cause conversion to system locale encoding. It will happen to work if system locale is actually utf-8, but won't work correctly e.g. in Windows.

Thx. I forget about it.

Show quoted text 7 lines
Next, a number of crude errors can be seen just by reading the source. For example, a processing instruction is delimited by '<?' and '?>', not by '<!' and '>'. It lacks escaping '<' by '&lt;' and '&' by '&amp' in both text and attributes. Attribute value normalization must not only replace #13, #9 and #10 by space, but also trim leading and trailing whitespace and replace all consequtive whitespaces by single space (and it is already done by DOM for non-CDATA attributes).

I will do that. Current implementation of processing is very slow also.

Show quoted text 5 lines
I'd suggest you to find a test suite (some examples are contained directly in the text of w3.org specification, others may be ripped from opensource projects that support canonicalization, like libxml2) and verify your unit against it.

I draw from libxml2. First i resolved problem with namespaces. That is not so clear in specification. Even simple xml canonization don't work. Now I can work with more complicate file.

New version should be send here?

Darek

Reply by email

Sergei Gorelkin
to Dariusz Mazur
Dariusz Mazur wrote 10 lines
> I'd suggest you to find a test suite (some examples are contained > directly in the text of w3.org specification, others may be ripped > from opensource projects that support canonicalization, like libxml2) > and verify your unit against it. I draw from libxml2. First i resolved problem with namespaces. That is not so clear in specification. Even simple xml canonization don't work. Now I can work with more complicate file. New version should be send here?

I was intending to implement canonicalization in the existing writer (xmlwrite.pp), but namespace support is required before it can be done. (namespace support is present in dom and xmlread, but not yet in xmlwrite). Actually, the writer is already doing most of the things, including proper encoding and escaping. The attributes are already sorted by name. What has to be changed:
1) sorting attributes with respect to namespace
2) write end-tags in full form
3) treat CDATA sections as text
4) don't omit defaulted attributes
5) omit xml declaration and DTD.
6) ...maybe missed something else.
The (1) is most difficult, (2) to (5) are pretty trivial.

So for me it seems a better idea to improve the existing writer rather than making a new one. I cannot speak on behalf of the whole team, however. Other opinions are welcome.

Sergei

Reply by email

Dariusz Mazur
to Sergei Gorelkin

Sergei Gorelkin pisze:

Show quoted text 25 lines
Dariusz Mazur wrote: >> I'd suggest you to find a test suite (some examples are contained >> directly in the text of w3.org specification, others may be ripped >> from opensource projects that support canonicalization, like >> libxml2) and verify your unit against it. > I draw from libxml2. First i resolved problem with namespaces. That > is not so clear in specification. Even simple xml canonization don't > work. Now I can work with more complicate file. > > New version should be send here? > I was intending to implement canonicalization in the existing writer (xmlwrite.pp), but namespace support is required before it can be done. (namespace support is present in dom and xmlread, but not yet in xmlwrite). Actually, the writer is already doing most of the things, including proper encoding and escaping. The attributes are already sorted by name. What has to be changed: 1) sorting attributes with respect to namespace 2) write end-tags in full form 3) treat CDATA sections as text 4) don't omit defaulted attributes 5) omit xml declaration and DTD. 6) ...maybe missed something else.

6.sorting namespaces
7. added default namespace (exists above main node - when we write only part of tree)

Show quoted text 6 lines
The (1) is most difficult, (2) to (5) are pretty trivial. So for me it seems a better idea to improve the existing writer rather than making a new one. I cannot speak on behalf of the whole team, however. Other opinions are welcome.

Now I see what want. Till now I work on old implementation of xmlwrite (C-style) and not notice that was change.
I can try reimplement it based on exists. it seem easy.

Darek

Reply by email

Dariusz Mazur
to Sergei Gorelkin: I was intending to implement canonicalization in the existing writer (xmlwrite.pp), but…

Sergei Gorelkin pisze:

Show quoted text 30 lines
Dariusz Mazur wrote: >> I'd suggest you to find a test suite (some examples are contained >> directly in the text of w3.org specification, others may be ripped >> from opensource projects that support canonicalization, like >> libxml2) and verify your unit against it. > I draw from libxml2. First i resolved problem with namespaces. That > is not so clear in specification. Even simple xml canonization don't > work. Now I can work with more complicate file. > > New version should be send here? > I was intending to implement canonicalization in the existing writer (xmlwrite.pp), but namespace support is required before it can be done. (namespace support is present in dom and xmlread, but not yet in xmlwrite). Actually, the writer is already doing most of the things, including proper encoding and escaping. The attributes are already sorted by name. What has to be changed: 1) sorting attributes with respect to namespace 2) write end-tags in full form 3) treat CDATA sections as text 4) don't omit defaulted attributes 5) omit xml declaration and DTD. 6) ...maybe missed something else. The (1) is most difficult, (2) to (5) are pretty trivial. So for me it seems a better idea to improve the existing writer rather than making a new one. I cannot speak on behalf of the whole team, however. Other opinions are welcome.

I've prepare new version of canonization. Its based on xmlwrite. Point 3,4 are not resolved (I don't it really need) but its trivial.

test against xmllib is not so easy, because we should follow the same indentation on xmlwrite

It works only with simple xml, but its good enough to make proper XADES signature
I can submit those unit also, if somebody help me and review it.

Darek

Reply by email