From time to time I get XML parsing exceptions even though I disabled data validation.

If c-data contains any character whose ordinal value is greater than 127 the XML parser raises an exception.

I realize fpc's parser is not the only one with this problem. So who's infinite wisdom decided that C-Data cannot have non-keyed input!?!

Is there a ANY case where dropping c-data is OK just because a user hasn't entered it? I'm just curious... I can't seem to find any good reason as to why this happened other than to force UTF or some other encoding on everyone.

The XML parser already has options for Validation. I'm hoping to get someone on the team to add a one line if Validation=true then check the values else just parse it already.

http://mantis.freepascal.org/view.php?id=22990

Another thing would be to have an exception level event. There is an event there, but still the exception is raised the event is called - stopping the parsing and throwing a wrench in my streaming application logic.

In a world where XML is just about the primary choice for data transmission - I think we take speed as the ut-most priority and don't use such a pivotal technology as a tool to get what someone else wants.

The bottom-line inference here is that WE ALL must encode ALL DATA before it can be streamed / parsed. That's bad news and EXPENSIVE.

Therefore, any help would be greatly appreciated.

Andrew Brunner hat am 26. September 2012 um 21:21 geschrieben:

[...] If c-data contains any character whose ordinal value is greater than 127 the XML parser raises an exception.

Have you tried setting the right encoding in the xml?

Sergei's note is correct:

---snip---
This is unrelated to validation. XML parser always checks that its input data conforms to the specified encoding.
The XML data should either be in utf8 encoding, or in utf16 with a BOM character at the beginning, or it should be labeled with 'encoding' attribute (additionally xmliconv unit must be added to uses clause if encoding is different from iso8859-1).
---snap---

Another thing would be to have an exception level event. There is an event there, but still the exception is raised the event is called - stopping the parsing and throwing a wrench in my streaming application logic.

Maybe you can explain what you are trying to do?

In a world where XML is just about the primary choice for data transmission - I think we take speed as the ut-most priority and don't use such a pivotal technology as a tool to get what someone else wants.

AFAIK such statements have seldom a positive effect on volunteer projects.

The bottom-line inference here is that WE ALL must encode ALL DATA before it can be streamed / parsed. That's bad news and EXPENSIVE.

Well, whatever you try, you must at least encode all < > and & characters.

Mattias

Mattias Gaertner wrote

Have you tried setting the right encoding in the xml? > http://mantis.freepascal.org/view.php?id=22990

I have, and it did and it did work (thanks Sergei :-)

Maybe you can explain what you are trying to do?

I have a cloud social virtual operating system and each read/write operation is done via XML. So adding content encoding mechanism for comparing each byte is extremely costly from a client/server standpoint. Just imagine 1M+ users and having to encode/decode each xml fragment just to get the parser to parse the data - unwanted latency.

AFAIK such statements have seldom a positive effect on volunteer projects.

My frustration is not with FPC team, because they are drawing code from projects like firefox. I am extremely sensitive towards wasted cpu cycles as efficiency in scale is maximized by reducing things like byte encoding. Some of my stream fragments can be as large as 1.4MB deflated from 8MB. Multiply that number by say 100 concurrent users on that 1 node and you'll begin to see my frustration.

To me, an XML parser is just looking for "" etc. Once it hits a CDATA section it should only look for ]]. Therefore I was surprised to learn that it required the encoding tag (which in itself just increased the average network packet size) that I must transmit from client to my server nodes.

A good XML parser would not care what is between core essentials. Content encoding is for human readability it is not for computers nor should it ever be a concern to something as low level as an XML parser. This is best for the Internet and it's servers - not my opinion.

Well, whatever you try, you must at least encode all < > and & characters.

Right. I guess when it comes to my assertions regarding all this - I am looking for an AHH OK notion as to why my servers have to decode/encode/stamp each XML fragment just so a parser - that is a machine that does not care what a euro looks like - can process it correctly.

I suppose we could talk about interoperability here. And of course it seems like an a-ha moment. But these are XML fragments that were created in my system. And I think for the sake of the developer of an app - should have tweaks for efficiency aside from the content encoding de'joure (UTF8).

I realize my poignancy could be harsh but this is an expensive problem for me. Each server can cost as much as $4,000 US.

To my knowledge, we are not ?

If you are sensible to that, drop XML and use JSON. It will parse much faster, contains far less overhead (smaller packet size). It was invented exactly because XML became too heavyweight.

I did some benchmarking. Using XML (well, SOAP) makes a typical application 6 times slower than a comparative binary transmission mechanism.

To me, an XML parser is just looking for "" etc. Once it hits a CDATA section it should only look for ]]. Therefore I was surprised to learn that it required the encoding tag (which in itself just increased the average network packet size) that I must transmit from client to my server nodes.

You cannot just choose which parts of XML to use, and which not. XML is highly standardized, and so are XML parsers. That is why people have standards.

Michael.

I did some benchmarking. Using XML (well, SOAP) makes a typical application 6 times slower than a comparative binary transmission mechanism.

Just curious, did you even do that test with JSON as well? Probably still slower than binary, but how much faster than XML?

tiOPF has normal XML and "compact XML" for remote persistence layer. The "compact XML" is the default, more cryptic names and values, but reduces network traffic a lot. I still want to add JSON support too.

Cheers,
G.