Gzip compression not working

[HaliFeri]

Dear Forum Members,

I have tried to enable the gzip compression of the XML communication but i could not managed to do it.
I have monitored the whole communication with Fiddler and even tried out a lot of modification to make it work.
In the request the "Accept-Encoding: gzip" is present, I tried with the default "Accept-Encoding: gzip, delfate" either, but the server response (raw data) was plain XML, not gzipped.

I have tried to send a gzipped request either: "Content-Encoding: gzip", and of course i have gzipped the request properly. The server was understood it well, but it sent the response back without any compression, in a plain XML.

I am using the latest 1.18.1.1 xroute server.
Please give me an advice how to enable the compression.

Thank you,
Ferenc

[Bernd Welter]

Hello Ferenc,

I moved this topic to the "Generic Questions" space because it isn't a routing question.
Furthermore I requested the developers to check this issue and provide feedback to the thread or to me.
As soon as I have new input I'll announce it here

Kind regards
Bernd

[lua]

Hello Ferenc,

the response behavior of the server is controlled in the server.xml file in the conf folder.
Here you can list the compressableMimeTypes. By default, text/xml is one of them, but make sure you find it in your file. Should you only have tested with very small response objects, there is also a minimal size defined where compression would not save much but only cost performance. SOAP requests should usually surpass the default threshold, however as their boilerplate parts are rather large.

Also, you can check if the raw request runner does it properly. On our reference system, it does just fine, the response header contains the proper Content-Encoding. The payload is never presented as binary stream since the browser will automatically decode response payload. Browsers will also send the Accept-Encoding automatically and override your own wishes, by the way.

Other than that we would need some more information about the clients you use, if you changed the server configuration, if there is some middleware interfering etc.

Hope this helps a bit,
best regards,
Andreas

[HaliFeri]

Dear Andreas,

thank you for the configuration information, it was very useful and good to know how to manage it.
The good news is, that today I have started the xserver and fiddler and i got gzipped response, so it must was a transient error somewhere (i bet it was in Fiddler, not in xroute of course ).

The main reason why I have checked the communication was the great size of the packets sent by the server.
An avarage response size is currently about 300-400 kb for a complex route. And it is already compressed. (The uncompressed response is about 4.5 MB)

What do you think, is it worth to try out the fast infoset mechanism, or it would not make the packets as smaller and the communication as faster, as much extra work and component it needs?
We are using .Net and we have a 10 Gigabit connection to the PTV server.

Best Regards,
Ferenc

[Bernd Welter]

Hello Ferenc,

to reduce the size of the xml documents you should make sure that the following conditions are kept safe:

• do not activate ResultListOptions that are not required by the application. Especially the Segments, the SegmentAttributes, the Nodes... and: the Polygon! These arrays may produce large numbers of elements and therefore blow up the size in an unwanted manner.
• instead of PLAIN use WKB style for geometries: especially the Polygon object's size will shrink in a significant way. To do so you have to add a CallerContextProperty with the key "ResponseGeometry" and the value "WKB".
• assign the detail level of the Manoeuvres list to the lowest option that satisfies your requirements, e.g. VERY_IMPORTANT instead of ALL.

I attached some samples based on a route from Karlsruhe/Germany to Budapest/Hungary. Here are some key values (uncompressed in KB):

• 50 kb : BinaryPathDescription
• 270 kb : ManoeuvreGroup
• 413 kb : Nodes
• 462 kb : Polygon (plain) / 128kb (WKB)
• 1.405 kb : RouteManoeuvres
• 1.872 kb : Segments incl. SegmentAttributes

Regards from Germany,
Bernd

[lua]

Hello Ferenc,

glad that it works now for you

Regarding FastInfoSet, when compared with gzip, it achieves less compression on its own but is much faster to encode an ddecode than gzip alone, faster even than plain XML. When combined with gzip, it will further improve compression a bit but will induce a performance cost, I guess because the binary format makes compression harder.

There are some throughput comparisons in our benchmarks, which are done in a LAN. Here are the results from xRoute 1.18.1 for Windows (Linux looks nearly the same, by the way):

These numbers will change when crossing the local network, favoring zipped and fastinfoseted transportation.

With .NET there is an additional problem, however: There is (to our knowledge) no free FastInfoSet library to use. There are commercial variants which we had tested a bit and they seemed to work fine. So, the potential gains are probably not worth paying extra, but if you are looking to squeeze every last bit of efficiency out of your setup, go ahead.

Regards,
Andreas

[lua]

Hello Bernd,

thank you for your in-depth analysis of the overhead of some of the routing options. These are very interesting, we should add those measurements to our setup some day.

One small correction though: WKB, WKT or PLAIN do not matter much when compressed. They will all be gzipped to within few percent difference. So use PLAIN if you want to, as long as you use compression you will be fine.

There is also one addition I'd like to make:
A small but nice feature of xRoute 1.18.1 has a significant impact on response size. The xserver.properties option responseGeometrySignificantFigures will reduce size by removing the last digits from coordinates - you know, the ones which come from floating point imprecisions and ensured that you never got your original coordinate back even if it had been in a former response. This can no longer happen, and numbers become smaller as well. You might even want to reduce the default from 11 to 10 or 9 in order to save even more: 11 significant figures suffice for a resolution of about .4mm when scaled to Earth's circumference, so 10 or 9 digits would also be good enough. The rounding from 17 (old behavior) to 11 usually reduces size of geometry-heavy responses by 10-15%, sometimes even when compressed. This is not so wonderous when you consider that the rounding errors are more or less noise and prevent the pattern matching of the zip algorithm from working optimally, costing you compression efficiency.

Happy tuning,
regards,
Andreas