Is it possible to apply better compression to Netfree HTTP traffic

  • I am throwing out an idea which isn't fully thought through. I am unsure if this feature is at all technically possible, and how much of a benefit it will be.

    The problem is that the http protocol allows various 'transfer-encoding's, some of which are compressed. This can cause enormous speed and bandwidth savings depending on the compression quality and the data being transferred. I have noticed that Netfree removes any {transfer|content}-encoding from the traffic that passes through it and passes it on to the end user uncompressed with transfer-encoding: chunked.

    The obvious problem with dealing with compressed data is that it must be decompressed for inspection/injection so it obviously must be decompressed in transit. The question is can it be recompressed before continuing on it's way?
    I am not an expert in the HTTP protocol but from a very quick skimming of some MDN articles it would seem that any transfer-encoding other than chunked must include a content-length header. This would mean that the entire HTTP response must have been received and have passed filtering before Netfree can pass it on to the client. This obviously would introduce a major latency. This would explain the lack of compression since as such Netfree can begin transferring content to the client before knowing the final content-length.

    My questions therefore are as follows:

    1. Am I correct that there is no way of transferring compressed data without knowing before hand the content-length.
    2. In the real world, how much extra volume is being transferred due to the lack of compression?
    3. Can compression be introduced in cases where the page static and the final content-length can be known before the full HTTP response has been received and filtered.
    4. And for some bonus points 🙂 I think that Netfree doesn't allow QUIC and HTTP/2. So, can anyone say, how much would customers benefit from HTTP/2 and is it at all possible to allow it with keeping it's benefits?

    I would love it if people can weigh in on this.

  • I think if you are interested in discussing this proposal, you should offer it in the Hebrew forum