`
13146489
  • 浏览: 251368 次
  • 性别: Icon_minigender_1
  • 来自: 成都
社区版块
存档分类
最新评论

JSON Compression algorithms: HPack VS CJSON

    博客分类:
  • J2EE
阅读更多
总结:HPack 优于 CJSON
json1 json2 json3 json4 json5
Original JSON size (bytes) 52966 104370 233012 493589 1014099
Minimized 33322 80657 180319 382396 776135
Compress CJSON 24899 48605 108983 231760 471230
Compress HPack 5727 10781 23162 49099 99575
Gzipped 2929 5374 11224 23167 43550
Gzipped and Minimized 2775 5035 10411 21319 42083
Gzipped and compressed with CJSON 2568 4605 9397 19055 37597
Gzipped and compressed with HPack 1982 3493 6981 13998 27358
原文地址:http://web-resource-optimization.blogspot.com/2011/06/json-compression-algorithms.html
About
JSON (Java Script Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It can be used as a data interchange format, just like XML. When comparing JSON to XML, it has several advantages over the last one. JSON is really simple, it has a self-documenting format, it is much shorter because there is no data configuration overhead. That is why JSON is considered a fat-free alternative to XML.
However, the purpose of this post is not to discuss the pros and cons of JSON over XML. Though it is one of the most used data interchanged format, there is still room for improvement. For instance, JSON uses excessively quotes and key names are very often repeated. This problem can be solved by JSON compression algorithms. There are more than one available. Here you'll find an analysis of two JSON compressors algorithms and a conclusion whether JSON compression is useful and when it should be used.

Compressing JSON with CJSON algorithm
CSJON compress the JSON with automatic type extraction. It tackles the most pressing problem: the need to constantly repeat key names over and over. Using this compression algorithm, the following JSON:
?
1
2
3
4
5
6
7
8
9
10
11
12
[
  { // This is a point
    "x": 100,
    "y": 100
  }, { // This is a rectangle
    "x": 100,
    "y": 100,
    "width": 200,
    "height": 150
  },
  {}, // an empty object
]
Can be compressed as:
?
1
2
3
4
5
6
7
8
9
10
{
  "templates": [
    [0, "x", "y"], [1, "width", "height"]
  ],
  "values": [
    { "values": [ 1,  100, 100 ] },
    { "values": [2, 100, 100, 200, 150 ] },
    {}
  ]
}
The more detailed description of the compression algorithm, along with the source code can be found here:
Compressing JSON with HPack algorithm
JSON.hpack is a lossless, cross language, performances focused, data set compressor. It is able to reduce up to 70% number of characters used to represent a generic homogeneous collection. This algorithms provides several level of compression (from 0 to 4). The level 0 compression performs the most basic compression by removing keys (property names) from the structure creating a header on index 0 with each property name. Next levels make it possible to reduce even more the size of the JSON by assuming that there are duplicated entries.
For the following JSON:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
[{
  name : "Andrea",
  age : 31,
  gender : "Male",
  skilled : true
}, {
  name : "Eva",
  age : 27,
  gender : "Female",
  skilled : true
}, {
  name : "Daniele",
  age : 26,
  gender : "Male",
  skilled : false
}]
the hpack algorithm produces a compressed version which looks like this:
?
1
[["name","age","gender","skilled"],["Andrea",31,"Male",true],["Eva",27,"Female",true],["Daniele",26,"Male",false]]
More details about hpack algorithm can be found at project home page.
Analysis
The purpose of this analysis is to compare each of the described JSON compressor algorithms. For this purpose we will use 5 files with JSON content having different dimensions, varying from 50K to 1MB. Each JSON file will be served to a browser using a servlet container (tomcat) with the following transformations:
Unmodified JSON - no change on the server side
Minimized JSON - remove whitespaces and new lines (most basic js optimization)
Compressed JSON using CJSON algorithm
Compressed JSON using HPack algorithm
Gzipped JSON - no change on the server side
Gzipped and minimized JSON
Gzipped and compressed using CJSON algorithm
Gzipped and compressed using HPack algorithm
RESULTS

This table contains the results of the benchmark. Each row of the table contains one of the earlier mentioned transformation. The table has 5 columns, one for each JSON file we process.
\
json1 json2 json3 json4 json5
Original JSON size (bytes) 52966 104370 233012 493589 1014099
Minimized 33322 80657 180319 382396 776135
Compress CJSON 24899 48605 108983 231760 471230
Compress HPack 5727 10781 23162 49099 99575
Gzipped 2929 5374 11224 23167 43550
Gzipped and Minimized 2775 5035 10411 21319 42083
Gzipped and compressed with CJSON 2568 4605 9397 19055 37597
Gzipped and compressed with HPack 1982 3493 6981 13998 27358

Relative size of transformations(%)

The relative size of transformation graphic is useful to see if the size of the json to compress affects the efficiency of compression or minimization. You can notice the following:
the minimization is much more efficient for smaller files. (~60%)
for large and very large json files, the minimization has constant efficiency (~75%)
compressors algorithms has the same efficency for any size of json file
CJson compressing algorithm is less efficient (~45%) than hpack algorithm (~8%)
CJson compressing algorithm is slower than hpack algorihtm
Gzipped content has almost the same size as the compressed content
Combining compression with gzip or minimization with gzip, doesn't improve significantly efficiency (only about 1-2%)
Conclusion
Both JSON compression algorithms are supported by wro4j since version 1.3.8 by the following processors: CJsonProcessor & JsonHPackProcessor. Both of them provide the following methods: pack & unpack. The underlying implementation uses Rhino engine to run the javascript code on the serverside.
JSON Compression algorithms considerably reduce json file size. There a several compression algorithms. We have covered two of them: CJson and HPack. HPack seems to be much more efficient than CJson and also significantly faster. When two entities exchange JSON and the source compress it before it reach the target, the client (target) have to apply the inverse operation of compression (unpacking), otherwise the JSON cannot be used. This introduce a small overhead which must be taken into account when deciding if JSON compression should be used or not.

When gziping of content is allowed, it has a better efficiency than any other compression algorithm. In conclusion, it doesn't worth to compress a JSON on the server if the client accept the gzipped content. The compression on the server-side does make sense when the client doesn't know how to work with gzipped content and it is important to keep the traffic volue as low as possible (due to cost and time).

Onother use-case for JSON compression algorithm is sending a large JSON content from client to server (which is sent ungzipped). In this case, it is important to unpack the JSON content on the server before consuming it.
分享到:
评论

相关推荐

    hpack-java:HPACK 头压缩格式的 Java 实现

    在HTTP/2协议中,为了提高网络传输效率,引入了一种新的头部压缩算法——HPACK(Header Compression for HTTP/2)。本篇文章将详细探讨HPACK头压缩格式的基本原理,并重点分析其在Java环境下的实现方式。 一、HPACK...

    hpack:HPack-PHP中的HTTP2标头压缩实现

    HTTP/2是HTTP协议的最新版本,相比HTTP/1.x,它引入了许多性能优化,其中一项就是使用HPACK(Header Compression)对HTTP头部进行压缩,从而减少数据传输量,提高网络效率。HPack-PHP是一个PHP实现的HPACK库,专门为...

    nim-hpack:HPACK(HTTP2的头压缩)

    在HTTP2协议中,为了提高网络传输效率和降低服务器负载,引入了一种名为HPACK(Header Compression for HTTP/2)的头压缩算法。nim-hpack是Nim编程语言实现的一个HPACK库,它允许开发者在HTTP2协议中高效地处理头部...

    Real-Time Video Compression: Techniques and Algorithms

    Real-Time Video Compression: Techniques and Algorithms introduces the XYZ video compression technique, which operates in three dimensions, eliminating the overhead of motion estimation. This book ...

    ccHPACK:HPACK在C ++中的实现

    HPACK(Header Compression for HTTP/2)是HTTP/2协议中用于头部压缩的一种算法,旨在减少在网络传输中头部信息占用的带宽,从而提高数据传输效率。在HTTP/1.x中,每个请求和响应的头部信息都是明文且未压缩的,这在...

    CagriUysal/Data-Compression-Algorithms:数据压缩算法对比-matlab开发

    以下用于压缩样本文本文件的数据压缩算法: 二元霍夫曼三元霍夫曼算术LZ编码运行 compare_algorithms.m 以获取每个压缩算法的结果(压缩前后的位数)并在图表中查看压缩率,或者如果您想要特定算法的结果,则单独...

    Compression-perception.rar_Compression_algorithms

    Compression algorithms perception of research and application

    Compression-Algorithms:霍夫曼,游程长度+体重,LZ77 ..

    在IT领域,压缩算法是数据处理和存储的关键技术之一,它们通过减少文件大小来提高存储效率和传输速度。本文将详细探讨三种常见的压缩算法:霍夫曼编码(Huffman Coding)、游程长度编码(Run-Length Encoding)以及...

    Chapter 8 Lossy Compression Algorithms.ppt

    Chapter 8 Lossy Compression Algorithms 着重讨论了在多媒体数据压缩中常用的方法和技术,特别是那些允许一定程度数据损失的算法。这些算法之所以被广泛应用,是因为它们能够实现比无损压缩更高的压缩比,这对于...

    Fractal Image Compression:Theory and Application

    This book presents the theory and application of new methods of image compression based on self-transformations of an image. These methods lead to a representation of an image as a fractal, an object ...

    A Survey on Compression Algorithms in Hadoop

    ### Hadoop压缩算法综述 #### 引言 随着信息技术的发展,大数据已成为当前信息技术领域中的一个热点话题。大数据是指在日常生活中产生的数据量极大、种类繁多的数据集合,这些数据可能包括结构化、非结构化或半...

    CZip-Compression-Algorithms:使用霍夫曼编码进行简单文件压缩的​​功能

    CZip压缩算法 这只是一个使用霍夫曼编码压缩文件的示例程序。 狂热地观看了硅谷的所有季节之后,我感觉就像在这样做。 已知错误 当我尝试将其应用于PNG时,由于某种原因,它不起作用。 我怀疑当它在已经具有某种压缩...

    Fractal Image Compression: a MFC project 分形图像压缩

    This MFC project implements the algorithm of book "Fractal Image Compression: Theory and Application". Tested on Visual Studio 2010 of Win 7. Can only read raw images. 分形图像压缩是一种高效的图像...

    GraphicExD for Delphi 2007,2009

    compression: PCD Huffmann specials: sizes: all resolutions, from 192 x 128 up to 6144 x 4096 (64 Base vaporware) rotated: clockwise and counter-clockwise Portable pixel/gray map images (*.ppm, *.pgm, ...

    Python库 | hpack-1.1.0-py2.py3-none-any.whl

    在HTTP/2协议中,头部信息被编码为HPACK(Header Compression for HTTP/2)格式,以减少网络传输的数据量。HPACK库如`hpack`,能够帮助开发者轻松地处理这些头部信息的编码和解码任务。它提供了一组API,使得应用...

    Go-HPACKlibrary是Go语言实现的HPACK压缩库

    Go-HPACK库是为Go编程语言专门设计的一个高效头部压缩库,它实现了HTTP/2协议中的HPACK(Header Compression for HTTP/2)算法。在HTTP/2中,为了提高网络传输效率,头部数据被压缩后再发送,而Go-HPACK库就是这个...

    relaxdays-challenge-compression::laptop:Dockerfile压缩

    cd relaxdays-challenge-compression/ Docker容器 安装 。 使用以下项目构建一个Docker容器: docker build -t relaxdays-challenge-compression . 压缩: docker run -v $( pwd ) :/data -it relaxdays-...

    compression::clamp:Deno压缩中间件

    deflate ) 创建具有应用压缩的Content-Encoding标头如果不支持编码,则发送409 Not Acceptable例子import { serve } from 'https://deno.land/std@0.90.0/http/server.ts'import { compression } from 'https:...

    Chapter 7 Lossless Compression Algorithms.ppt

    在本章《无损压缩算法》中,我们将深入探讨如何有效地减少表示特定信息所需的数据位数,即数据压缩。无损压缩是指压缩和解压缩过程中不会导致任何信息损失的编码方式,而有损压缩则会在还原过程中引入一定程度的信息...

    Compression-and-Decompression-Algorithms:使用滑动窗口LZ77和LZ78方法的压缩和解压缩算法

    "Compression-and-Decompression-Algorithms-master"这个文件名可能是指一个项目仓库,其中包含了使用Java实现的LZ77和LZ78压缩解压缩算法的源代码。通过查看和学习这个项目的代码,你可以更好地理解这两种算法的...

Global site tag (gtag.js) - Google Analytics