Oj Callback Performance

< Oj Home >

Oj Callback Performance	peter@ohler.com
Author: Peter Ohler Published: June 1, 2013

Optimized JSON (Oj) is a C extension to Ruby. The Oj callback parsers are similar to the callback parsers provided by the Yajl and Json gems. Oj has 2 callback parsers. The first one, Simple API JSON (saj) is deprecated as of Oj 2.1. The second Simple Callback Parser (scp) has a more flexible API and uses a non-recursive parser. The same underlying parser as the other Oj parsers as of version 2.1.

To benchmark the Oj callback parsers a Ruby Object was selected that included all the various types that can be expected in a JSON document as well as some nested arrays and hashes to add some variety. The file used for generating the benchmark results is in GitHub. The JSON used for the tests is:

{"a":"Alpha","b":true,"c":12345,"d":[true,[false,[-123456789,null],3.9676,["Something else.",false],null]],"e":{"zero":null,"one":1,"two":2,"three":[3],"four":[0,1,2,3,4]},"f":null,"h":{"a":{"b":{"c":{"d":{"e":{"f":{"g":null}}}}}}},"i":[[[[[[[null]]]]]]]}

With that JSON as a building block, longer JSON document are created by repeating that structure in an array. That allows benchmarking larger JSON documents as well as the smaller one.

Benchmarks were done against the latest version of yajl-ruby 1.1.0 and json 1.8.0.

Results

Oj takes a different approach to the callbacks. Any object that implements the callback methods can be used as the handler. If a method is not implemented or public then it is skipped. This allows for very fast parsing if only a subset of the callbacks are desired. The benchmarks show the Oj callback parsers with all methods public and with none public. This establishes the bounds for the performance. Note that in all cases both for Oj and for the other parser handlers, the callback functions are empty. This highlights the parser performance instead of the user provided callback performance.

In the tables, a higher ratio is better. The ratio is a comparison to the slowest parser in the test. The times listed are in seconds and are the number of seconds it took to parse the number of iterations noted in the test title.

Small: 1,000,000 parses of a 255 byte JSON

Gem	Parse Time	Rate	Ratio
Oj::ScHandler No Handlers	1.59 seconds	630K parses/sec	14.23
Oj::Saj No Handlers	1.60 seconds	626K parses/sec	14.13
Oj::ScHandler Full Handlers	8.02 seconds	125K parses/sec	2.82
Oj::Saj Full Handlers	8.03 seconds	124K parses/sec	2.81
JSON::Ext	20.71 seconds	48K parses/sec	1.09
Yajl	22.59 seconds	44K parses/sec	1.00

Both Yajl and the Json extension gems are fairly close in performance while Oj stand out by being 180% faster with a full set of callbacks and 14 times faster with no callbacks. The no callbacks handler is useful when checking the syntax of an JSON document.

Large: 100 parses of a 1000K byte JSON

Gem	Parse Time	Rate	Ratio
Oj::ScHandler No Handlers	0.49 seconds	205 parses/sec	23.98
Oj::Saj No Handlers	0.58 seconds	174 parses/sec	20.30
Oj::Saj Full Handlers	2.81 seconds	36 parses/sec	4.16
Oj::ScHandler Full Handlers	3.07 seconds	33 parses/sec	3.81
Yajl	8.69 seconds	12 parses/sec	1.35
JSON::Ext	11.70 seconds	9 parses/sec	1.00

For larger files of 1M the results change considerably. Yajl is 35% faster than the Json gem while Oj lengthens it's lead to 3.8 times faster for full callbacks and 24 times faster for an empty callback handler.

< Oj Home >