johnzon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hendrik Dev <hendrikde...@gmail.com>
Subject New test cases and JsonStreamParser issues
Date Sat, 05 Jul 2014 07:53:37 GMT
Hi,

i have enhanced the set of test json files [1] (and related test
cases) to make sure that the json parser works correctly and fails
correctly if json is invalid.

The current JsonStreamParser.java class does not pass all tests. So i
started to fix this but JsonStreamParser was hard to understand for me
and i discovered a main problem (see below) which i could not fix and
so i wrote
a new implementation (JsonByteBufferStreamParser and
JsonCharBufferStreamParser [2]). These implementations did pass all
new (and old) tests (some older tests had issues because of IMHO wrong
tests, see [3]).

The main problem with JsonStreamParser was that it fails if the last
character in char[] loadedChars, before a direct overflow, is a ending
doublequote. Than the overflow happens and something goes wrong with
preserving the previously readed value.
So if the buffer size (=org.apache.fleece.default-char-buffer) and the
json input have the right "bad" correlation then parsing would fail.
There are now testcases for this.

I did some JMH based benchmarking for the the new parser
implementations [4] (based on the ideas from [5]). They show more or
less equals figures [6] compared to the original JsonStreamParser.

Please let me know what you think about that.
Thanks
Hendrik


[1]
https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/resources/json
https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/java/org/apache/fleece/core

[2]
https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/src/main/java/org/apache/fleece/core/JsonByteBufferStreamParser.java
https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/src/main/java/org/apache/fleece/core/JsonCharBufferStreamParser.java

[3]
https://github.com/salyh/fleece_tmp/commit/8377bdf88f86e0ca07b344a87a48241788e4fd7b#commitcomment-6902656

[4]
https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/resources/bench
https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/java/org/apache/fleece/core/jmh/benchmark

[5]
https://github.com/RichardHightower/json-parsers-benchmark

[6]
https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/avg_benchmark_jmh_result_f2_t16_w3_i5.txt
https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/thr_benchmark_jmh_result_f2_t16_w3_i5.txt

-- 
Hendrik Saly (salyh, hendrikdev22)
@hendrikdev22
PGP: 0x22D7F6EC

Mime
View raw message