johnzon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Romain Manni-Bucau <rmannibu...@gmail.com>
Subject Re: New test cases and JsonStreamParser issues
Date Sat, 05 Jul 2014 09:36:51 GMT
Hi

Awesome,

can you please:

1) fix the code style (add {} etc, all is configured in fleece pom now
to use checkstyle)
2) check you have permissions to use jsons you added (some comes from
a well known project right?)
3) create a  jira and attach you patch to let us applying it?

Thank you a lot!




Romain Manni-Bucau
Twitter: @rmannibucau
Blog: http://rmannibucau.wordpress.com/
LinkedIn: http://fr.linkedin.com/in/rmannibucau
Github: https://github.com/rmannibucau


2014-07-05 9:53 GMT+02:00 Hendrik Dev <hendrikdev22@gmail.com>:
> Hi,
>
> i have enhanced the set of test json files [1] (and related test
> cases) to make sure that the json parser works correctly and fails
> correctly if json is invalid.
>
> The current JsonStreamParser.java class does not pass all tests. So i
> started to fix this but JsonStreamParser was hard to understand for me
> and i discovered a main problem (see below) which i could not fix and
> so i wrote
> a new implementation (JsonByteBufferStreamParser and
> JsonCharBufferStreamParser [2]). These implementations did pass all
> new (and old) tests (some older tests had issues because of IMHO wrong
> tests, see [3]).
>
> The main problem with JsonStreamParser was that it fails if the last
> character in char[] loadedChars, before a direct overflow, is a ending
> doublequote. Than the overflow happens and something goes wrong with
> preserving the previously readed value.
> So if the buffer size (=org.apache.fleece.default-char-buffer) and the
> json input have the right "bad" correlation then parsing would fail.
> There are now testcases for this.
>
> I did some JMH based benchmarking for the the new parser
> implementations [4] (based on the ideas from [5]). They show more or
> less equals figures [6] compared to the original JsonStreamParser.
>
> Please let me know what you think about that.
> Thanks
> Hendrik
>
>
> [1]
> https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/resources/json
> https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/java/org/apache/fleece/core
>
> [2]
> https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/src/main/java/org/apache/fleece/core/JsonByteBufferStreamParser.java
> https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/src/main/java/org/apache/fleece/core/JsonCharBufferStreamParser.java
>
> [3]
> https://github.com/salyh/fleece_tmp/commit/8377bdf88f86e0ca07b344a87a48241788e4fd7b#commitcomment-6902656
>
> [4]
> https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/resources/bench
> https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/java/org/apache/fleece/core/jmh/benchmark
>
> [5]
> https://github.com/RichardHightower/json-parsers-benchmark
>
> [6]
> https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/avg_benchmark_jmh_result_f2_t16_w3_i5.txt
> https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/thr_benchmark_jmh_result_f2_t16_w3_i5.txt
>
> --
> Hendrik Saly (salyh, hendrikdev22)
> @hendrikdev22
> PGP: 0x22D7F6EC

Mime
View raw message