johnzon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hendrik Dev <hendrikde...@gmail.com>
Subject Re: New test cases and JsonStreamParser issues
Date Sun, 06 Jul 2014 16:50:40 GMT
new testfiles added (FLEECE-4), i created new ones for those who
potentially had IP problems. The new files contains only random data.

Regarding code formatting: do you have a eclipse code style formatter
export file (matching checkstyle) which i can import?

Thanks
Hendrik

On Sat, Jul 5, 2014 at 11:36 AM, Romain Manni-Bucau
<rmannibucau@gmail.com> wrote:
> Hi
>
> Awesome,
>
> can you please:
>
> 1) fix the code style (add {} etc, all is configured in fleece pom now
> to use checkstyle)
> 2) check you have permissions to use jsons you added (some comes from
> a well known project right?)
> 3) create a  jira and attach you patch to let us applying it?
>
> Thank you a lot!
>
>
>
>
> Romain Manni-Bucau
> Twitter: @rmannibucau
> Blog: http://rmannibucau.wordpress.com/
> LinkedIn: http://fr.linkedin.com/in/rmannibucau
> Github: https://github.com/rmannibucau
>
>
> 2014-07-05 9:53 GMT+02:00 Hendrik Dev <hendrikdev22@gmail.com>:
>> Hi,
>>
>> i have enhanced the set of test json files [1] (and related test
>> cases) to make sure that the json parser works correctly and fails
>> correctly if json is invalid.
>>
>> The current JsonStreamParser.java class does not pass all tests. So i
>> started to fix this but JsonStreamParser was hard to understand for me
>> and i discovered a main problem (see below) which i could not fix and
>> so i wrote
>> a new implementation (JsonByteBufferStreamParser and
>> JsonCharBufferStreamParser [2]). These implementations did pass all
>> new (and old) tests (some older tests had issues because of IMHO wrong
>> tests, see [3]).
>>
>> The main problem with JsonStreamParser was that it fails if the last
>> character in char[] loadedChars, before a direct overflow, is a ending
>> doublequote. Than the overflow happens and something goes wrong with
>> preserving the previously readed value.
>> So if the buffer size (=org.apache.fleece.default-char-buffer) and the
>> json input have the right "bad" correlation then parsing would fail.
>> There are now testcases for this.
>>
>> I did some JMH based benchmarking for the the new parser
>> implementations [4] (based on the ideas from [5]). They show more or
>> less equals figures [6] compared to the original JsonStreamParser.
>>
>> Please let me know what you think about that.
>> Thanks
>> Hendrik
>>
>>
>> [1]
>> https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/resources/json
>> https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/java/org/apache/fleece/core
>>
>> [2]
>> https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/src/main/java/org/apache/fleece/core/JsonByteBufferStreamParser.java
>> https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/src/main/java/org/apache/fleece/core/JsonCharBufferStreamParser.java
>>
>> [3]
>> https://github.com/salyh/fleece_tmp/commit/8377bdf88f86e0ca07b344a87a48241788e4fd7b#commitcomment-6902656
>>
>> [4]
>> https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/resources/bench
>> https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/java/org/apache/fleece/core/jmh/benchmark
>>
>> [5]
>> https://github.com/RichardHightower/json-parsers-benchmark
>>
>> [6]
>> https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/avg_benchmark_jmh_result_f2_t16_w3_i5.txt
>> https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/thr_benchmark_jmh_result_f2_t16_w3_i5.txt
>>
>> --
>> Hendrik Saly (salyh, hendrikdev22)
>> @hendrikdev22
>> PGP: 0x22D7F6EC



-- 
Hendrik Saly (salyh, hendrikdev22)
@hendrikdev22
PGP: 0x22D7F6EC

Mime
View raw message