hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <>
Subject [jira] [Commented] (HIVE-13894) Fix more json related JDK8 test failures Part 2
Date Wed, 01 Jun 2016 18:21:59 GMT


Siddharth Seth commented on HIVE-13894:

bq. All issues we have seen with java8 are order issues on.q.out files only. This is due to
a change in the HashMap implementation that Java8 did. To fix the problem, we had to use LinkedHashMap
in some places, or just change the order of the .q.out file (that seems easier). Even if we
use LinkedHashMap, we had to re-order the .q.out file.
bq. Also, in some cases, like when HashMap is used within external libs like antrl, avro or
json (as in this case) regenerating the golden files in the only viable option (using LinkedHashMap
is not an option as it means adding new code & associated cost just to re-order the output
from these external libs)

HashMap has never guaranteed a sort ordering - which is why it can change between Java 7 and
Java8. I will not be surprised if a JDK from some other company (Java8) - will have a different
sort ordering. A LinkedHashMap solves the problem - but potentially introduces an unnecessary
performance hit and as Mohit pointed out, we cannot guarantee the output from external libs.
To me, the ideal fix would be to change the validation infrastructure to allow a sort within
a single line. Probably a large change, and can be taken up at a later point. Changing the
q file output works for now, at least for jenkins runs - until the next java version.

> Fix more json related JDK8 test failures Part 2
> -----------------------------------------------
>                 Key: HIVE-13894
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Mohit Sabharwal
>            Assignee: Mohit Sabharwal
>         Attachments: HIVE-13894.patch
> After merge of java8 branch to master, some more json ordering related failures 

This message was sent by Atlassian JIRA

View raw message