calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Hyde <jh...@apache.org>
Subject Re: CassandraAdapterTest failure
Date Wed, 01 Aug 2018 19:34:51 GMT
The test is failing every time for me on JDK 10.  The command “mvn -Pit clean test” will
probably reproduce it for most people.

Can other folks please try to reproduce this? I’m getting close to saying that we should
back this change out even though apparently only I can reproduce the failure.

Julian

 




> On Jul 30, 2018, at 1:35 PM, Andrei Sereda <andrei@sereda.cc> wrote:
> 
> Julian, I have amended the commit message in f0b00f0c
> 
> 
> On Mon, Jul 30, 2018 at 4:22 PM Julian Hyde <jhyde@apache.org> wrote:
> 
>> I’m running tests on the PR now.
>> 
>> Since commit comments are a soap-box of mine, I will remark that rather
>> than
>> 
>>  [CALCITE-2428] Fix cassandra unit test initialization. (Andrei Sereda)
>> 
>> the commit comment should be
>> 
>>  [CALCITE-2428] Cassandra unit test fails to parse version string (Andrei
>> Sereda)
>> 
>> Julian
>> 
>> 
>>> On Jul 30, 2018, at 1:13 PM, Michael Mior <mmior@apache.org> wrote:
>>> 
>>> Thanks Andrei for digging into this! Since I haven't reproduced this
>>> failure myself, if someone else could check out the PR, that would be
>>> great.
>>> --
>>> Michael Mior
>>> mmior@apache.org
>>> 
>>> 
>>> 
>>> Le lun. 30 juil. 2018 à 13:29, Andrei Sereda <andrei@sereda.cc> a écrit
>> :
>>> 
>>>> Agree, flaky tests are pretty annoying. I'll try to watch more carefully
>>>> new "embedded data-source" issues (fongo, ES, cassandra). They
>> introduced
>>>> more "non-determinism" because they now run as part of regular build
>> which
>>>> means executed much more often than IT.
>>>> 
>>>> Last commit was for a more deterministic issue (incorrect parsing of new
>>>> java version format <http://openjdk.java.net/jeps/223>).
>>>> 
>>>> 
>>>> On Mon, Jul 30, 2018 at 12:58 PM Julian Hyde <jhyde@apache.org> wrote:
>>>> 
>>>>> On balance, I don’t think we should back out CassandraAdapterTest.
But
>> we
>>>>> do need to continue working to make it more resilient. Any test that
>>>>> generates too many false negatives over the long run should be
>> disabled,
>>>>> and this is no exception.
>>>>> 
>>>>> And as the other test results show, it’s not the only flaky part of
the
>>>>> test suite, when the machine is stressed. One thing we can do is to
>> avoid
>>>>> timeouts whenever possible.
>>>>> 
>>>>> Julian
>>>>> 
>>>>> 
>>>>>> On Jul 30, 2018, at 9:30 AM, Andrei Sereda <andrei@sereda.cc>
wrote:
>>>>>> 
>>>>>> Pls check and confirm that the following PR fixes the issue:
>>>>>> https://github.com/apache/calcite/pull/770
>>>>>> It addresses build failures due to version parsing.
>>>>>> 
>>>>>> Stale folders / files will be addressed in a different PR.
>>>>>> 
>>>>>> 
>>>>>> On Mon, Jul 30, 2018 at 10:10 AM Andrei Sereda <andrei@sereda.cc>
>>>> wrote:
>>>>>> 
>>>>>>> This must be something specific to Cassandra Unit. Will check
>>>>>>> 
>>>>>>> On Mon, Jul 30, 2018, 08:59 Sergey Nuyanzin <snuyanzin@gmail.com>
>>>>> wrote:
>>>>>>> 
>>>>>>>> There is one more strange thing (at least on Windows): while
>>>> building a
>>>>>>>> file with name ".toDelete" is generated under calcite\cassandra
and
>>>>> it's
>>>>>>>> not removed by the end of tests
>>>>>>>> Is there a way to make cassandra generates these files in
target
>>>>> directory
>>>>>>>> e.g.?
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Mon, Jul 30, 2018 at 3:52 PM, Andrei Sereda <andrei@sereda.cc>
>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Most of the problems are during test init Phase. Most
likely with
>>>>>>>> version
>>>>>>>>> string (eg. 11-ea for JDK11).
>>>>>>>>> I'll fix that.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Mon, Jul 30, 2018 at 8:16 AM Michael Mior <mmior@apache.org>
>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> I tested myself a fair bit under Ubuntu before pushing
this and
>>>>> didn't
>>>>>>>>> see
>>>>>>>>>> any of these issues myself. That said, I agree that
it's important
>>>>> the
>>>>>>>>> test
>>>>>>>>>> suite be stable. I'm fine with reverting for now
or (more
>>>> preferably
>>>>>>>> IMO)
>>>>>>>>>> just disabling these tests by default.
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> Michael Mior
>>>>>>>>>> mmior@apache.org
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Le lun. 30 juil. 2018 à 00:59, Julian Hyde <jhyde@apache.org>
a
>>>>>>>> écrit :
>>>>>>>>>> 
>>>>>>>>>>> I ran on several JDK versions, all on Ubuntu
Linux. The machine
>>>> was
>>>>>>>>>>> fairly heavily loaded (I was copying one filesystem
to another at
>>>>>>>> the
>>>>>>>>>>> time).
>>>>>>>>>>> 
>>>>>>>>>>> Here's the error from OpenJDK10:
>>>>>>>>>>> 
>>>>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1,
Skipped: 0, Time
>>>>>>>>>>> elapsed: 0.001 s <<< FAILURE! - in
>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest
 Time
>>>> elapsed:
>>>>>>>>>>> 0.001 s  <<< ERROR!
>>>>>>>>>>> java.lang.ExceptionInInitializerError
>>>>>>>>>>>      at
>>>>>>>>>>> 
>>>>>>>>>> 
>>>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>>>>>>>>> CassandraAdapterTest.java:106)
>>>>>>>>>>>      at
>>>>>>>>>>> 
>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>>>>>>>>> CassandraAdapterTest.java:56)
>>>>>>>>>>> Caused by: java.lang.StringIndexOutOfBoundsException:
begin 0,
>> end
>>>>>>>> -1,
>>>>>>>>>>> length 5
>>>>>>>>>>>      at
>>>>>>>>>>> 
>>>>>>>>>> 
>>>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>>>>>>>>> CassandraAdapterTest.java:106)
>>>>>>>>>>>      at
>>>>>>>>>>> 
>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>>>>>>>>> CassandraAdapterTest.java:56)
>>>>>>>>>>> 
>>>>>>>>>>> Here's the error in JDK 11:
>>>>>>>>>>> 
>>>>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1,
Skipped: 0, Time
>>>>>>>>>>> elapsed: 0 s <<< FAILURE! - in
>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest
 Time
>>>> elapsed:
>>>>>>>> 0
>>>>>>>>>>> s  <<< ERROR!
>>>>>>>>>>> java.lang.ExceptionInInitializerError
>>>>>>>>>>>      at
>>>>>>>>>>> 
>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.enabled(
>>>>>>>>> CassandraAdapterTest.java:81)
>>>>>>>>>>>      at
>>>>>>>>>>> 
>>>>>>>>>> 
>>>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>>>>>>>>> CassandraAdapterTest.java:88)
>>>>>>>>>>>      at
>>>>>>>>>>> 
>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>>>>>>>>> CassandraAdapterTest.java:56)
>>>>>>>>>>> 
>>>>>>>>>>> Here's the failure from JDK 9:
>>>>>>>>>>> 
>>>>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>> [ERROR] Tests run: 1, Failures: 1, Errors: 0,
Skipped: 0, Time
>>>>>>>>>>> elapsed: 0.001 s <<< FAILURE! - in
>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest
 Time
>>>> elapsed:
>>>>>>>>>>> 0.001 s  <<< FAILURE!
>>>>>>>>>>> java.lang.AssertionError: Cassandra daemon did
not start within
>>>>>>>> timeout
>>>>>>>>>>> 
>>>>>>>>>>> Here's the failure from JDK 10:
>>>>>>>>>>> 
>>>>>>>>>>> [INFO] Running org.apache.calcite.jdbc.CalciteRemoteDriverTest
>>>>>>>>>>> [ERROR] Tests run: 28, Failures: 1, Errors: 0,
Skipped: 0, Time
>>>>>>>>>>> elapsed: 0.642 s <<< FAILURE! - in
>>>>>>>>>>> org.apache.calcite.jdbc.CalciteRemoteDriverTest
>>>>>>>>>>> [ERROR]
>>>>>>>>>>> testRemoteExecuteQuery(org.apache.calcite.jdbc.
>>>>>>>>> CalciteRemoteDriverTest)
>>>>>>>>>>> Time elapsed: 0.047 s  <<< FAILURE!
>>>>>>>>>>> java.lang.AssertionError:
>>>>>>>>>>> 
>>>>>>>>>>> Expected: "EXPR$0=1; EXPR$1=a\nEXPR$0=null; EXPR$1=b"
>>>>>>>>>>>   but: was ""
>>>>>>>>>>>      at
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>> org.apache.calcite.jdbc.CalciteRemoteDriverTest.testRemoteExecuteQuery(
>>>>>>>>> CalciteRemoteDriverTest.java:280)
>>>>>>>>>>> 
>>>>>>>>>>> In conclusion: It's worrying that the suite shows
4 different
>>>> cracks
>>>>>>>>>>> under 4 different JDKs. Clearly the load on my
machine was making
>>>>>>>>>>> problems worse, and granted, the problems are
just testing
>>>> problems,
>>>>>>>>>>> not real bugs. But flaky test suites waste time
and effort. There
>>>>>>>> are
>>>>>>>>>>> indications that the new embedded Cassandra test
is more flaky
>>>> than
>>>>>>>>>>> most.
>>>>>>>>>>> 
>>>>>>>>>>> Julian
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Sun, Jul 29, 2018 at 11:33 AM, Andrei Sereda
<andrei@sereda.cc
>>>>> 
>>>>>>>>>> wrote:
>>>>>>>>>>>> What version of java / OS / maven do you
have ? What is your
>>>> maven
>>>>>>>>>>> command
>>>>>>>>>>>> ?
>>>>>>>>>>>> 
>>>>>>>>>>>> Things pass for me on MacOS X java 8,9 and
10.
>>>>>>>>>>>> 
>>>>>>>>>>>> On Sun, Jul 29, 2018 at 2:05 PM Julian Hyde
<jhyde@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> I'm seeing the following error when I
run the tests on
>> bd0e14002
>>>>>>>>>>>>> origin/master. Anyone else see it?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors:
1, Skipped: 0, Time
>>>>>>>>>>>>> elapsed: 0.001 s <<< FAILURE!
- in
>>>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest
 Time
>>>>>>>> elapsed:
>>>>>>>>>>>>> 0.001 s  <<< ERROR!
>>>>>>>>>>>>> java.lang.ExceptionInInitializerError
>>>>>>>>>>>>>      at
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>>>>>>>>> CassandraAdapterTest.java:106)
>>>>>>>>>>>>>      at
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>>>>>>>>> CassandraAdapterTest.java:56)
>>>>>>>>>>>>> Caused by: java.lang.StringIndexOutOfBoundsException:
begin 0,
>>>>>>>> end
>>>>>>>>> -1,
>>>>>>>>>>>>> length 5
>>>>>>>>>>>>>      at
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>>>>>>>>> CassandraAdapterTest.java:106)
>>>>>>>>>>>>>      at
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>>>>>>>>> CassandraAdapterTest.java:56)
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Best regards,
>>>>>>>> Sergey
>>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 
>>>> 
>> 
>> 


Mime
View raw message