calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Beikov <christian.bei...@gmail.com>
Subject Re: Elasticsearch integration test failures
Date Wed, 13 Jun 2018 04:31:57 GMT
Interesting, I tested both environments back then and it seemed to work. 
Good thing that we are about to replace the IT tests with using local ES 
nodes. This will finally allow us to run these tests regularly.

Regarding ES2 and ES5, I'd suggest you have separate 
ElasticsearchProjectRule for each version. In Elasticsearch2Table and 
Elasticsearch5Table you add the rules to the ElasticsearchTableScan 
constructor.


Mit freundlichen Grüßen,
------------------------------------------------------------------------
*Christian Beikov*
Am 13.06.2018 um 06:15 schrieb Andrei Sereda:
> The issue seems to have started since CALCITE-1967 (ES5 support).
> ES2 expects keyword _source in scripted fields while ES5 param._source.
> See  ElasticsearchProject
> <https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/adapter/elasticsearch/ElasticsearchProject.java#L79>
> .
>
> Also ES2 doesn't like when both script_fields and _source are present in
> the query. So if there are some explicit projections all have to be present
> in script_fields.
>
> Any elegant  way to differentiate at runtime ES2 vs ES5 inside
> ElasticsearchProject ?
>
>
>
> On Mon, Jun 11, 2018 at 5:46 PM Christian Beikov <christian.beikov@gmail.com>
> wrote:
>
>> I'm fine with a single PR as long as you split commits up.
>>
>> Andrei Sereda <andrei@sereda.cc> schrieb am Mo., 11. Juni 2018, 23:24:
>>
>>> Hi Christian,
>>>
>>> I'm working in parallel on CALCITE-2331 evaluation of predicate "(A or B)
>>> and C" for ES. Can I fix existing bugs as part of that (or separate)
>> commit
>>> ?
>>>
>>> I was planning to do it just not in the same PR.
>>>
>>> On Mon, Jun 11, 2018 at 4:31 PM, Christian Beikov <
>>> christian.beikov@gmail.com> wrote:
>>>
>>>> For me that's fine, though it would be great if you could fix those
>> given
>>>> your time permits it.
>>>>
>>>> Andrei Sereda <andrei@sereda.cc> schrieb am Mo., 11. Juni 2018, 22:12:
>>>>
>>>>> There are some tests which have been failing for while. Is it OK if I
>>>>> temporary mark them @Ignore(d) ?
>>>>>
>>>>> On Mon, Jun 11, 2018 at 3:48 PM, Christian Beikov <
>>>>> christian.beikov@gmail.com> wrote:
>>>>>
>>>>>> If you migrate the existing tests to use the local node in that PR,
>>> I'd
>>>>>> consider merging that PR given that Travis CI reports no test
>>> failures.
>>>>>>
>>>>>> Mit freundlichen Grüßen,
>>>>>> ------------------------------------------------------------
>>>> ------------
>>>>>> *Christian Beikov*
>>>>>> Am 11.06.2018 um 14:33 schrieb Andrei Sereda:
>>>>>>
>>>>>>> Do the tests succeed with the local node?
>>>>>>> IT tests haven't been migrated to local node yet. I wanted to
do
>> it
>>> as
>>>>>>> separate commit.
>>>>>>> There are new tests which run against embedded instance
>>> successfully.
>>>>> Pls
>>>>>>> see PR 716 for details.
>>>>>>>
>>>>>>> I don't see a reason for running ES tests again on a "real" ES
>>>> instance.
>>>>>>> What does that even mean?
>>>>>>> The only reason, I see, in connecting to external instance is
>>> testing
>>>>>>> compatibility with different versions of the engine.
>>>>>>>
>>>>>>> On Mon, Jun 11, 2018 at 7:16 AM, Michael Mior <mmior@apache.org>
>>>> wrote:
>>>>>>> Good point. Since ES is written in Java and we're running the
same
>>>> code
>>>>>>>> testing on a local node, it's probably not necessary to test
>>> against
>>>> a
>>>>>>>> separate ES cluster.
>>>>>>>> --
>>>>>>>> Michael Mior
>>>>>>>> mmior@apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>> Le lun. 11 juin 2018 à 01:44, Christian Beikov <
>>>>>>>> christian.beikov@gmail.com
>>>>>>>> a écrit :
>>>>>>>>
>>>>>>>> I don't see a reason for running ES tests again on a "real"
ES
>>>>> instance.
>>>>>>>>> What does that even mean?
>>>>>>>>>
>>>>>>>>> With a local node, we would be testing functionality
against
>>> almost
>>>>> the
>>>>>>>>> real thing. The only difference is that a real setup
has a
>>> cluster,
>>>>> but
>>>>>>>>> that doesn't change the functionality. So I'm all in
on using
>>> just a
>>>>>>>>> local node which is also something I wanted to do for
a long
>> time,
>>>> but
>>>>>>>>> you got to it before I did.
>>>>>>>>>
>>>>>>>>> Do the tests succeed with the local node?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Mit freundlichen Grüßen,
>>>>>>>>> ------------------------------------------------------------
>>>>>>>>> ------------
>>>>>>>>> *Christian Beikov*
>>>>>>>>> Am 07.06.2018 um 20:47 schrieb Andrei Sereda:
>>>>>>>>>
>>>>>>>>>> Hi Volodymyr,
>>>>>>>>>>
>>>>>>>>>> We're trying to bridge the gap between unit and integration
>> tests
>>>>>>>>> (which
>>>>>>>>> are run infrequently) using fakes
>>>>>>>>>> <https://martinfowler.com/bliki/InMemoryTestDatabase.html>
in
>>>>> Calcite.
>>>>>>>>>> Currently the plan is to use fongo (see CALCITE-2345
for mongo
>>>>> adapter)
>>>>>>>>> and
>>>>>>>>>
>>>>>>>>>> local elastic node (see CALCITE-2347 for elastic).
>>>>>>>>>>
>>>>>>>>>> During unit tests code will run against fake instances.
For IT
>>>>> profile
>>>>>>>>> they
>>>>>>>>>
>>>>>>>>>> will run against real instances (as before).
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Andrei.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Jun 7, 2018 at 2:32 PM, Julian Hyde <jhyde@apache.org>
>>>>> wrote:
>>>>>>>>>> Unfortunately, I don’t know. We don’t run the
integration tests
>>> as
>>>>>>>>>> often
>>>>>>>>> as we should.
>>>>>>>>>>> On Jun 7, 2018, at 10:20 AM, Vova Vysotskyi <vvovyk@gmail.com
>>>>>>>>>>> wrote:
>>>>>>>>> Hi all,
>>>>>>>>>>>> I tried to run Elasticsearch integration
tests for the
>> current
>>>>>>>>>>> master,
>>>>>>>>> and
>>>>>>>>>>>> many tests failed.
>>>>>>>>>>>>
>>>>>>>>>>>> Is it a known problem, or just an issue of
my env?
>>>>>>>>>>>>
>>>>>>>>>>>> I noticed that one failure appears after
the change made in
>>>>>>>>>>>>
>>>>>>>>>>> CALCITE-2316.
>>>>>>>>>> Most of the other failures are observed for 1.16
version also.
>>>>>>>>>>>> Kind regards,
>>>>>>>>>>>> Volodymyr Vysotskyi
>>>>>>>>>>>>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message