sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Attila Szabó <mau...@apache.org>
Subject Re: Running 3rd party test on the CI
Date Mon, 10 Apr 2017 15:37:05 GMT
Hello everyone,

My arguments in this topic are the following:

Having reliable ( ~ 24/7 online, of course not with five 9 constraints )
external database resources sounds great, would solve most of the
problems/concerns right now we do see about making 3rd party tests
automated. If this is the best practice in the Kudu team and we could be
connected to the same donating companies than it's instantly a +2 on my
side. The two drawbacks/concerns I do find here are:
- can we depend on these resources in long term ( ~ years ), and what
should we / the build do if by any reason some of the resources are not
- if the resources are only available for the CI system how will we
handle/avoid the "works for me" situations. If available for all of the
contributors how to avoid the DBs become messy?

On the docker side:
It's advantage is only it's flexibility. If the CI host machine contains
enough resources to fire up a DB instance ( or even all, depends on how do
we group/organize the testing of each db server ), then we could be sure
about that the success of the execution won't depend on any configuration,
network, etc. Issues. ( and even count on having a clean db at every start
). But in this case the contributor setup could have 0 difference compared
to the CI setup. Of course it's drawback is the very same e.g. meaningful
performance testing shouldn't be done in a Docker environment ( according
to my humblest oppinion ), whilst with donated bare metal resources it
would be also achievable.

On my side I would be happy with both of the solution, the only reason why
I would favor Docker based testing is the configuration identity. But this
is an argument I could easily let go.

The only strong argument I do have in this topic is the following:
If possible I would like to have one single and clean working solution, and
not the mixture of any of the ideas listed ( above or later ). Thus we
could avoid several flakey and configuration based testing issues.


PS: Many thanks for Anna bringing this topic to the dev list!

On Apr 10, 2017 1:50 PM, "Anna Szonyi" <szonyi@cloudera.com> wrote:

Hi All,

We started a discussion about running thirdparty integration tests in our
CI on separate threads (another e-mail chain and a review), and I thought
it might make sense to bring these threads together.

We were discussing the best way of running the thirdparty tests on the
apache CI, as we think it would be very beneficial.

We were floating around the idea of trying to get resources (mysql, cubrid,
etc.) "donated", if that's possible (as far as I understand that's more or
less how Kudu deals with the same issue).

Attila proposed to investigate using docker images.

I would like to bring this up on the dev list - in case anyone has any
experience with this (using external resources from apache CIs), or anyone
has any concerns/negative experiences.

Please chime in here if you have any ideas/workable solutions for this or
any objections.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message