sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anna Szonyi <szo...@cloudera.com>
Subject Re: Sqoop CI changes
Date Fri, 07 Apr 2017 18:34:55 GMT
Hey Attila,

Thanks for your input! I agree that adding the 3rd party tests to the CI
would be really beneficial. As there are some resourcing problems that need
to be solved for that to happen (I started a discussion with the Kudu team,
however if you have any specific help to add there, I would be very happy
to accept any assistance with that) - I'll start a separate thread where we
can discuss this issue with the community, see if anyone else has any
inputs on it :).

However if no one has any objections about the original proposal, I will
follow through with that in the meantime.

Thanks and Regards,
Anna


On Fri, Apr 7, 2017 at 1:04 PM, Attila Szabó <maugli@apache.org> wrote:

> Hello everyone,
>
> First of all I'd like to thank that Anna is willing to invest some efforts
> on making our CI system better, and sorry for my delayed answer ( however I
> do still hope it helps regardless the stage of the ongoing efforts )
>
> I'd like to share the following thoughts here:
> - Although it would make sense to eliminate the four ( right now totally
> equal ) CI cycles and create only one, but regardless some "static noise"
> it doesn't cause any serious issues for our current commit flow.
> - Creating a precommit hook sounds like a great idea, I would encourage the
> community to move on that path.
> - However: According to my humblest opinion the biggest problem with the
> current CI is that it doesn't execute the so called " 3rd party tests" (
> which is generally our DB integration test layer), and thus it provides
> only a limited safety belt for us ( and we've seen quite a few regression
> on this front in the past one year ). Although we do have solution for
> running those tests manually from command line, it's quite difficult to
> setup/test those things from a single desktop, thus cause serious
> difficulties in validating some changeset before commit.
> - Still an issue we have on this front is our build
> system/scripts/mechanism, which again could slow down the Dev+commit flow.
> Although we've started efforts on this front, the final solution was not
> fully delivered yet.
>
> As a conclusion of the elements above:
> Anna! Would you mind first focusing on the build scripts and the "3rd
> party" CI automation instead of eliminating the obsolete stuff? IMHO that
> would be a much better usage of your efforts and would provide a much
> bigger impact for the community.
>
> With my kindest regards,
> Attila
>
> On Mar 28, 2017 2:44 PM, "Erzsebet Szilagyi" <liz.szilagyi@cloudera.com>
> wrote:
>
> > Great ideas!
> >
> > I agree with Bogi and Szabolcs on the redundant test jobs.
> >
> > Would this pre-commit hook launch the same process as the current
> > post-commit hook, or would this do something different?
> > I think in the first case we could rework the post-commit check into the
> > pre-commit hook, in the latter I'm curious about what exactly this check
> > would add.
> > In general I support the idea: we have seen a number of problems that
> could
> > have been avoided, so this shall be a very useful change!
> >
> > Thank you,
> > Liz
> >
> > On Fri, Mar 24, 2017 at 9:39 AM, Szabolcs Vasas <vasas@cloudera.com>
> > wrote:
> >
> > > Hi Anna,
> > >
> > > Removing the redundant test execution jobs sounds great, I think you
> can
> > go
> > > ahead with that.
> > >
> > > Regarding the pre-commit hook: what would be the purpose of it exactly?
> > > Would it execute the unit tests before the patch is committed?
> > >
> > > Regards,
> > > Szabolcs
> > >
> > > On Thu, Mar 23, 2017 at 4:03 PM, Anna Szonyi <szonyi@cloudera.com>
> > wrote:
> > >
> > > > Hi All,
> > > >
> > > > I would like to make the following changes to the Sqoop CI system:
> > > > Disable the SCM polling for the Sqoop-hadoop23 Sqoop-hadoop20 and
> > > > Sqoop-hadoop100 jobs (and later delete the jobs themselves),
> > > > as the current trunk version of sqoop no longer contains these
> > profiles,
> > > so
> > > > these runs are redundant.
> > > >
> > > > I would also like to propose the creation of a pre-commit hook for
> > Sqoop
> > > > (like the existing one for Sqoop2).
> > > >
> > > > Please let me know if you have any objections.
> > > >
> > > > Thanks,
> > > > Anna
> > > >
> > >
> > >
> > >
> > > --
> > > Szabolcs Vasas
> > > Software Engineer
> > > <http://www.cloudera.com>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message