sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erzsebet Szilagyi <liz.szila...@cloudera.com>
Subject Re: Release 1.5 (maybe 1.4.7 ? ), CI, build system, etc.
Date Fri, 09 Dec 2016 13:13:51 GMT
Hi Attila,
I welcome your suggestions!

I agree it's time for a new release. Seeing this lack of enthusiasm I think
it would be best to aim for 1.4.7 and hope the community-buzz attracts even
more of it so we could soon plan for 1.5 too.

I see there's already a lot being done to improve on the issues you've
highlighted. Could you give us a (very short) update on the changes?

Thanks,
Liz



On Mon, Dec 5, 2016 at 5:57 PM, Abraham Elmahrek <abraham@elmahrek.com>
wrote:

> Attila,
>
> I think improving build infrastructure for Sqoop makes a lot of sense. I
> think focusing on build system and tests works. Please be cognizant of the
> following:
>
>    - Docs for Sqoop1 and Sqoop2 are built in different ways. It might be
>    interesting to update how docs are built in Sqoop1.
>    - Apache Rat and Cobertura are being used in Sqoop1 I believe. Please
>    make sure these aren't left out.
>
> Also, would the build system change or stay the same? c.f.
> https://builds.apache.org/job/Sqoop-hadoop100/1036/search/?q=Sqoop.
>
> Abe
>
> On Wed, Nov 30, 2016 at 12:27 PM Attila Szabo <maugli@apache.org> wrote:
>
> > Dear PMC, dear community,
> >
> > In the past few months some of us has been already identified that there
> > would be a need to improve the test coverage, cleanup the CI system, make
> > the build system much more straightforward for the trunk version of
> Sqoop.
> >
> > I think our goals are very simple here:
> >
> >    - It's very difficult to onboard new committers for the community if
> the
> >    component itself is very difficult to test/learn to test, and the
> > CI/JIRA
> >    system sends out false failures after each commit.
> >    - It would be also nice to leverage again from the safety belt of CI,
> >    and not push all of the testing burden on the shoulder of the
> committer
> >    alone (of course the contributors has to do tests, but as said it's
> not
> >    straightforward now to execute all tests, and also it's still the
> >    committers responsibility to ensure nothing is broken after a commit,
> > thus
> >    every safety belt there is needed).
> >
> >
> > It's been also identified that it would be good to release a new version
> of
> > trunk, thus including the improvements of the past 8-9 months, and also
> > show the livingness of the component for the outside of the world.
> >
> > Most probably because of lack of time we didn't achieve all of these
> goals
> > in the past few months. Thus right now I'd like to take the initiative,
> and
> > share my thoughts and goals with you, trying to push forward the above
> > mentioned things, but of course before I'd like to collect the invaluable
> > input and wisdom of the community.
> >
> > The plan would be the following:
> >
> >    - A few weeks ago I've opened three JIRAs (depending on each other
> >    linearly) upstream (SQOOP-3050, SQOOP-3051 and SQOOP-3052).
> >    - SQOOP-3050 is about fix the current test cases (mainly failing
> because
> >    of configuration issues)
> >    - SQOOP-3051 is about cleaning up the build.xml from the obsolete
> >    profiles (e.g. currently 20,23,100 are failing with tons of test
> cases,
> > and
> >    even 200 is not able to correctly run HCatalog tests for example
> > because of
> >    incompatible class changes).
> >    - SQOOP-3052 is about to create a new and more robust build system for
> >    Sqoop (e.g. Maven/Gradle, maybe both)
> >    - Anna Szonyi (a quite new contributor, but quite active in the past 1
> >    month) has jumped on SQOOP-3050 and it seems she's finished with those
> >    efforts (so all unit+third_party test cases are running with the
> > recently
> >    created hadoop260 profile).
> >    - AFAIK Anna is also willing to jump on SQOOP-3051, which would be
> about
> >    cleanup the old profiles from the build.xml and ivysettings, and keep
> > only
> >    one profile which is capable for compiling/packaging a correct version
> > of
> >    Sqoop against we can run all of the available test cases.
> >    - In connection with the goals SQOOP-3052 I do remember that Sowmya
> and
> >    Venkat has identified their willingness to do that. If they currently
> > have
> >    time to do that (after SQOOP-3051 is committed which is about to
> happen
> > by
> >    the end of the week) I would be very glad to see that achievement
> > coming by
> >    their contribution. If by any reason they would not have time for that
> > or
> >    not interested any more, I'm also willing to find volunteers to do
> that.
> >    - Personally I would be very eager to drive through these build
> related
> >    changes through the CI system + JIRA + JIRA bots + etc.
> >    - On the top of these things I'd like to also make one another thing
> >    happen before the next release, and that would be about to create a
> > proper
> >    quoting support for MySQL and PostgreSQL as well. If I'm not mistaken
> > the
> >    Oracle quoting was originated from Sowmya, so it would be also a good
> >    candidate for her to design+create a generalized and centralized
> quoting
> >    mechanism for Sqoop, and have 2-3 different strategies/db (e.g.
> >    Oracle/MySQL/PostgreSQL, etc.). I would be happy to provide my help
> (by
> >    creating the related JIRA tasks, and sharing my insights, and
> coaching)
> > to
> >    her, or to anyone willing to do this feature if Sowmya wouldn't have
> > time
> >    right now to do that.
> >    - If we're aiming for 1.4.7 I think this scope should be enough.
> >    - If we're aiming for 1.5. I think we should also include the
> >    elimination of the com.cloudera classes+packages.
> >    - I would be also volunteering to drive+own the release and the
> release
> >    process.
> >
> > What do you think about this plan? Could you please share your thoughts?
> >
> > Many thanks in advance,
> > Attila
> >
>



-- 
Erzsebet Szilagyi
Software Engineer
[image: www.cloudera.com] <http://www.cloudera.com>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message