sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Szabolcs Vasas <va...@apache.org>
Subject Re: Sqoop build infrastructure improvements
Date Mon, 10 Dec 2018 13:30:10 GMT
Hi Attila,

Thanks for raising this, I think it is a great idea to migrate as soon as
possible, I have started a separate mail chain which can be used as a proof
of community consensus.

On Sun, Dec 9, 2018 at 9:48 PM Attila Szabó <maugli@apache.org> wrote:

> Hello everyone,
>
> So if I'm not mistaken and according to the INFRA mail we've received
> during the weekend gitbox.apache.org is exactly what we wanted to have :
> Having both ASF commits and push privileges on Github side..
> We would just need to have an official consensus in the dev community (
> documented on the mail list), fire an INFRA Jira ticket, and after the
> merge update our site with the new "how to contribute" information.
>
> I think this would be a quite good timing for the Sqoop project as we're
> right now in the middle of these infrastructural reworks.
>
> Kind regards,
> Attila
>
> On Fri, Nov 30, 2018, 2:38 PM Szabolcs Vasas <vasas@apache.org wrote:
>
> > Thanks for the question, I think we should stick to the commit format we
> > had earlier so yes, we should go for squash and push.
> > The easiest solution could be to download the diff file instead of the
> > patch (e.g. https://github.com/apache/sqoop/pull/60.diff instead of
> > https://github.com/apache/sqoop/pull/60.patch) and that does the trick.
> >
> > The "This closes..." commit message just closes the PR but does not
> delete
> > the feature branch, asfgit most probably does not have delete permission
> > for these branches anyway.
> >
> >
> > On Fri, Nov 30, 2018 at 11:45 AM Attila Szabó <maugli@apache.org> wrote:
> >
> > > Hey Szabi,
> > >
> > > Thanks for the prompt response!
> > >
> > > I've thought the repos are connected back and forth and the close PR is
> > the
> > > official way. In case of we still stack to the patch file and git apply
> > > then commit and push approach.
> > >
> > > My only question in this case :
> > > Do we have any agreement on how we commit these PRs. I would vote for
> > > Squash and push, but of course I'm open for the discussion.
> > >
> > > BTW :
> > > Is "This closes" message deletes the branch automatically?
> > >
> > > On front of Github + Jira:
> > > I'm aware Github has the feature to connect with Jira so I'm pretty
> sure
> > it
> > > doable. Also I'm not sure if any ASF project has done it ever, but I'll
> > ask
> > > around in other communities.
> > >
> > > Cheers,
> > > Attila
> > >
> > >
> > >
> > > On Nov 30, 2018 11:03 AM, "Szabolcs Vasas" <vasas@apache.org> wrote:
> > >
> > > Hi Attila,
> > >
> > > I think we won't be able to commit the pull requests on GitHub directly
> > > because our GitHub repo is just a mirror of the original Apache Sqoop
> > repo
> > > <https://git-wip-us.apache.org/repos/asf/sqoop.git>.
> > > However the commit process is still simplified, the ASF GitHub Bot JIRA
> > > comment contains the patch download link (e.g.
> > > https://github.com/apache/sqoop/pull/60.patch) and the commit message
> > > (e.g. This
> > > closes #60) you need to include to close the pull request. The rest of
> > the
> > > process remains the same, you need to apply the patch manually and push
> > the
> > > commit to git-wip-us repo.
> > >
> > > Regarding the GitHub UI improvement: I am not sure GitHub provides
> such a
> > > feature, do you know an example repository where this is implemented?
> > >
> > > Regards,
> > > Szabolcs
> > >
> > >
> > >
> > > On Fri, Nov 30, 2018 at 3:21 AM Attila Szabó <maugli@apache.org>
> wrote:
> > >
> > > > Hi Szabi,
> > > >
> > > > First of all:
> > > > Big Kudos for the more mature gradle build! I think this is a great
> > step
> > > > for the whole project!
> > > >
> > > > On the front of PRs:
> > > > I would only make it official if the user management / authorization
> > > > handling could be somehow automatically bound to the id.apache.org +
> > > > project privileges.
> > > > A good example:
> > > > Today I reviewed SQOOP-3396 but I would not had been able to merge it
> > > > because it seems on the Github project I do not have push / merge
> > > > privileges (regardless that I'm a Sqoop committer and also memeber of
> > the
> > > > ASF group on github with my user).
> > > > So if we can somehow bound these things together, and the majority of
> > the
> > > > ppl would like to use it instead of the Review Board then let it
> > happen!
> > > > I'm fine with both tools until there's no difference between the
> Github
> > > and
> > > > ASF repos from user management POV.
> > > >
> > > > On the top of this:
> > > > I'm not sure if it belongs to our table, or the ASF INFRA team, but
> it
> > > > would be nice if the PRs and the JIRA tickets would be connected
> > > > automatically on the Github UI as well, and thus the navigation to
> > > > issues.apache.org would be easier!
> > > >
> > > > On the front of the gradle build:
> > > > I've found a smaller issue with clean+unittest within the same
> command.
> > > > I've opened a ticket (SQOOP-3415) and a PR (just the follow the new
> > > > standard) with a solution proposal.
> > > >
> > > > My2cents,
> > > > Attila
> > > >
> > > > On Wed, Nov 28, 2018 at 4:54 PM Szabolcs Vasas
> > > <vasas@cloudera.com.invalid
> > > > >
> > > > wrote:
> > > >
> > > > > Dear Sqoop community,
> > > > >
> > > > > We have been working on quite a few exciting build infrastructure
> > > > > improvements recently, I am sending this email to summarize them.
> > > > >
> > > > > *Gradle can now execute all the Sqoop tests in a single JVM*
> > > > > This improvement makes the Gradle test tasks significantly faster
> > since
> > > > we
> > > > > do not have to start up a new JVM for every test class. It also
> made
> > > > > possible to introduce fine grained test categories which were
> > essential
> > > > to
> > > > > be able to parallelize the test execution in our CI systems. For
> more
> > > > > information please refer to COMPILING.txt
> > > > > <https://github.com/apache/sqoop/blob/trunk/COMPILING.txt>.
> > > > >
> > > > > *Apache Sqoop Jenkins job
> > > > > <https://builds.apache.org/job/Sqoop-hadoop200/> now builds
and
> > tests
> > > > with
> > > > > Gradle*
> > > > > Since our Gradle build became much more stable and faster it made
> > sense
> > > > to
> > > > > reconfigure our Jenkins job to benefit from these improvements. The
> > job
> > > > is
> > > > > faster now (~30 minutes instead of ~40) and it executes all of the
> > > tests
> > > > > which can be run without external RDBMS or cloud systems (while the
> > old
> > > > Ant
> > > > > based job executed the unit test suite only).
> > > > >
> > > > > *Travis CI is enabled for Apache Sqoop*
> > > > > The new Travis CI job <https://travis-ci.org/apache/sqoop>
now
> runs
> > > for
> > > > > every commit and every pull request on Apache Sqoop GitHub
> repository
> > > and
> > > > > it executes all of the tests except the Oracle third party test
> > cases.
> > > > One
> > > > > of the biggest benefit of Travis CI is that it can be really easily
> > > > > configured for the individual forks as well so contributors get a
> > well
> > > > > configured CI job for their own feature branches for free. For more
> > > > > information please refer to COMPILING.txt
> > > > > <https://github.com/apache/sqoop/blob/trunk/COMPILING.txt>.
> > > > >
> > > > >
> > > > > Since we have a CI job now which integrates very well with GitHub
> > pull
> > > > > requests I suggest deprecating the old Review Board and patch file
> > > based
> > > > > contribution process and use pull requests in the future. We had
a
> > mail
> > > > > chain about the same proposal last year and it seemed that the
> > > community
> > > > > was happy about the idea so I think we can evaluate it for some
> time
> > > and
> > > > if
> > > > > everything goes well we can update our how to contribute wiki.
> > > > >
> > > > > Feel free to reply to this chain with your questions and
> suggestions
> > on
> > > > the
> > > > > above!
> > > > >
> > > > > Regards,
> > > > > Szabolcs
> > > > >
> > >
> > > <http://www.cloudera.com>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message