sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Attila Szabó <mau...@apache.org>
Subject Re: Sqoop build infrastructure improvements
Date Fri, 30 Nov 2018 02:21:33 GMT
Hi Szabi,

First of all:
Big Kudos for the more mature gradle build! I think this is a great step
for the whole project!

On the front of PRs:
I would only make it official if the user management / authorization
handling could be somehow automatically bound to the id.apache.org +
project privileges.
A good example:
Today I reviewed SQOOP-3396 but I would not had been able to merge it
because it seems on the Github project I do not have push / merge
privileges (regardless that I'm a Sqoop committer and also memeber of the
ASF group on github with my user).
So if we can somehow bound these things together, and the majority of the
ppl would like to use it instead of the Review Board then let it happen!
I'm fine with both tools until there's no difference between the Github and
ASF repos from user management POV.

On the top of this:
I'm not sure if it belongs to our table, or the ASF INFRA team, but it
would be nice if the PRs and the JIRA tickets would be connected
automatically on the Github UI as well, and thus the navigation to
issues.apache.org would be easier!

On the front of the gradle build:
I've found a smaller issue with clean+unittest within the same command.
I've opened a ticket (SQOOP-3415) and a PR (just the follow the new
standard) with a solution proposal.


On Wed, Nov 28, 2018 at 4:54 PM Szabolcs Vasas <vasas@cloudera.com.invalid>

> Dear Sqoop community,
> We have been working on quite a few exciting build infrastructure
> improvements recently, I am sending this email to summarize them.
> *Gradle can now execute all the Sqoop tests in a single JVM*
> This improvement makes the Gradle test tasks significantly faster since we
> do not have to start up a new JVM for every test class. It also made
> possible to introduce fine grained test categories which were essential to
> be able to parallelize the test execution in our CI systems. For more
> information please refer to COMPILING.txt
> <https://github.com/apache/sqoop/blob/trunk/COMPILING.txt>.
> *Apache Sqoop Jenkins job
> <https://builds.apache.org/job/Sqoop-hadoop200/> now builds and tests with
> Gradle*
> Since our Gradle build became much more stable and faster it made sense to
> reconfigure our Jenkins job to benefit from these improvements. The job is
> faster now (~30 minutes instead of ~40) and it executes all of the tests
> which can be run without external RDBMS or cloud systems (while the old Ant
> based job executed the unit test suite only).
> *Travis CI is enabled for Apache Sqoop*
> The new Travis CI job <https://travis-ci.org/apache/sqoop> now runs for
> every commit and every pull request on Apache Sqoop GitHub repository and
> it executes all of the tests except the Oracle third party test cases. One
> of the biggest benefit of Travis CI is that it can be really easily
> configured for the individual forks as well so contributors get a well
> configured CI job for their own feature branches for free. For more
> information please refer to COMPILING.txt
> <https://github.com/apache/sqoop/blob/trunk/COMPILING.txt>.
> Since we have a CI job now which integrates very well with GitHub pull
> requests I suggest deprecating the old Review Board and patch file based
> contribution process and use pull requests in the future. We had a mail
> chain about the same proposal last year and it seemed that the community
> was happy about the idea so I think we can evaluate it for some time and if
> everything goes well we can update our how to contribute wiki.
> Feel free to reply to this chain with your questions and suggestions on the
> above!
> Regards,
> Szabolcs

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message