sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Szabolcs Vasas <va...@cloudera.com.INVALID>
Subject Sqoop build infrastructure improvements
Date Wed, 28 Nov 2018 15:53:19 GMT
Dear Sqoop community,

We have been working on quite a few exciting build infrastructure
improvements recently, I am sending this email to summarize them.

*Gradle can now execute all the Sqoop tests in a single JVM*
This improvement makes the Gradle test tasks significantly faster since we
do not have to start up a new JVM for every test class. It also made
possible to introduce fine grained test categories which were essential to
be able to parallelize the test execution in our CI systems. For more
information please refer to COMPILING.txt
<https://github.com/apache/sqoop/blob/trunk/COMPILING.txt>.

*Apache Sqoop Jenkins job
<https://builds.apache.org/job/Sqoop-hadoop200/> now builds and tests with
Gradle*
Since our Gradle build became much more stable and faster it made sense to
reconfigure our Jenkins job to benefit from these improvements. The job is
faster now (~30 minutes instead of ~40) and it executes all of the tests
which can be run without external RDBMS or cloud systems (while the old Ant
based job executed the unit test suite only).

*Travis CI is enabled for Apache Sqoop*
The new Travis CI job <https://travis-ci.org/apache/sqoop> now runs for
every commit and every pull request on Apache Sqoop GitHub repository and
it executes all of the tests except the Oracle third party test cases. One
of the biggest benefit of Travis CI is that it can be really easily
configured for the individual forks as well so contributors get a well
configured CI job for their own feature branches for free. For more
information please refer to COMPILING.txt
<https://github.com/apache/sqoop/blob/trunk/COMPILING.txt>.


Since we have a CI job now which integrates very well with GitHub pull
requests I suggest deprecating the old Review Board and patch file based
contribution process and use pull requests in the future. We had a mail
chain about the same proposal last year and it seemed that the community
was happy about the idea so I think we can evaluate it for some time and if
everything goes well we can update our how to contribute wiki.

Feel free to reply to this chain with your questions and suggestions on the
above!

Regards,
Szabolcs

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message