samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shanthoosh <...@git.apache.org>
Subject [GitHub] samza pull request #640: [WIP][Not ready for review]SAMZA-1885: Improve samz...
Date Sat, 15 Sep 2018 04:19:40 GMT
GitHub user shanthoosh opened a pull request:

    https://github.com/apache/samza/pull/640

    [WIP][Not ready for review]SAMZA-1885: Improve samza build time.

    Samza codebase currently uses gradle 2.8 (which was released in 2014) and builds the codebase
serially executing each of the gradle tasks one at a time. Not taking advantage of recent
gradle features has caused slow builds (taking around 15 minutes to build samza codebase locally).
This  hampers the overall developer productivity within the team. Proposal is to have a fasten
builds by a series of changes.
    
    **Changes:**
    
    1. Upgrade to gradle version: 4.7(changes in gradle.properties, gradle-wrapper.jar).
    2. Enable parallel gradle builds.
    3. Set maximumParallelForks to 15(The maximum number of forked test processes to execute
in parallel. The default value is 1 (no parallel test execution). It cannot exceed the value
of max-workers for the current build.
    Reference: https://docs.gradle.org/current/dsl/org.gradle.api.tasks.testing.Test.html)

    4. Fix the tests that cannot be run in parallel(TestsJobsResource, TestTaskResource).

    
    **Result:** 
    
    Reduces the build time by a factor of 2.
    Before these changes, it took around 13 minutes 40 seconds to build the open source samza
codebase locally.
    After these changes, it took around around 6 minutes 14 seconds to build samza codebase
locally.
    
    **Pending:** 
    
    Ran the build locally multiple(50) times. After these changes, only the TestRunner API
tests fail consistently due to static state in it to maintain inemory(Will fix it in subsequent
patches).
    
    
    
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/shanthoosh/samza improve_build_time

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/samza/pull/640.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #640
    
----
commit 347f8fef986ffaf87b6275d72370aea0f6ac9daf
Author: Shanthoosh Venkataraman <spvenkat@...>
Date:   2018-09-14T17:43:20Z

    Changes:
    
    1. Switch to newer version of gradle.
    2. Turn on gradle parallel builds.

----


---

Mime
View raw message