samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shanthoosh <...@git.apache.org>
Subject [GitHub] samza pull request #640: [WIP][Not ready for review]SAMZA-1885: Improve samz...
Date Sat, 15 Sep 2018 19:18:23 GMT
GitHub user shanthoosh reopened a pull request:

    https://github.com/apache/samza/pull/640

    [WIP][Not ready for review]SAMZA-1885: Improve samza build time.

    Samza codebase currently uses gradle 2.8 (which was released in 2014) and builds the codebase
serially executing each of the gradle tasks one at a time. Not taking advantage of recent
gradle features has caused slow builds (taking around 15 minutes to build samza codebase locally).
This  hampers the overall developer productivity within the team. 
    
    **Changes:**
    
    1. Upgrade to gradle version: 4.7(changes in gradle.properties, gradle-wrapper.jar).
    2. Enable parallel gradle builds.
    3. Set maximumParallelForks to 5(The maximum number of forked test processes to execute
in parallel. The default value is 1 (no parallel test execution). It cannot exceed the value
of max-workers for the current build.
    Reference: https://docs.gradle.org/current/dsl/org.gradle.api.tasks.testing.Test.html)

    4. Fix the tests that cannot be run in parallel(TestsJobsResource, TestTaskResource).

    
    **Result:** 
    
    Reduces the build time by a factor of 2.
    Before these changes, it took around 13 minutes 40 seconds to build the open source samza
codebase locally.
    After these changes, it took around around 6 minutes 14 seconds to build samza codebase
locally.
    
    **Pending:** 
    
    Ran the build locally multiple(50) times. After these changes, only the TestRunner API
tests fail consistently due to static partition state held in it(Will fix it as a part of
this patch).
    
    
    
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/shanthoosh/samza improve_build_time

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/samza/pull/640.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #640
    
----
commit b48a116bb7a91beeac014ae30f963555332223ef
Author: Shanthoosh Venkataraman <svenkata@...>
Date:   2018-09-15T18:43:08Z

    Changes:
    
    1. Switch to newer version of gradle.
    2. Turn on gradle parallel builds.

----


---

Mime
View raw message