samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shanthoosh <>
Subject [GitHub] samza pull request #640: [WIP][Not ready for review]SAMZA-1885: Improve samz...
Date Sat, 15 Sep 2018 19:18:23 GMT
GitHub user shanthoosh reopened a pull request:

    [WIP][Not ready for review]SAMZA-1885: Improve samza build time.

    Samza codebase currently uses gradle 2.8 (which was released in 2014) and builds the codebase
serially executing each of the gradle tasks one at a time. Not taking advantage of recent
gradle features has caused slow builds (taking around 15 minutes to build samza codebase locally).
This  hampers the overall developer productivity within the team. 
    1. Upgrade to gradle version: 4.7(changes in, gradle-wrapper.jar).
    2. Enable parallel gradle builds.
    3. Set maximumParallelForks to 5(The maximum number of forked test processes to execute
in parallel. The default value is 1 (no parallel test execution). It cannot exceed the value
of max-workers for the current build.

    4. Fix the tests that cannot be run in parallel(TestsJobsResource, TestTaskResource).

    Reduces the build time by a factor of 2.
    Before these changes, it took around 13 minutes 40 seconds to build the open source samza
codebase locally.
    After these changes, it took around around 6 minutes 14 seconds to build samza codebase
    Ran the build locally multiple(50) times. After these changes, only the TestRunner API
tests fail consistently due to static partition state held in it(Will fix it as a part of
this patch).

You can merge this pull request into a Git repository by running:

    $ git pull improve_build_time

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #640
commit b48a116bb7a91beeac014ae30f963555332223ef
Author: Shanthoosh Venkataraman <svenkata@...>
Date:   2018-09-15T18:43:08Z

    1. Switch to newer version of gradle.
    2. Turn on gradle parallel builds.



View raw message