samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nickpan47 <...@git.apache.org>
Subject [GitHub] samza pull request #642: Samza 1814: consolidate JobNode and JobGraph config...
Date Mon, 17 Sep 2018 10:02:54 GMT
GitHub user nickpan47 opened a pull request:

    https://github.com/apache/samza/pull/642

    Samza 1814: consolidate JobNode and JobGraph configuration generation for high and low-level
API applications

    High-level changes:
    - Move configuration generation to JobNodeConfigurationGenerator
    - Move the intermediate partition calculation to IntermediationStreamPartitionPlanner
    - Consolidate the code in JobPlanner and ExecutionPlanner for high and low-level API plan/configuration
generation

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/nickpan47/samza SAMZA-1814

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/samza/pull/642.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #642
    
----
commit 8e6fc2dac5af11f0748fdcd32175c2ae95418fc7
Author: prateekm <prateekm@...>
Date:   2018-06-09T22:01:10Z

    Remove all usages of StreamSpec and ApplicationRunner from the operator spec and impl
layers.

commit 7d7aa50882d656143ea6fbc8064aca390024dd31
Author: Prateek Maheshwari <pmaheshwari@...>
Date:   2018-06-13T19:56:51Z

    Updated with Cameron and Daniel's feedback.

commit 8561930133a0da52061bceee3596d02c6c7f9370
Author: Prateek Maheshwari <prateekm@...>
Date:   2018-06-15T22:25:50Z

    Merge branch 'master' into stream-spec-cleanup

commit 7e71dc7e04c5e1a0357663a76068d3dead4ebfd8
Author: Yi Pan (Data Infrastructure) <yipan@...>
Date:   2018-06-27T22:22:17Z

    Merge with master

commit f1cb8f0eb29feefe7e01c4a159f8792b5df0ce39
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-07-10T17:55:20Z

    Merge branch 'master' into single-app-api-May-21-18

commit d43e923192b8b78a9eb6ba4ba15a05151ec457f2
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-07-11T04:30:12Z

    WIP: proto-type with ApplicationRunnable and no ApplicationRunner exposed to user

commit 42782d815133ee75757c831f60a67f6980069e18
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-07-17T21:54:06Z

    Merge branch 'prateek-remove-app-runner-stream-spec' into app-spec-with-app-runtime-Jul-16-18

commit 95577b74c7330f11ee055c363b944c745f034ee0
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-07-18T10:09:52Z

    WIP: trying to figure out the two interface classes for spec: a) spec builder in init();
b) spec reader in all other lifecycle methods

commit 30a4e5f0ae6ff1b99fc8a6d08e742d8bff06e0cb
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-07-23T09:02:06Z

    WIP: application runner refactor - proto-type for SEP-13

commit fb1bc49e04c5a420c22c3589c39eb08a2964d01d
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-07-30T20:55:59Z

    Merge branch 'master' into app-spec-with-app-runtime-Jul-16-18

commit 973eb5261432da50a3540c96a101136f9c854f1d
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-03T19:08:30Z

    WIP: compiles, still working on LocalContainerRunner refactor

commit f20cdcda68ebf96924ece4ad16fbe25a9b370323
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-06T08:32:03Z

    WIP: adding unit tests. Pending update on StreamProcessorLifecycleListener, LocalContainerRunner,
and SamzaContainerListener

commit c4bb0dce6f812d19a5dcf02f19b7c2e455120bfd
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-06T17:46:11Z

    Merge branch 'master' into app-runtime-with-processor-callbacks

commit a82708bb0f509affba19a81a0b2a10b2d8191df8
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-13T06:52:45Z

    SAMZA-1789: unify ApplicationDescriptor and ApplicationRunner for high- and low-level
APIs in YARN and standalone environment

commit 014337170811527f7d3dcbe5af9e6ec353dc3efd
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-13T07:48:21Z

    Merge branch 'master' into app-runtime-with-processor-callbacks

commit db96da830a11d001eec7c068bf8a7d03a9a11799
Author: Yi Pan (Data Infrastructure) <yipan@...>
Date:   2018-08-17T00:15:48Z

    SAMZA-1789: WIP - revision to address review feedbacks.

commit 3b2f04d5434fba4a3793f85e7e57d09802d5d627
Author: Yi Pan (Data Infrastructure) <yipan@...>
Date:   2018-08-19T16:18:23Z

    SAMZA-1789: moved all impl classes from samza-api to samza-core.

commit 4382d45dbdb2a62070d424661f2459e3d2af9942
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-19T16:51:40Z

    Merge branch 'master' into app-runtime-with-processor-callbacks

commit 6e446fe6d201ab8b1e30ceeeb546a990ae2b77b2
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-23T01:36:33Z

    SAMZA-1789: address Cameron's review comments.

commit 1621c4d00a185cf0905a7500420955cefa56938c
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-23T04:55:01Z

    SAMZA-1789: a few more fixes to address Cameron's reviews

commit 247dcff4cdbf3c417290742d632001898524a8fd
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-23T05:16:14Z

    Merge branch 'master' into app-runtime-with-processor-callbacks

commit 055bd91e475770a785b8a1e1d4aedb188561b5d2
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-23T05:41:06Z

    SAMZA-1789: fix unit test with ThreadJobFactory

commit 8d4d3ffda2650fa41757d8ded94a7fb0d878feb0
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-24T20:37:39Z

    Merge with master

commit e7af6932d520168dfcda3b06acb7e587f2e5f8a2
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-24T21:10:26Z

    Merge branch 'master' into app-runtime-with-processor-callbacks

commit a072118d09ea4050589b75483366cea2c5d6fa0d
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-27T04:44:23Z

    Merge branch 'master' into app-runtime-with-processor-callbacks

commit 12c09af06430a9b59412f369a4e86a70ee686418
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-27T06:09:11Z

    SAMZA-1789: Fix a merging error (with SAMZA-1813)

commit 33753f72d14b6fa8a413b15bee0062d003f27059
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-29T02:03:11Z

    Merge branch 'master' into app-runtime-with-processor-callbacks

commit f04404cc2e13e2b41c0ddd656fca7bc4fec721d2
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-29T03:51:20Z

    SAMZA-1789: move createStreams out of the loop in prepareJobs

commit f2969f8dfa1bbe19bf823836daa03c55cb064051
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-31T01:56:17Z

    SAMZA-1789: fixed ApplicationDescriptor to use InputDescriptor and OutputDescriptor; addressed
Prateek's comments.

commit f4b3d43a478ab9300d4b3e15257288323ff62348
Author: Yi Pan (Data Infrastructure) <nickpan47@...>
Date:   2018-08-31T06:05:11Z

    SAMZA-1789: Fxing TaskApplication examples and some checkstyle errors

----


---

Mime
View raw message