samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From prateekm <...@git.apache.org>
Subject [GitHub] samza pull request #25: SAMZA-1054: Refactor Operator APIs
Date Mon, 28 Nov 2016 22:41:25 GMT
GitHub user prateekm opened a pull request:

    https://github.com/apache/samza/pull/25

    SAMZA-1054: Refactor Operator APIs

    Some suggestions for an Operator API refactor and misc. cleanup. It does contain some
implementation changes, mostly due to deleted, extracted or merged classes. (e.g. OperatorFactory
+ ChainedOperators == OperatorImpls).
    
    Since git marked several moved classes as (delete + new) instead, it's probably best to
apply the diff locally and  browse the code in an IDE.
    
    Some of the changes, in no particular order:
    * Extracted XFunction interfaces into a .functions package in -api.
    * -api's internal.Operators is now the -operators's spec.* package. Extracted interfaces
and classes. Factory methods are now in OperatorSpecs.
    * -api's MessageStreams is now -api's MessageStream interface and -operators's MessageStreamImpl.
    * -api's internal.Windows classes are now in -api's .window package. Extracted interfaces
and classes, but no implementation changes.
    * OperatorFactory + ChainedOperators is now OperatorImpls, which is used from StreamOperatorAdaptorTask.
    * Added a NoOpOperatorImpl, which acts as the root node for the OperatorImpl DAG returned
by OperatorImpls.
    * Removed usages of reactivestreams APIs since current code looks simpler without them.
We can add them back when we need features like backpressure etc.
    * Removed the InputSystemMessage interface.
    * Made field names consistent (e.g Fn suffix for functions everywhere etc.).
    * Some method/class visibility changes due to moved classes. Haven't done this for all
the classes yet.
    * General documentation changes, mostly to make public APIs clearer. Haven't done this
for all the classes yet.
    
    There are additional questions/tasks that we can address in future RBs:
    * Updating Window and Trigger APIs.
    * Merging samza-operator into samza-core.
    * Questions about Message timestamp and Offset comparison semantics.
    * Questions about OperatorSpec serialization (e.g. ID generation).
    * Questions about StateStoreImpl and StoreFunctions.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/prateekm/samza master

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/samza/pull/25.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #25
    
----
commit 25f2f54d8ce9a418629c71c872eddf50b01c5a03
Author: Prateek Maheshwari <pmaheshw@linkedin.com>
Date:   2016-11-28T22:31:12Z

    SAMZA-1054: Refactor Operator APIs
    
    Some suggestions for an Operator API refactor and misc. cleanup. It does contain some
implementation changes, mostly due to deleted, extracted or merged classes. (e.g. OperatorFactory
+ ChainedOperators == OperatorImpls).
    
    Since git marked several moved classes as (delete + new) instead, it's probably best to
apply the diff locally and  browse the code in an IDE.
    
    Some of the changes, in no particular order:
    * Extracted XFunction interfaces into a .functions package in -api.
    * -api's internal.Operators is now the -operators's spec.* package. Extracted interfaces
and classes. Factory methods are now in OperatorSpecs.
    * -api's MessageStreams is now -api's MessageStream interface and -operators's MessageStreamImpl.
    * -api's internal.Windows classes are now in -api's .window package. Extracted interfaces
and classes, but no implementation changes.
    * OperatorFactory + ChainedOperators is now OperatorImpls, which is used from StreamOperatorAdaptorTask.
    * Added a NoOpOperatorImpl, which acts as the root node for the OperatorImpl DAG returned
by OperatorImpls.
    * Removed usages of reactivestreams APIs since current code looks simpler without them.
We can add them back when we need features like backpressure etc.
    * Removed the InputSystemMessage interface.
    * Made field names consistent (e.g Fn suffix for functions everywhere etc.).
    * Some method/class visibility changes due to moved classes. Haven't done this for all
the classes yet.
    * General documentation changes, mostly to make public APIs clearer. Haven't done this
for all the classes yet.
    
    There are additional questions/tasks that we can address in future RBs:
    * Updating Window and Trigger APIs.
    * Merging samza-operator into samza-core.
    * Questions about Message timestamp and Offset comparison semantics.
    * Questions about OperatorSpec serialization (e.g. ID generation).
    * Questions about StateStoreImpl and StoreFunctions.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message