samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Milinda Pathirage" <mili...@apache.org>
Subject Re: Review Request 33749: WIP: SAMZA-650 window store implementation
Date Thu, 07 May 2015 14:22:19 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33749/#review82821
-----------------------------------------------------------



samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoiner.java
<https://reviews.apache.org/r/33749/#comment133651>

    I think that stream-to-stream joining is not practical for possibly infinite streams.
We may need to define some constraints or some other restrictions to make this practical.


- Milinda Pathirage


On May 4, 2015, 6:58 a.m., Yi Pan (Data Infrastructure) wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33749/
> -----------------------------------------------------------
> 
> (Updated May 4, 2015, 6:58 a.m.)
> 
> 
> Review request for samza, Chris Riccomini, Guozhang Wang, Milinda Pathirage, Navina Ramesh,
and Naveen Somasundaram.
> 
> 
> Bugs: SAMZA-650
>     https://issues.apache.org/jira/browse/SAMZA-650
> 
> 
> Repository: samza
> 
> 
> Description
> -------
> 
> WIP: SAMZA-650 window store implementation
> 
> First patch to implemement window store and message store.
> 
> - Added window store initial implementation
> - Added MessageStore initial implementation as well
> - Completed skeleton APIs in window operators to illustrate the use cases of window store
and message store
> - Added a StreamTask test case w/ a single window operator usage
> - Unit tests are still WIP and will be updated later
> 
> 
> Diffs
> -----
> 
>   samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Relation.java PRE-CREATION

>   samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Stream.java PRE-CREATION

>   samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Table.java PRE-CREATION

>   samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Tuple.java PRE-CREATION

>   samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/Operator.java PRE-CREATION

>   samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SqlOperatorFactory.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/spec/OperatorSpec.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/data/IncomingMessageTuple.java PRE-CREATION

>   samza-sql-core/src/main/java/org/apache/samza/sql/exception/OperatorException.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperator.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorFactoryImpl.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorSpec.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoinSpec.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoiner.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionSpec.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/operators/relation/JoinSpec.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/operators/stream/InsertStreamSpec.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/BoundedTimeWindow.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/FullStateTimeWindow.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/FullStateTimeWindowAutoOp.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/FullStateTimeWindowOp.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/FullStateWindowOp.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowOp.java PRE-CREATION

>   samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowOpSpec.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowSpec.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowState.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/FilteredMessageIterator.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/HashPrefixedMessageStore.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/MessageStore.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/MessageStoreSpec.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/OffsetKey.java PRE-CREATION

>   samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/OrderedStoreKey.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/PrefixedKey.java PRE-CREATION

>   samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/Range.java PRE-CREATION

>   samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/TimeAndOffsetKey.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/TimeKey.java PRE-CREATION

>   samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/WindowOutputStream.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/WindowState.java PRE-CREATION

>   samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/WindowStore.java PRE-CREATION

>   samza-sql-core/src/main/java/org/apache/samza/system/sql/LongOffset.java PRE-CREATION

>   samza-sql-core/src/main/java/org/apache/samza/system/sql/Offset.java PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/task/sql/OperatorMessageCollector.java
PRE-CREATION 
>   samza-sql-core/src/main/java/org/apache/samza/task/sql/StoreMessageCollector.java PRE-CREATION

>   samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomOperatorTask.java PRE-CREATION

>   samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomWindowOperatorTask.java
PRE-CREATION 
>   samza-sql-core/src/test/java/org/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION

> 
> Diff: https://reviews.apache.org/r/33749/diff/
> 
> 
> Testing
> -------
> 
> ./gradlew clean :samza-sql-core:build
> 
> 
> Thanks,
> 
> Yi Pan (Data Infrastructure)
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message