samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yi Pan \(Data Infrastructure\)" <yi...@linkedin.com>
Subject Re: Review Request 44604: split deployment logic
Date Fri, 25 Mar 2016 17:14:31 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44604/#review125429
-----------------------------------------------------------



Overall lgtm. I have a few comments below. Thanks a lot!


samza-core/src/main/scala/org/apache/samza/job/local/ProcessJobFactory.scala (line 17)
<https://reviews.apache.org/r/44604/#comment188223>

    One question on this: how about ThreadJobFactory? Is it the current limitation that split
deployment only works for ProcessJobFactory and YarnJobFactory? Is it possible to set fwkPath
for ThreadJobFactory as well? If yes, I would recommend to open a JIRA to track that, and
document it.



samza-yarn/src/main/java/org/apache/samza/job/yarn/ContainerUtil.java (line 109)
<https://reviews.apache.org/r/44604/#comment188225>

    This segment of code is almost dup from the ProcessJobFactory. Can we fold this in the
config.get()? Or some utility class?


- Yi Pan (Data Infrastructure)


On March 15, 2016, 5:33 p.m., Boris Shkolnik wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44604/
> -----------------------------------------------------------
> 
> (Updated March 15, 2016, 5:33 p.m.)
> 
> 
> Review request for samza and Yi Pan (Data Infrastructure).
> 
> 
> Bugs: SAMZA-849
>     https://issues.apache.org/jira/browse/SAMZA-849
> 
> 
> Repository: samza
> 
> 
> Description
> -------
> 
> samza split deployment. See SAMZA-849 for details and design.
> 
> 
> Diffs
> -----
> 
>   samza-api/src/main/java/org/apache/samza/job/CommandBuilder.java 97bb05afe7ccd17a9a2443cd1cdf96dd8d1cc740

>   samza-core/src/main/scala/org/apache/samza/config/JobConfig.scala 4f3e9a297ce2c0df0f5f25e0aad62f7bed774cd6

>   samza-core/src/main/scala/org/apache/samza/job/ShellCommandBuilder.scala c3f60acf9ca88326ef7f2cfbbff2289f70cdd46e

>   samza-core/src/main/scala/org/apache/samza/job/local/ProcessJobFactory.scala 0792a59cb7b042220c8dbfc0713c5ef42e93ab25

>   samza-core/src/test/scala/org/apache/samza/job/TestShellCommandBuilder.scala fd014b326c8cca914513fa3ac9d913ea78deca23

>   samza-shell/src/main/bash/run-class.sh 46603c50f1fe02758aae5a568f06b1a65dff9734 
>   samza-shell/src/main/bash/run-container.sh 069279c88c88bd7d8b4a37510b38094446bb034b

>   samza-yarn/src/main/java/org/apache/samza/job/yarn/ContainerUtil.java 6580b9ab2cb293c98f31fee2766db7471e5d99ee

>   samza-yarn/src/main/scala/org/apache/samza/job/yarn/YarnJob.scala 1aa26bba00d2c3d42ef3b8951fa723ffff14d410

>   samza-yarn/src/test/java/org/apache/samza/job/yarn/TestSamzaTaskManager.java 9da1edf6ff165ef0306de8730853ad30551a9831

> 
> Diff: https://reviews.apache.org/r/44604/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Boris Shkolnik
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message