> On Jan. 4, 2016, 11:11 p.m., Yi Pan (Data Infrastructure) wrote:
> > samza-core/src/main/scala/org/apache/samza/coordinator/JobCoordinator.scala, line
204
> > <https://reviews.apache.org/r/41663/diff/2-3/?file=1181044#file1181044line204>
> >
> > I am a bit confused here. If the refreshJobModel() function actually returns
a different jobModel than what's already set in jobModelRef, isn't the whole point is to replace
it w/ the new one?
>
> Yi Pan (Data Infrastructure) wrote:
> Ah, I saw it now. Actually, I think that we should keep this line in initializaJobModel()
method, but remove the following lines in getCoordinator():
> {code}
> val jobModel: JobModel = jobModelGenerator()
> jobModelRef.set(jobModel)
> {code}
The thing I noticed is that refreshJobModel and initializeJobModel are both private. The only
time when the job model is getting refreshed is when getJobCoordinator is called. Now that
I think about it, after the change to the JobServlet, we don't really make a call to refresh
the job model. The only other place where we do a jobCoordinator.jobmodel() is in SamzaTaskManager.
AT this point, we use the jobModel simply to invoke the readContainerLocality method on it.
Technically, nothing refreshes the jobModel after initialization. Better discussed in person
:)
- Navina
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41663/#review112677
-----------------------------------------------------------
On Jan. 4, 2016, 10:40 p.m., Navina Ramesh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41663/
> -----------------------------------------------------------
>
> (Updated Jan. 4, 2016, 10:40 p.m.)
>
>
> Review request for samza, Boris Shkolnik, Yan Fang, Chris Riccomini, Jake Maes, Jagadish
Venkatraman, Xinyu Liu, and Yi Pan (Data Infrastructure).
>
>
> Bugs: SAMZA-843
> https://issues.apache.org/jira/browse/SAMZA-843
>
>
> Repository: samza
>
>
> Description
> -------
>
> SAMZA-843 : Slow start of Samza jobs with large number of containers
>
>
> Diffs
> -----
>
> samza-autoscaling/src/main/java/org/apache/samza/autoscaling/deployer/ConfigManager.java
87346bc9f3d92e3ae86ed264c82d12a4d27f3188
> samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala ddce1481cee41dfb997103dbe3d5df44a123564e
> samza-core/src/main/scala/org/apache/samza/coordinator/JobCoordinator.scala 112ec1c2935fb6c2ebe9b5d9090263cea94c0cf9
> samza-core/src/main/scala/org/apache/samza/coordinator/server/JobServlet.scala a3baddbe81c96f58e718e062e964485eaa2dc701
> samza-core/src/main/scala/org/apache/samza/util/Util.scala 58fbb8f8177a109d35659a29ef6660e239334de2
> samza-core/src/test/scala/org/apache/samza/container/TestSamzaContainer.scala 365ff0a8ece808a8143ee3580f1f42238dd292d1
> samza-core/src/test/scala/org/apache/samza/coordinator/TestJobCoordinator.scala 80cccf3104e6e0ce7457303f40baa2f4fa807782
>
> Diff: https://reviews.apache.org/r/41663/diff/
>
>
> Testing
> -------
>
> Added some additional unit tests
> ./gradlew clean build
>
>
> Thanks,
>
> Navina Ramesh
>
>
|