samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@linkedin.com.INVALID>
Subject Re: Recommended JDK version?
Date Wed, 10 Dec 2014 20:43:07 GMT
Hey Jay,

Framing the problem strictly as developer productivity is not entirely
accurate. I agree with you from that standpoint. I'd never want to remove
JDK 6 support, since I'm growing to dislike all new things. :)

But the reality is that we're foregoing features that we could take
advantage of. For example, YARN 2.7 is slated to have disk I/O isolation
CGroup support. Without an upgrade in Samza, this can't be supported
(other than taking whatever the NM gives us by default). This is a
specific example, but more will pop up as time passes. Another would be
Scala 2.11. If we don't build against Scala 2.11, then you actually can't
write Samza StreamTasks against it, which some users have asked for.

So I think the real question is about timing and messaging (roadmap,
anyone?). People need to know if and when, so that they can make
decisions. We could push off Scala 2.11 and YARN 2.7 support for a year
from now (YARN 2.4 is already 8 months old, and 0.8 just shipped with it).
Personally, I'm OK waiting longer than six months. The thing is, I don't
think that there's going to be a real forcing function for it for several
years, BUT things will get steadily worse in the meantime.

The second note that I have is that I think the long term goal should
remain to deprecate and remove Scala from Samza's implementation, and in
general be extraordinarily cautious about which dependencies we pull into
Samza.

So, 1 year? I'm OK with that.

Cheers,
Chris

On 12/10/14 11:15 AM, "Jay Kreps" <jay.kreps@gmail.com> wrote:

>My two cents:
>
>I think this is usually a trade off of optimizing for the project
>developers (who ideally would want to use the newest thing) and the
>project
>users, where many will be stuck on old versions and can't use it at all if
>it doesn't work with their version. For the most part I think the real,
>tangible productivity gain for project developers on new versions is not
>that high but the gain for users stuck on old versions is really high
>(since they can't use it at all). Software that is heavily used (or that
>wants to be heavily used) should optimize for the users and take a pretty
>conservative stance on compatibility.
>
>-Jay
>
>On Wed, Dec 10, 2014 at 8:59 AM, Chris Riccomini <
>criccomini@linkedin.com.invalid> wrote:
>
>> Hey all,
>>
>> When Hadoop drops JDK6 support, we will be forced to upgrade as part of
>> our YARN support. If 0.9.0 uses Hadoop 2.7, then our next release will
>> require it. They seem to be releasing every 3-4 months, so it's likely
>> than in the Feb-Apr '15 time frame, we'll be forced to move to JDK 7
>> anyway.
>>
>> With this in mind, I think we should move forward with an end-of-March
>>JDK
>> 6 deprecation date. As Jakob mentioned in
>> https://issues.apache.org/jira/browse/SAMZA-455, I think we should also
>> put something up on the website to make it abundantly clear what's
>> happening. Does that sound workable?
>>
>> @Jakob/@Zhijie any idea what the schedule looks like for the Hadoop 2.7
>> release?
>>
>> @Eric sorry to hear versioning issues ended up being unworkable for you
>> with Samza. Out of curiosity, was it the JDK support, or the Scala
>> versioning that caused problems?
>>
>> Cheers,
>> Chris
>>
>> On 12/9/14 5:37 PM, "Paul Brown" <prb@mult.ifario.us> wrote:
>>
>> >JDK6 will have been EOL'd for *two years* come February next year[1].
>> >IMHO, the only reason to still build for older JDKs is as a convenient
>> >proxy for Android support.
>> >
>> >[1] http://www.oracle.com/technetwork/java/eol-135779.html
>> >
>> >‹
>> >prb@mult.ifario.us | Multifarious, Inc. | http://mult.ifario.us/
>> >
>> >On Tue, Dec 9, 2014 at 1:47 PM, Eric Sammer <esammer@scalingdata.com>
>> >wrote:
>> >
>> >> On Tue, Dec 9, 2014 at 1:04 PM, Jakob Homan <jghoman@gmail.com>
>>wrote:
>> >>
>> >> > Eric had been requesting the JDK6 support on Twitter
>> >> > (https://twitter.com/esammer/status/516681461219737600). In
>>response,
>> >> > SAMZA-455 was opened but not much lobbying was done there.
>> >> >
>> >> > @Eric, is it still the case that you feel JDK6 is a hard
>>requirement?
>> >> > You made a strong case in the JIRA.  I note that Hadoop is
>>currently
>> >> > going JDK7 in 2.7.
>> >> >
>> >>
>> >> Thanks for remembering. :)
>> >>
>> >> At the end of the day, there's some threshold where, if N% of
>>projects
>> >>drop
>> >> support, users are forced to upgrade. When they do, they tend to do
>> >> everything on a box (and its cluster) together. Mixed-mode
>>deployments
>> >> (e.g. Samza @ JDK7, Kafka @ 7, HDFS @ 6) is a recipe for disaster.
>>The
>> >> short way of saying that is minVersion(commonProjectsUsedTogether) is
>> >>the
>> >> ideal version to support. If Hadoop and others are dropping support,
>> >>it's
>> >> probably fine. I think the most important thing is that it's clearly
>> >> communicated prior to doing so (insert big discussion about
>>deprecation
>> >> cycles on "supported platforms"). We weren't able to use Samza as
>>part
>> >>of
>> >> our product as a result of minimum versions. Scala-based projects
>>have
>> >> historically been an enormous pain in this regard. I don't know how
>>many
>> >> others will be in that boat.
>> >>
>> >>
>> >> > Would it work for Samza 0.8 to be the last to provide JDK6 support?
>> >> > It's likely that Samza 0.9 won't be out for at least a few months,
>>and
>> >> > Eric had proposed a strawman approach of continuing JDK6 support
>>for
>> >> > six months (back in early November).  So, it's likely that 0.9
>>would
>> >> > reasonably closely meet that timeframe and could be the first
>> >> > JDK7-only release...
>> >> >
>> >>
>> >> I think that's probably fine. It sounds like 0.8 will have a good
>> >>lifecycle
>> >> prior to 0.9 taking over, giving folks enough runway to plan for a
>>JVM
>> >> upgrade. Like I said, when we evaluated Samza, we were blocked on the
>> >> dependency. With our timing, it forced us on to other projects, as
>>much
>> >>as
>> >> we really liked and wanted to use Samza. I think there's a big
>>divide in
>> >> terms of tolerance of supported platforms between building internal
>> >>systems
>> >> (i.e. SaaS, or in-house) and building "enterprise software" (i.e.
>> >>software
>> >> you ship to folks). I don't pretend our requirements are indicative
>>of
>> >>the
>> >> majority or important to everyone. I also respect the desire for
>>forward
>> >> motion in what's supported, and what features are accessible.
>> >>
>> >> The next discussion is probably around which version of Scala to
>>track
>> >>for
>> >> the Samza community over the next N months. There are some obvious
>> >> contentious positions[1] on Java 8 being required there as well.
>>That's
>> >>an
>> >> even tougher nut to crack. Some of the related projects still have
>>some
>> >> issues running on 8 (ZK, or at least a few months ago when I tried
>>it).
>> >>
>> >> [1] http://scala-lang.org/news/2.12-roadmap
>> >>
>> >> Thanks all!
>> >>
>> >>
>> >> > -jakob
>> >> >
>> >> > On Tue, Dec 9, 2014 at 8:39 AM, Chris Riccomini
>> >> > <criccomini@linkedin.com.invalid> wrote:
>> >> > > Hey all,
>> >> > >
>> >> > > We've reached a bit of an impasse between upgrading to Scala
>>2.11:
>> >> > >
>> >> > >   https://issues.apache.org/jira/browse/SAMZA-469
>> >> > >
>> >> > > And deprecating JDK 6:
>> >> > >
>> >> > >   https://issues.apache.org/jira/browse/SAMZA-455
>> >> > >
>> >> > > It looks as though Scalatra 2.3, which is required for Scala 2.11
>> >> > support,
>> >> > > was built using JDK 7. This means that upgrading to Scala 2.11
>> >>forces
>> >> us
>> >> > > to deprecate JDK 6. It is possible for us to work around this
by
>> >> > > eliminating the Scalatra dependency, but this would require some
>> >>work
>> >> in
>> >> > > samza-yarn.
>> >> > >
>> >> > > Thoughts?
>> >> > >
>> >> > > Cheers,
>> >> > > Chris
>> >> > >
>> >> > > On 11/4/14 6:58 AM, "Martin Kleppmann" <martin@kleppmann.com>
>> wrote:
>> >> > >
>> >> > >>Hi Tommy,
>> >> > >>
>> >> > >>There was a discussion about minimum JDK requirements a few
>>months
>> >>ago,
>> >> > >>and at the time, nobody was asking for JDK 6 support, so we
>>bumped
>> >>the
>> >> > >>requirement up to JDK 7. However, in the meantime, there have
>>been
>> >>some
>> >> > >>requests for JDK 6.
>> >> > >>
>> >> > >>I've tried to summarise the state of the discussion on this
>>ticket:
>> >> > >>https://issues.apache.org/jira/browse/SAMZA-455 -- please chime
>>in
>> >> > there.
>> >> > >>
>> >> > >>Thanks,
>> >> > >>Martin
>> >> > >>
>> >> > >>On 4 Nov 2014, at 13:05, Tommy Becker <tobecker@tivo.com>
wrote:
>> >> > >>
>> >> > >>> Hey folks,
>> >> > >>> I've noticed that the Samza jars from Maven are compiled
for
>>JDK
>> >>7.
>> >> I
>> >> > >>>don't see anything about a minimum JDK version on the site.
 We
>>are
>> >> > >>>currently still on JDK 6 and I'm trying to decide whether
we
>> >>should go
>> >> > >>>ahead and upgrade or whether we can recompile Samza for
JDK 6.
>> >>What
>> >> are
>> >> > >>>your thoughts?
>> >> > >>>
>> >> > >>> -Tommy
>> >> > >>>
>> >> > >>>
>> >> > >>> ________________________________
>> >> > >>>
>> >> > >>> This email and any attachments may contain confidential
and
>> >> privileged
>> >> > >>>material for the sole use of the intended recipient. Any
review,
>> >> > >>>copying, or distribution of this email (or any attachments)
by
>> >>others
>> >> is
>> >> > >>>prohibited. If you are not the intended recipient, please
>>contact
>> >>the
>> >> > >>>sender immediately and permanently delete this email and
any
>> >> > >>>attachments. No employee or agent of TiVo Inc. is authorized
to
>> >> conclude
>> >> > >>>any binding agreement on behalf of TiVo Inc. by email.
Binding
>> >> > >>>agreements with TiVo Inc. may only be made by a signed
written
>> >> > agreement.
>> >> > >>
>> >> > >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> E. Sammer
>> >> CTO - ScalingData
>> >>
>>
>>

Mime
View raw message