metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Casey Stella <ceste...@gmail.com>
Subject Re: [DISCUSS] Upgrading to Storm 1.0.x
Date Tue, 11 Oct 2016 19:07:19 GMT
I think this would be good (+1 for me), but we should be quite careful that
the testing for this PR is...exhaustive as it'll touch so much.
At the very least, we'll need to ensure that:

   - Ansible full-dev and quick-dev continue to work
   - Ambari install continues to function
   - Data flows through the topologies and into kibana
   - Pcap ingestion continues to function

Also, just to point out that with new versions come new dependencies (and
versions), so you may experience some classpath issues.

Casey

On Tue, Oct 11, 2016 at 10:35 AM, Carolyn Duby <cduby@hortonworks.com>
wrote:

> Storm 1.0 will allow config of kafka with SSL.  Should I comment on
> METRON-495 or maybe it is better as a separate lira?
>
> Thanks
> Carolyn
>
>
>
> On 10/11/16, 10:07 AM, "David Lyle" <dlyle65535@gmail.com> wrote:
>
> >I'm +1 on this.
> >
> >On Tue, Oct 11, 2016 at 9:54 AM, Justin Leet <justinjleet@gmail.com>
> wrote:
> >
> >> Hi all,
> >>
> >> I wanted to start a thread around upgrading our Storm version from
> 0.10.x
> >> to 1.0.x.  I created a Jira at
> >> https://issues.apache.org/jira/browse/METRON-495 to mirror this
> discussion
> >> and the results and opinions.
> >>
> >> As listed at https://storm.apache.org/2016/04/12/storm100-released.html
> ,
> >> there's a variety of improvements with Storm 1.0.  The obvious and
> likely
> >> most important improvement is to performance, but a variety of other
> >> improvements are noted on that link.
> >>
> >> There are several changes that have to occur in order to make this
> upgrade.
> >>
> >>
> >> As noted in http://storm.apache.org/releases/current/index.html,
> Storm's
> >> code packages moved from backtype.storm to org.apache.storm, meaning all
> >> topologies have to be recompiled if that change.  There is a runtime
> >> converter to run things in place with backtype.storm, but this doesn't
> >> appear to be enough for our case because a couple interfaces change from
> >> byte[] to ByteBuffer (somewhat, but not entirely, related to
> >> https://issues.apache.org/jira/browse/STORM-1449). Even without this
> >> issue,
> >> the long term solution is to use the new package naming.
> >>
> >> In addition, our dev instances right now spin up an HDP 2.4 instance,
> which
> >> matches our current version of Storm. HDP 2.5 uses Storm 1.0.1, so to
> >> migrate to Storm 1.0.x, I'd prefer to match that version, rather than
> going
> >> to 1.0.2.
> >>
> >> If there's anything I missed, or anything the group should be aware of
> in
> >> this transition (Anybody lived done this upgrade on another project and
> >> have input?), I'd love to hear it.
> >>
> >> Thanks,
> >> Justin
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message