metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From cestella <...@git.apache.org>
Subject [GitHub] incubator-metron issue #486: METRON-793: Migrate to storm-kafka-client kafka...
Date Wed, 22 Mar 2017 21:53:49 GMT
Github user cestella commented on the issue:

    https://github.com/apache/incubator-metron/pull/486
  
    @justinleet good points overall.  Yeah, I'm not thrilled with the HDP dependency either,
as I stated already.
    
    Regarding "If flux gets support for the Builder items, would we just axe those classes
entirely?"
    I believe that the extension will live on.  It has a couple of benefits beyond the builder
pattern in flux:
    * The parsers aren't using flux and do need a way to expose the configuration of the spout
via properties.  This is part of the extension here.
    * A component of extension in the storm-kafka-client Builder is via polymorphism (e.g.
the TupleBuilder, etc).  This will just never work well in flux, I think.
    * There is an assumption in the storm-kafka-client builder that we're passing in brokers
directly, rather than reading them from zookeeper.
    * This API is shifting dramatically.  It's totally different in storm 1.0.3 vs 1.0.1.
 Creating a layer here insulates us from API changes and localizes their impact.
    
    I made the TODOs because, while what we have here matched the capabilities of what we
had before, we could do better in supporting multiple topics.  I wanted to bring out the places
that would need to shift to support that.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message