metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From cestella <>
Subject [GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint
Date Wed, 12 Jul 2017 08:21:41 GMT
Github user cestella commented on the issue:
    Looking at this implementation and working a bit on the PoC for index data mutation, I
think the abstraction here isn't in the right place.  It's too bound-up in the REST layer
whereas data access has traditionally existed further down in the abstraction layer.  It's
a smell that metron-rest has to depend on ES at all.  That dependency should be provided at
runtime entirely.  Also, we're going to need data access in batch as well and I'd like a sensible
abstraction in place to enable that (e.g. merging modified records when reading as part of
a Spark or MR job).
    It seems to me that we need a DAO abstraction living in `metron-writer` or even `metron-indexing`
for interacting with indices with the concrete implementations existing in `metron-elasticsearch`
and `metron-solr` for ES and Solr respectively.  I think `metron-rest` should work entirely
on abstract classes.  The REST layer should choose the DAO implementation via reflection and
we should adjust the classpath in the shell script to include one of `metron-elasticsearch`
or `metron-solr`.  In essence, I'm recommending something similar to how JDBC works where
you choose your driver implementation via reflection (often) and work with base JDBC classes
from there.

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

View raw message