samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan <danharve...@gmail.com>
Subject Re: Extra Systems and other extensions.
Date Fri, 17 Apr 2015 13:16:30 GMT
https://issues.apache.org/jira/browse/SAMZA-654 - Elasticsearch Producer

Please comment there if you have any opinions on requirements for the
clients / configuration / serialization with using this producer. It's
currently fit for our use case but I know there will be more.

https://issues.apache.org/jira/browse/SAMZA-655 - Env variable rewriter

Again any comments on the format of the variables names welcome!

 - Dan


On 15 April 2015 at 17:38, Chinmay Soman <chinmay.cerebro@gmail.com> wrote:

> +1 ! I was going to do this for my use case as well. Would love to have
> this !
>
> On Wed, Apr 15, 2015 at 9:24 AM, Roger Hoover <roger.hoover@gmail.com>
> wrote:
>
> > Dan,
> >
> > This is great.  Would love to have a common ElasticSearch system
> producer.
> >
> > Cheers,
> >
> > Roger
> >
> > On Tue, Apr 14, 2015 at 1:34 PM, Dan <danharvey42@gmail.com> wrote:
> >
> > > Thanks Jakob, I agree they'll be more maintained and tested if they're
> in
> > > the main repo so that's great.
> > >
> > > I'll sort out Jira's and get some patches of what we've got working now
> > out
> > > for review.
> > >
> > >  - Dan
> > >
> > >
> > > On 14 April 2015 at 19:56, Jakob Homan <jghoman@gmail.com> wrote:
> > >
> > > > Hey Dan-
> > > > I'd love for the Elastic Search stuff to be added to the main code,
> as
> > > > a separate module.  Keeping these in the main source code keeps them
> > > > more likely to be maintained and correct.
> > > >
> > > > The EvironemtnConfigRewriter can likely go in the same place as the
> > > > ConfigRewriter interface, since it doesn't depend on Kafka as the
> > > > current RegexTopicRewriter does.
> > > >
> > > > If you could open JIRAs for these, that would be great.  Happy to
> > > > shepard the code in.
> > > > -Jakob
> > > >
> > > > On 14 April 2015 at 11:46, Dan <danharvey42@gmail.com> wrote:
> > > > > Hey,
> > > > >
> > > > > At state.com we've started to write some generic extensions to
> Samza
> > > > that
> > > > > we think would be more generally useful. We've got
> > > > > a ElasticsearchSystemProducer/Factory to output to an Elasticsearch
> > > index
> > > > > and EnvironmentConfigRewriter to modify config from environment
> > > variable.
> > > > >
> > > > > What's the best way for us to add things like this? Do you want
> more
> > > > > modules in the main project or should we just create some separate
> > > > projects
> > > > > on github?
> > > > >
> > > > > It would be good to get core extensions like these shared to be
> > tested
> > > > and
> > > > > used by more people
> > > > >
> > > > > Thanks,
> > > > > Dan
> > > >
> > >
> >
>
>
>
> --
> Thanks and regards
>
> Chinmay Soman
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message