metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Casey Stella <ceste...@gmail.com>
Subject Re: Elasticsearch Indexing Bolt NullPointerException
Date Wed, 08 Jun 2016 13:31:42 GMT
Hey Domenic,

Yep, it can be challenging to track down the docs here.  Let me try to run
down the options for you:

   - topology.workers : Under the config section of the topology
   - topology.acker.executers : Under the config section of the topology
   - parallelism : Under the individual bolts and spouts

Take for instance the following example:

> name: "enrichment"
> config:
>     topology.workers: 1
>     topology.acker.executors: 0
>
> bolts:
> # Enrichment Bolts
>     -   id: "enrichmentSplitBolt"
>         className: "org.apache.metron.enrichment.bolt.EnrichmentSplitterBolt"
>         constructorArgs:
>             - "${kafka.zk}"
>         configMethods:
>             -   name: "withEnrichments"
>                 args:
>                     - ref: "enrichments"
>         parallelism: 1
>
>

On Tue, Jun 7, 2016 at 11:29 AM, Puzio, Domenic <
Domenic.Puzio@capitalone.com> wrote:

> I have a related question. Is there any documentation on setting the Storm
> configurations like parallelism.hint and num.tasks for individual bolts in
> the Enrichment topology? I know this is done in Flux, but I am not sure
> what these configs will look like and where they need to go.
>
>
> Thank you!
>
> Domenic
>
> On 6/7/16, 11:09 AM, "Rider,Jonathan" <Jonathan.Rider@capitalone.com>
> wrote:
>
> >Hello,
> >
> >We’re currently testing all of our parsers on our QA environment and as
> we added more parsers, the enrichment topology started slowing down
> dramatically. Namely, the indexingBolt had a process latency of 12ms and
> now it’s 200ms, (Execute latency went from 2 to 25).
> >
> >Upon further inspection, the indexing bolt has lots and lots of
> NullPointerExceptions. The logs are less than helpful, as they look like
> this:
> >
> >
> >2016-06-07 11:03:29.360 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.360 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.360 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.360 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.361 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.361 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.361 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.361 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.361 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.361 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.361 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.362 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.362 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.362 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.362 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.362 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.362 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.363 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.363 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.363 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.363 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.363 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >
> >Has anyone seen this kind of error before and know what is causing it?
> >
> >Thanks,
> >
> >Jonathan Rider
> >________________________________________________________
> >
> >The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.
> ________________________________________________________
>
> The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message