storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Satish Duggana <satish.dugg...@gmail.com>
Subject Re: not all the servers in storm cluster processing messages
Date Thu, 09 Jun 2016 17:54:29 GMT
Hi Michael,
Kafka spout is given parallelism-hint as 1 and the topic has only one
partition. So, only one kafka spout task is run in one of the workers of
the topology. Are you asking that bolts connected directly/indirectly from
that kafka spout are all executed in the same worker as kafka spout? If so,
What are the groupings and parallellism-hint that you have configured on
those bolts?

Thanks,
Satish.


On Thu, Jun 9, 2016 at 10:41 PM, Kazansky, Michael <
michael.kazansky@jpmchase.com> wrote:

> Hi All,
>
> We have 3 servers in storm cluster. To prevent the order of the messages
> which come to Kafka the Kafka partition was set to 1. KafkaSpout
> parallelism hint is set to 1 as well. When messages flow through the Storm
> we noticed that only one server in the cluster is busy processing messages
> other 2 are idle although the number of workers  is set to 3 in Config. Is
> it excepted behavior or something is missing?
>
> If it is expected behavior is there any way to have all 3 servers in the
> cluster processing messages?
>
> Storm version is 0.9.5
>
> Thanks,
>
> Michael Kazansky
>
>
>
> This communication is for informational purposes only. It is not intended
> as an offer or solicitation for the purchase or sale of any financial
> instrument or as an official confirmation of any transaction. All market
> prices, data and other information are not warranted as to completeness or
> accuracy and are subject to change without notice. Any comments or
> statements made herein do not necessarily reflect those of JPMorgan Chase &
> Co., its subsidiaries and affiliates (collectively, "JPMC"). This
> transmission may contain information that is proprietary, privileged,
> confidential and/or exempt from disclosure under applicable law. If you are
> not the intended recipient, you are hereby notified that any disclosure,
> copying, distribution, or use of the information contained herein
> (including any reliance thereon) is STRICTLY PROHIBITED. If you received
> this transmission in error, please immediately contact the sender and
> destroy the material in its entirety, whether in electronic or hard copy
> format. Although this transmission and any attachments are believed to be
> free of any virus or other defect that might affect any computer system
> into which it is received and opened, it is the responsibility of the
> recipient to ensure that it is virus free and no responsibility is accepted
> by JPMC for any loss or damage arising in any way from its use. Please note
> that any electronic communication that is conducted within or through
> JPMC's systems is subject to interception, monitoring, review, retention
> and external production in accordance with JPMC's policy and local laws,
> rules and regulations; may be stored or otherwise processed in countries
> other than the country in which you are located; and will be treated in
> accordance with JPMC policies and applicable laws and regulations. Please
> refer to http://www.jpmorgan.com/pages/disclosures for disclosures
> relating to European legal entities.
>

Mime
View raw message