storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rui Abreu <rui.ab...@gmail.com>
Subject Re: Scheme in Spouts
Date Thu, 01 Oct 2020 14:17:58 GMT
Hi Paul,

You have a *public T setRecordTranslator(RecordTranslator<K,V>
translator)  *in
https://storm.apache.org/releases/current/javadocs/org/apache/storm/kafka/spout/KafkaSpoutConfig.Builder.html,
which you should be using to create your KafkaSpoutConfig objects.

On Wed, 30 Sep 2020 at 12:20, Paul Jose <paul.j2@ugamsolutions.com> wrote:

>
>
> Hi,
>
> Regarding this same thread, where is the scheme supposed to be passed to
> the kafka consumer config?
> Previously this was a property of the spoutconfig. However, the property
> is not present right now.
> The ConsumerConfig.java class doesn't have any property called scheme.
> Where is the SchemeAsMultiScheme class supposed to be passed to?
>
> Best Regards,
>
> *Paul Jose*
>
> Software Engineer, Tech
>
> Ph: +91 8758608538
>
>
> *www.ugamsolutions.com <http://www.ugamsolutions.com/>*
> ------------------------------
> *From:* Paul Jose <paul.j2@ugamsolutions.com>
> *Sent:* 25 September 2020 13:20
> *To:* user@storm.apache.org <user@storm.apache.org>
> *Subject:* Re: Scheme in Spouts
>
> Hi Rui,
>
> Thank you so much. Don't know how I overlooked that. This works well.
> Thanks again.
>
> Best Regards,
>
> *Paul Jose*
> ------------------------------
> *From:* Rui Abreu <rui.abreu@gmail.com>
> *Sent:* 24 September 2020 16:59
> *To:* user@storm.apache.org <user@storm.apache.org>
> *Subject:* Re: Scheme in Spouts
>
> Hi Paul,
>
> The class still exists under a new package:
>
>
> https://storm.apache.org/releases/current/javadocs/org/apache/storm/spout/SchemeAsMultiScheme.html
>
> On Thu, 24 Sep 2020 at 12:14, Paul Jose <paul.j2@ugamsolutions.com> wrote:
>
>
>
> Hi,
>
> My name is Paul. I am tasked with upgrading a storm topology running on
> the 0.10.0 version to 2.2.0.
> Flux was used for the configuration.
> This is the configuration used for the spout in 0.10.0.
>
> components:
>
>   - className: com.platform.metric.spouts.SimpleJSONScheme
>     id: jsonScheme
>
>   - className: backtype.storm.spout.SchemeAsMultiScheme
>     constructorArgs:
>       -
>         ref: jsonScheme
>     id: stringMultiScheme
>
>   - className: storm.kafka.ZkHosts
>     constructorArgs:
>       - "${kafka.zk.host}"
>     id: zkHosts
>
>   - className: storm.kafka.SpoutConfig
>     constructorArgs:
>       -
>         ref: zkHosts
>       - "${kafka.topic}"
>       - ""
>       - stormKafka1
>     id: kafkaSpoutConfig
>     properties:
>       -
>         name: ignoreZkOffsets
>         value: false
>       -
>         name: "startOffsetTime"
>         value: -1
>       -
>         name: scheme
>         ref: stringMultiScheme
>       -
>         name: fetchSizeBytes
>         value: 8388608
>
> I've updated most of the topology to 2.2.0 but I can't seem to find any
> mention of schemes in 2.2.0.
> Can somebody point me in the right direction on how to upgrade. Do I have
> to use the Deserialiser class and apply the implmentation there and then
> pass it as teh value for the "value.deserialiser" property of the spout?
>
> Any direction or suggestion would be really helpful.
>
> Best Regards,
>
> *Paul Jose*
> ---------------------------------------------------------------------------------------Disclaimer----------------------------------------------------------------------------------------------
>
> ****Views and opinions expressed in this e-mail belong to their author and
> do not necessarily represent views and opinions of Ugam. Our employees are
> obliged not to make any defamatory statement or infringe any legal right.
> Therefore, Ugam does not accept any responsibility or liability for such
> statements. The content of this email is confidential and intended for the
> recipient specified in message only. It is strictly forbidden to share any
> part of this message with any third party, without a written consent of the
> sender. If you have received this message by mistake, please reply to this
> message and follow with its deletion, so that we can ensure such a mistake
> does not occur in the future. Warning: Sufficient measures have been taken
> to scan any presence of viruses however the recipient should check this
> email and any attachments for the presence of viruses as full security of
> the email cannot be ensured despite our best efforts. Therefore, Ugam
> accepts no liability for any damage inflicted by viewing the content of
> this email.. ****
> Please do not print this email unless it is necessary. Every unprinted
> email helps the environment.
>
> ---------------------------------------------------------------------------------------Disclaimer----------------------------------------------------------------------------------------------
>
> ****Views and opinions expressed in this e-mail belong to their author and
> do not necessarily represent views and opinions of Ugam. Our employees are
> obliged not to make any defamatory statement or infringe any legal right.
> Therefore, Ugam does not accept any responsibility or liability for such
> statements. The content of this email is confidential and intended for the
> recipient specified in message only. It is strictly forbidden to share any
> part of this message with any third party, without a written consent of the
> sender. If you have received this message by mistake, please reply to this
> message and follow with its deletion, so that we can ensure such a mistake
> does not occur in the future. Warning: Sufficient measures have been taken
> to scan any presence of viruses however the recipient should check this
> email and any attachments for the presence of viruses as full security of
> the email cannot be ensured despite our best efforts. Therefore, Ugam
> accepts no liability for any damage inflicted by viewing the content of
> this email.. ****
> Please do not print this email unless it is necessary. Every unprinted
> email helps the environment.
>
>

Mime
View raw message