storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bobby Evans <bo...@apache.org>
Subject Re: [DRPC] Storm DRPC reliability
Date Thu, 08 Mar 2018 21:15:42 GMT
Yes there are lots of teams here at Yahoo using it in production.
https://github.com/yahoo/bullet-storm is one example that we open sourced
that uses DRPC.

There are no plans to remove it, but at the same time we are not planning
to turn it into a kafka competitor.  It has its place, but for a lot of use
cases a true pub/sub system might be a better choice.

As for documentation a lot of code gets written by people who don't pay
enough attention to the documentation.  So any help you can give to improve
it would be great.  It might be just asking questions, like this, or
actually fixing the docs yourself.

As for the broken example, that was me.  When I added in security I broke
backwards compatibility and didn't end up updating the examples.  Please
file a JIRA for that one too and assign it to me. I'll fix the examples and
do what I can to also restore backwards compatibility.

Thanks,

Bobby


On Thu, Mar 8, 2018 at 1:33 PM Łukasz Gebel <luk.gebel@gmail.com> wrote:

> Hi Bobby,
>
> thank you for a quick response. I'd rather use DRPC topology components
> (like DRPCSpout) and plug them into standard Topology (however I think
> deprecation of LinearDRPCTopologyBuider is quite misleading, so I'll create
> a ticket :)) .
> Let me ask some questions:
> - Do you know anyone used Storm DRPC in production use cases successfully?
> - Do you know if there are any plans to remove support for DRPC in future
> Storm releases (its code looks abandoned, last commit in 2016)?
> - Is there a chance that someone will document which properties can be
> defined in DRPCClient (currently example in Apache Storm documentation is
> not working, it takes 2 arguments, 3 are needed, additional one is config
> map).?
>
>  It would really help to get such information, as in current state DRPC
> functionality looks quite unreliable to me.
>
> Cheers,
> Luk.
>
> 2018-03-08 18:13 GMT+01:00 Bobby Evans <bobby@apache.org>:
>
>> Luk,
>>
>> DRPC itself is not deprecated, just the Linear DRPC builder.  In fact I
>> would argue that we should remove that deprecation.  It was deprecated
>> because.
>>
>> "Trident subsumes the functionality provided by this class, so it's
>> deprecated"
>>
>>
>> https://github.com/apache/storm/blob/ffa607e2464a361a8f2fa548cc8043f5a8818d04/storm-client/src/jvm/org/apache/storm/drpc/LinearDRPCTopologyBuilder.java#L53
>>
>> But not everyone wants to nor can use trident for their topologies and as
>> such it is still valid.
>>
>> If you want to file a JIRA at https://issues.apache.org/jira under the
>> STORM project to remove the deprecation and assign it to me (revans2) I
>> would be happy to take care of that for you.
>>
>> Thanks,
>>
>> Bobby
>>
>>
>> On Thu, Mar 8, 2018 at 9:18 AM Łukasz Gebel <luk.gebel@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> DRPC functionality in Apache Storm looks deprecated, however it's still
>>> useful in some use cases.
>>> Can anyone provide information how reliable is this functionality?
>>>
>>> My concerns: LinearDRPCTopologyBuilder is deprecated,
>>> documentation:
>>> http://storm.apache.org/releases/current/Distributed-RPC.html seems not
>>> to be updated (DRPC client init is outdated), there is no information which
>>> properties you should set in config (you have to dig in code) and so on.
>>>
>>> On the other hand it is possible to use components like DRPCSpout,
>>> ReturnResult (bolt) and plug them to TopologyBuilder and their work. And
>>> these components are not deprecated.
>>>
>>> Does anyone know if you can trust it, and if it will or won't be removed
>>> in newest versions of Storm? Did you have some unexpected problems using it?
>>>
>>> I really appreciate any help in this matter, as it is hard to find
>>> examples of successful usages of Storm DRPC in the Internet.
>>>
>>> Best regards,
>>> Luk.
>>>
>>
>

Mime
View raw message