storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniela S" <daniela_4...@gmx.at>
Subject Aw: Re: Pull data from Redis every minute
Date Thu, 02 Jun 2016 08:45:23 GMT
<html><head></head><body><div style="font-family: Verdana;font-size:
12.0px;"><div>
<div>Thank you! But how can I ensure that my spout pulls the actual values from Redis
every minute from scratch? How does the spout work in such a case? Would it only pull values
if there is a change in Redis or will they only be pulled once?</div>

<div>&nbsp;</div>

<div>Would it be easier to do the lookup within the windowed bolt or with the tick tuple
in a bolt instead of the spout?</div>

<div>&nbsp;</div>

<div>Thank you and regards,</div>

<div>Daniela</div>

<div>&nbsp;
<div name="quote" style="margin:10px 5px 5px 10px; padding: 10px 0 10px 10px; border-left:2px
solid #C3D9E5; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div style="margin:0 0 10px 0;"><b>Gesendet:</b>&nbsp;Sonntag, 08. Mai
2016 um 19:44 Uhr<br/>
<b>Von:</b>&nbsp;&quot;Spico Florin&quot; &lt;spicoflorin@gmail.com&gt;<br/>
<b>An:</b>&nbsp;&quot;user@storm.apache.org&quot; &lt;user@storm.apache.org&gt;<br/>
<b>Betreff:</b>&nbsp;Re: Pull data from Redis every minute</div>

<div name="quoted-content">hi!
<div>&nbsp;for me it looks like proceesing the data in a specific window. you coul
achive this by using the new feature in storm 1.0 namely window bolt. via spout you get the
data thta you need and in the window bolt do the sum. be careful with the time thta you are
using processing time versus event time. if your data has a timestsmp and you would like to
use this field in your logic time then you need the event time and the notion of watermark.
you coul find more info about this on&nbsp;</div>

<div><a href="https://community.hortonworks.com/articles/14171/windowing-and-state-checkpointing-in-apache-storm.html"
target="_blank">https://community.hortonworks.com/articles/14171/windowing-and-state-checkpointing-in-apache-storm.html</a></div>

<div>i hope it helps.</div>

<div>florin</div>

<div><br/>
On Saturday, May 7, 2016, Erik Weathers &lt;<a href="eweathers@groupon.com" target="_parent">eweathers@groupon.com</a>&gt;
wrote:

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204)
solid;padding-left: 1.0ex;">
<div>If you need something to be done at a regular interval within Storm you might wanna
leverage the &quot;tick tuple&quot; concept:
<div>
<ul>
	<li><a href="http://kitmenke.com/blog/2014/08/04/tick-tuples-within-storm/" target="_blank">http://kitmenke.com/blog/2014/08/04/tick-tuples-within-storm/</a></li>
</ul>
</div>
</div>

<div class="gmail_extra">&nbsp;
<div class="gmail_quote">On Fri, May 6, 2016 at 11:53 PM, Daniela S <span>&lt;<a>daniela_4444@gmx.at</a>&gt;</span>
wrote:

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204)
solid;padding-left: 1.0ex;">
<div>
<div style="font-family: Verdana;font-size: 12.0px;">
<div>Hi,</div>

<div>&nbsp;</div>

<div>I would like to use Redis as a kind of cache for my active devices. So i receive
a message over Kafka with an active device and store it in&nbsp;Redis.&nbsp;</div>

<div>Now I would like to pull all active devices from Redis and to add a specific value
to each device and to build a sum afterwards.</div>

<div>How can I ensure that the spout fetches the list of active devices e.g. every minute?
Because the sum has to be built every minute from scratch, as the additional specific value&nbsp;changes
every minute for every device.</div>

<div>&nbsp;</div>

<div>Thank you in advance.</div>

<div>&nbsp;</div>

<div>Regards,</div>

<div>Daniela</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div></div></body></html>

Mime
View raw message