metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Casey Stella <ceste...@gmail.com>
Subject Re: Question on Hdfs Writer
Date Mon, 28 Aug 2017 15:29:42 GMT
The storm metron-writer bolt has to write to many different sensor's HDFS
files.  The hdfs bolts provided by storm only support writing to one file
(or set of files).  The alternative would have been a bolt per sensor,
which did not support our architecture.

In short, the hdfs bolt from storm would have necessitated a bolt per
sensor and adding new sensors would have required topology restart and
recreation; this isn't something that we wanted.

On Mon, Aug 28, 2017 at 11:25 AM, Otto Fowler <ottobackwards@gmail.com>
wrote:

> What is the reasoning behind not using the Storm provided hdfs bolts for
> metron-writer?  Anyone remember?
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message