storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Annabel Melongo <melongo_anna...@yahoo.com>
Subject RE: Storm-HDFS bolt file paths according to input Tuples
Date Mon, 04 Jan 2016 23:31:02 GMT

    
A solution might be to save these file in a common storage place, folder, daravase.. and
then use said database as your input spout.


Sent from my Verizon Wireless 4G LTE smartphone

-------- Original message --------
From: Shivpriya Tamboskar <Shivpriya_Tamboskar@symantec.com> 
Date: 01/04/2016  16:40  (GMT-05:00) 
To: user@storm.apache.org 
Subject: Storm-HDFS bolt file paths according to input Tuples 

Hello,  I am trying to write data into HDFS using the storm-hdfs bolt http://storm.apache.org/documentation/storm-hdfs.html
The problem I am trying to solve is that I want the file paths to be set according to the
input tuples.

FileNameFormat fileNameFormat = new DefaultFileNameFormat()
        .withPath("/foo/“);
I am getting inputs from various users and what want to do is set the file paths according
to the username e.g.  "/user1/“ , “/user2/“
As far as I know once the path is set for the bolt it can’t be changed unless you have file
rotation policy set which is not what I want to do.
Has anyone else tried to solve this before ? Any suggestions would be appreciated.
Thanks Shivpriya 

Mime
View raw message