storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Poulosky <ppoul...@yahoo-inc.com>
Subject Re: open file in spout with python
Date Mon, 23 Jan 2017 15:03:24 GMT
It's hard to know how to debug this without instrumentation.  Do you know what's failing? 
Are you bundling the file you want to read with your topology, otherwise it will have be installed
on every node running a supervisor.
 

    On Saturday, January 21, 2017 10:41 PM, sam mohel <sammohel5@gmail.com> wrote:
 

 Is there any help ? 

On Saturday, January 21, 2017, sam mohel <sammohel5@gmail.com> wrote:

I'm new to python , i'm trying to make spout read from file line by line . i tried to write
this statments but didn't work 


import storm
import random
# Define some sentences

class SimSpout(storm.Spout):
    # Not much to do here for such a basic spout
    def initialize(self, conf, context):
    ## Open the file with read only permit
    f = open('mydata.txt', 'r')
        self._conf = conf
        self._context = context

        storm.logInfo("Spout instance starting...")

    # Process the next tuple
    def nextTuple(self):
        # check if it reach at the EOF to close it 
        line = f.readline()
        while line != "":
        # Emit a random sentence
        storm.logInfo("Emiting %s" % line)
        storm.emit([line])

# Start the spout when it's invoked
SimSpout().run()

Another thing i want to ask about , i have shell script that take my data text file to make
process on it to get the result , now if i want to make it in storm i should make spout open
my data text file as its my data source and read line by line to take every line it got it
to bolt that make a process on it which will contain the code in shell written in python ,
Right ? 



   
Mime
View raw message