chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From AD <straightfl...@gmail.com>
Subject piping data into Cassandra
Date Wed, 26 Oct 2011 01:34:16 GMT
hello,

 I currently push apache logs into Chukwa.  I am trying to figure out how to
get all those logs into Cassandra and run mapreduce there.  Is the best
place to do this in Demux (right my own version of TSProcessor?)

 Also the data flow seems to miss a step.  The page
http://incubator.apache.org/chukwa/docs/r0.4.0/dataflow.html says in 3.3
that

   - demux moves complete files to: dataSinkArchives/[yyyyMMdd]/*/*.done
 - the next step is to move files from
postProcess/demuxOutputDir_*/[clusterName]/[dataType]/[dataType]_[yyyyMMdd]_[HH].R.evt

  How do they get from dataSinkArchives to postProcess?  does this run
inside of DemuxManager or a separate process (bin/chukwa demux) ?

 Thanks
 AD

Mime
View raw message