chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <eric...@gmail.com>
Subject Re: do not write data to hbase
Date Wed, 27 Jun 2012 14:53:07 GMT
Hi Mason,

There is a bug in SeqFileWriter that it doesn't forward data to next writer.  It should be
the last writer in the pipeline stage for pipeline writer to work.

Sent from my iPhone

On Jun 27, 2012, at 1:51 AM, Dimboo Zhu <dianbo.zhu@gmail.com> wrote:

> Hi all, 
> 
> I configed chukwa to write data into hdfs and hbase using the following lines.
> 
> <property>
>     <name>chukwaCollector.pipeline</name>            
>      <value>
> org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter,
> org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter,
> org.apache.hadoop.chukwa.datacollection.writer.hbase.HBaseWriter
>      </value>
> </property>
> 
> But it looks like do nothing to hbase. 
> Anybody can help? Thanks,
> 
> Dianbau
> 
> 
> 2012-06-27 15:53:25,039 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 87% complete
> 2012-06-27 15:53:48,188 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
> 2012-06-27 15:53:48,192 [main] INFO  org.apache.pig.tools.pigstats.SimplePigStats - Script
Statistics:
> 
> HadoopVersion   PigVersion      UserId  StartedAt       FinishedAt      Features
> 1.0.3   0.10.0  akazam  2012-06-27 15:48:49     2012-06-27 15:53:48     GROUP_BY
> 
> Success!
> 
> Job Stats (time in seconds):
> JobId   Maps    Reduces MaxMapTime      MinMapTIme      AvgMapTime      MaxReduceTime
  MinReduceTime   AvgReduceTime   Alias   Feature Outputs
> job_201206211450_9575   1       0       6       6       6       0       0       0   
   CleanseBuffer,ConcatBuffer,HDFSMetrics  MAP_ONLY        ClusterSummary,
> job_201206211450_9576   1       1       12      12      12      21      21      21  
   CleanseBuffer,GroupBuffer,MapReduceMetrics,MapReduceSummary,TimeSeries  GROUP_BY,COMBINER
      ClusterSummary,
> job_201206211450_9578   1       1       18      18      18      12      12      12  
   CleanseBuffer,ComputeBuffer,ConcatBuffer,SystemMetrics,TimeSeries       GROUP_BY,COMBINER
      ClusterSummary,
> job_201206211450_9580   1       1       9       9       9       12      12      12  
   CleanseBuffer,GroupBuffer,MapReduceMetrics,MapReduceSummary,TimeSeries  GROUP_BY,COMBINER
      ClusterSummary,
> 
> Input(s):
> Successfully read 0 records (319 bytes) from: "hbase://Hadoop"
> Successfully read 0 records (326 bytes) from: "hbase://SystemMetrics"
> Successfully read 0 records (319 bytes) from: "hbase://Hadoop"
> Successfully read 0 records (319 bytes) from: "hbase://Hadoop"
> 
> Output(s):
> Successfully stored 0 records in: "ClusterSummary"
> Successfully stored 0 records in: "ClusterSummary"
> Successfully stored 0 records in: "ClusterSummary"
> Successfully stored 0 records in: "ClusterSummary"
> 
> Counters:
> Total records written : 0
> Total bytes written : 0
> Spillable Memory Manager spill count : 0
> Total bags proactively spilled: 0
> Total records proactively spilled: 0
> 
> Job DAG:
> job_201206211450_9580
> job_201206211450_9578
> job_201206211450_9576
> job_201206211450_9575
> 
> 

Mime
View raw message