hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yair Even-Zohar" <ya...@audiencescience.com>
Subject map error (MultipleOutputs and firstbadlink related )
Date Fri, 06 Mar 2009 19:42:49 GMT
I'm running just the map part of a MR and I'm writing the output of that
map using MultipleOutputs.  That is, each job writes to X number of
files via MultipleOutputs.

 

The default is 2 map jobs and it works well for about X=30.

Now, when I increase the number of output files per job (from 30 to 60 )
I get IOExeption with "firstbadlink" and hbase/hadoop claim it has
problems reading data.
Not surprisingly, the same behavior happens if I force a split of the
data to 4 etc.
 
I have upped the "dfs.datanode.max.xcievers" to 1024 but that made no
change.
Any ideas?

 

Thanks

-Yair

 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message