hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stack <st...@duboce.net>
Subject Re: org.apache.hadoop.hbase.UnknownScannerException
Date Thu, 06 Nov 2008 14:01:25 GMT
The master output looks 'normal' to me.

The regionserver output is warnings about a misconfiguration.

I don't see any USE's in the logs.

I tested with 0.18.2 hadoop and it seemed fine.

You might try adding logging to your map and see what it outputs.  Could
also enable DEBUG level logging in hbase.

St.Ack


On Thu, Nov 6, 2008 at 1:20 AM, Edward J. Yoon <edwardyoon@apache.org>wrote:

> The below logs are while throws UnknownScannerException. I think it
> seems have a trouble with hadoop-0.18.2
>
> HMaster:
>
> 2008-11-06 18:14:18,236 INFO
> org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner
> scanning meta region {regionname: -ROOT-,,0, startKey: <>, server:
> 163.152.38.46:60020}
> 2008-11-06 18:14:18,270 INFO
> org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner
> scan of meta region {regionname: -ROOT-,,0, startKey: <>, server:
> 163.152.38.46:60020} complete
> 2008-11-06 18:14:21,312 INFO
> org.apache.hadoop.hbase.master.BaseScanner: RegionManager.metaScanner
> scanning meta region {regionname: .META.,,1, startKey: <>, server:
> 163.152.23.228:60020}
> 2008-11-06 18:14:21,340 INFO
> org.apache.hadoop.hbase.master.BaseScanner: RegionManager.metaScanner
> scan of meta region {regionname: .META.,,1, startKey: <>, server:
> 163.152.23.228:60020} complete
> 2008-11-06 18:14:21,340 INFO
> org.apache.hadoop.hbase.master.BaseScanner: all meta regions scanned
> 2008-11-06 18:15:18,236 INFO
> org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner
> scanning meta region {regionname: -ROOT-,,0, startKey: <>, server:
> 163.152.38.46:60020}
> 2008-11-06 18:15:18,277 INFO
> org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner
> scan of meta region {regionname: -ROOT-,,0, startKey: <>, server:
> 163.152.38.46:60020} complete
> 2008-11-06 18:15:21,312 INFO
> org.apache.hadoop.hbase.master.BaseScanner: RegionManager.metaScanner
> scanning meta region {regionname: .META.,,1, startKey: <>, server:
> 163.152.23.228:60020}
> 2008-11-06 18:15:21,336 INFO
> org.apache.hadoop.hbase.master.BaseScanner: RegionManager.metaScanner
> scan of meta region {regionname: .META.,,1, startKey: <>, server:
> 163.152.23.228:60020} complete
> 2008-11-06 18:15:21,336 INFO
> org.apache.hadoop.hbase.master.BaseScanner: all meta regions scanned
>
>
> HRegionServer:
>
> 2008-11-06 18:03:00,941 INFO org.apache.hadoop.mapred.TaskTracker:
> TaskTracker up at: localhost/127.0.0.1:58508
> 2008-11-06 18:03:00,941 INFO org.apache.hadoop.mapred.TaskTracker:
> Starting tracker tracker_cic.dkorea.ac.kr:localhost/127.0.0.1:58508
> 2008-11-06 18:03:00,968 INFO org.apache.hadoop.mapred.TaskTracker:
> Starting thread: Map-events fetcher for all reduce tasks on
> tracker_cic.dkorea.ac.kr:localhost/127.0.0.1:58508
> 2008-11-06 18:15:41,552 INFO org.apache.hadoop.mapred.TaskTracker:
> LaunchTaskAction: attempt_200811061802_0001_m_000000_1
> 2008-11-06 18:15:41,856 WARN org.apache.hadoop.fs.FileSystem:
> "vldb.korea.ac.kr:54310" is a deprecated filesystem name. Use
> "hdfs://vldb.korea.ac.kr:54310/" instead.
> 2008-11-06 18:15:42,445 WARN org.apache.hadoop.fs.FileSystem:
> "vldb.korea.ac.kr:54310" is a deprecated filesystem name. Use
> "hdfs://vldb.korea.ac.kr:54310/" instead.
> 2008-11-06 18:15:42,938 WARN org.apache.hadoop.fs.FileSystem:
> "vldb.korea.ac.kr:54310" is a deprecated filesystem name. Use
> "hdfs://vldb.korea.ac.kr:54310/" instead.
>
> /Edward
>
> On Thu, Nov 6, 2008 at 5:15 PM, stack <stack@duboce.net> wrote:
> > Looks like your map was fine but then it hung for three minutes: i.e. >
> > default scanner timout.
> >
> > 08/11/06 16:36:09 INFO mapred.JobClient:  map 50% reduce 16%
> > 08/11/06 16:39:01 INFO mapred.JobClient: Task Id :
> > attempt_200811061631_0001_m_000001_0, Status : FAILED
> > org.apache.hadoop.hbase.UnknownScannerException:
> > org.apache.hadoop.hbase.UnknownScannerException: Name:
> >
> >
> > Anything in your Map that could hang that long?  Can you add logging?
> >
> > After the 3 minute hang, if a Client goes back to the Server, it'll get
> the
> > USE.
> >
> > St.Ack
> >
> >
> >
> > On Wed, Nov 5, 2008 at 11:52 PM, Edward J. Yoon <edwardyoon@apache.org
> >wrote:
> >
> >> After upgrade hadoop to 0.18.2 from 0.18.1, I received an
> >> UnknownScannerException. And, Map in/output bytes are only 288 bytes.
> >>
> >> Does anyone know about this?
> >>
> >> ----
> >> hadoop@vldb ~/hama-trunk $ bin/hama examples add 2 2
> >> 08/11/06 16:35:34 INFO hama.AbstractMatrix: Initializing the matrix
> >> storage.
> >> 08/11/06 16:35:39 INFO hama.AbstractMatrix: Create Matrix
> >> DenseMatrix_randsmsif
> >> 08/11/06 16:35:39 INFO hama.AbstractMatrix: Create the 2 * 2 random
> >> matrix : DenseMatrix_randsmsif
> >> 08/11/06 16:35:39 INFO hama.AbstractMatrix: Initializing the matrix
> >> storage.
> >> 08/11/06 16:35:45 INFO hama.AbstractMatrix: Create Matrix
> >> DenseMatrix_randcnzdg
> >> 08/11/06 16:35:45 INFO hama.AbstractMatrix: Create the 2 * 2 random
> >> matrix : DenseMatrix_randcnzdg
> >> 08/11/06 16:35:46 INFO hama.AbstractMatrix: Initializing the matrix
> >> storage.
> >> 08/11/06 16:35:52 INFO hama.AbstractMatrix: Create Matrix
> >> DenseMatrix_randvhcsa
> >> 08/11/06 16:35:52 WARN mapred.JobClient: Use GenericOptionsParser for
> >> parsing the arguments. Applications should implement Tool for the
> >> same.
> >> 08/11/06 16:35:52 WARN mapred.JobClient: Use genericOptions for the
> >> option -libjars
> >> 08/11/06 16:35:54 WARN mapred.JobClient: No job jar file set.  User
> >> classes may not be found. See JobConf(Class) or
> >> JobConf#setJar(String).
> >> 08/11/06 16:35:54 INFO mapred.JobClient: Running job:
> job_200811061631_0001
> >> 08/11/06 16:35:55 INFO mapred.JobClient:  map 0% reduce 0%
> >> 08/11/06 16:35:59 INFO mapred.JobClient:  map 50% reduce 0%
> >> 08/11/06 16:36:09 INFO mapred.JobClient:  map 50% reduce 16%
> >> 08/11/06 16:39:01 INFO mapred.JobClient: Task Id :
> >> attempt_200811061631_0001_m_000001_0, Status : FAILED
> >> org.apache.hadoop.hbase.UnknownScannerException:
> >> org.apache.hadoop.hbase.UnknownScannerException: Name:
> >> -1687189961475873913
> >>        at
> >>
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1077)
> >>        at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> >>        at
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>        at java.lang.reflect.Method.invoke(Method.java:597)
> >>        at
> >> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:554)
> >>        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
> >>
> >>        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> >> Method)
> >>        at
> >>
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> >>        at
> >>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> >>        at
> java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> >>        at
> >>
> org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:82)
> >>        at
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:856)
> >>        at
> >>
> org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1155)
> >>        at
> >>
> org.apache.hama.mapred.VectorInputFormatBase$TableRecordReader.next(VectorInputFormatBase.java:174)
> >>        at
> >>
> org.apache.hama.mapred.VectorInputFormatBase$TableRecordReader.next(VectorInputFormatBase.java:56)
> >>        at
> >>
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:165)
> >>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:45)
> >>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> >>        at
> >> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
> >>
> >> 08/11/06 16:40:22 INFO mapred.JobClient: Task Id :
> >> attempt_200811061631_0001_m_000001_1, Status : FAILED
> >> org.apache.hadoop.hbase.UnknownScannerException:
> >> org.apache.hadoop.hbase.UnknownScannerException: Name:
> >> 745631933222911092
> >>        at
> >>
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1077)
> >>        at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> >>        at
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>        at java.lang.reflect.Method.invoke(Method.java:597)
> >>        at
> >> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:554)
> >>        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
> >>
> >>        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> >> Method)
> >>        at
> >>
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> >>        at
> >>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> >>        at
> java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> >>        at
> >>
> org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:82)
> >>        at
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:856)
> >>        at
> >>
> org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1155)
> >>        at
> >>
> org.apache.hama.mapred.VectorInputFormatBase$TableRecordReader.next(VectorInputFormatBase.java:174)
> >>        at
> >>
> org.apache.hama.mapred.VectorInputFormatBase$TableRecordReader.next(VectorInputFormatBase.java:56)
> >>        at
> >>
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:165)
> >>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:45)
> >>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> >>        at
> >> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
> >>
> >> 08/11/06 16:42:06 INFO mapred.JobClient: Task Id :
> >> attempt_200811061631_0001_m_000001_2, Status : FAILED
> >> org.apache.hadoop.hbase.UnknownScannerException:
> >> org.apache.hadoop.hbase.UnknownScannerException: Name:
> >> -8279337588883268282
> >>        at
> >>
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1077)
> >>        at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> >>        at
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>        at java.lang.reflect.Method.invoke(Method.java:597)
> >>        at
> >> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:554)
> >>        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
> >>
> >>        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> >> Method)
> >>        at
> >>
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> >>        at
> >>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> >>        at
> java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> >>        at
> >>
> org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:82)
> >>        at
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:856)
> >>        at
> >>
> org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1155)
> >>        at
> >>
> org.apache.hama.mapred.VectorInputFormatBase$TableRecordReader.next(VectorInputFormatBase.java:174)
> >>        at
> >>
> org.apache.hama.mapred.VectorInputFormatBase$TableRecordReader.next(VectorInputFormatBase.java:56)
> >>        at
> >>
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:165)
> >>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:45)
> >>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> >>        at
> >> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
> >>
> >> 08/11/06 16:42:08 INFO mapred.JobClient:  map 100% reduce 16%
> >> 08/11/06 16:42:17 INFO mapred.JobClient: Job complete:
> >> job_200811061631_0001
> >> 08/11/06 16:42:17 INFO mapred.JobClient: Counters: 13
> >> 08/11/06 16:42:17 INFO mapred.JobClient:   File Systems
> >> 08/11/06 16:42:17 INFO mapred.JobClient:     Local bytes read=310
> >> 08/11/06 16:42:17 INFO mapred.JobClient:     Local bytes written=706
> >> 08/11/06 16:42:17 INFO mapred.JobClient:   Job Counters
> >> 08/11/06 16:42:17 INFO mapred.JobClient:     Launched reduce tasks=1
> >> 08/11/06 16:42:17 INFO mapred.JobClient:     Launched map tasks=6
> >> 08/11/06 16:42:17 INFO mapred.JobClient:   Map-Reduce Framework
> >> 08/11/06 16:42:17 INFO mapred.JobClient:     Reduce input groups=2
> >> 08/11/06 16:42:17 INFO mapred.JobClient:     Combine output records=0
> >> 08/11/06 16:42:17 INFO mapred.JobClient:     Map input records=2
> >> 08/11/06 16:42:17 INFO mapred.JobClient:     Reduce output records=2
> >> 08/11/06 16:42:17 INFO mapred.JobClient:     Map output bytes=288
> >> 08/11/06 16:42:17 INFO mapred.JobClient:     Map input bytes=0
> >> 08/11/06 16:42:17 INFO mapred.JobClient:     Combine input records=0
> >> 08/11/06 16:42:17 INFO mapred.JobClient:     Map output records=4
> >> 08/11/06 16:42:17 INFO mapred.JobClient:     Reduce input records=4
> >> 08/11/06 16:42:21 INFO client.HBaseAdmin: Disabled DenseMatrix_randsmsif
> >> 08/11/06 16:42:21 INFO client.HBaseAdmin: Deleted DenseMatrix_randsmsif
> >> 08/11/06 16:42:28 INFO client.HBaseAdmin: Disabled DenseMatrix_randcnzdg
> >> 08/11/06 16:42:28 INFO client.HBaseAdmin: Deleted DenseMatrix_randcnzdg
> >> 08/11/06 16:42:34 INFO client.HBaseAdmin: Disabled DenseMatrix_randvhcsa
> >> 08/11/06 16:42:35 INFO client.HBaseAdmin: Deleted DenseMatrix_randvhcsa
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon @ NHN, corp.
> >> edwardyoon@apache.org
> >> http://blog.udanax.org
> >>
> >
>
>
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message