hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohit Kelkar <rohitkel...@gmail.com>
Subject Re: Scanner problem after bulk load hfile
Date Mon, 15 Jul 2013 04:27:09 GMT
Thanks Amit, I am also using 0.94.2 . I am also pre-splitting and I tried
the table.clearRegionCache() but still doesn't work.

- R


On Sun, Jul 14, 2013 at 3:45 AM, Amit Sela <amits@infolinks.com> wrote:

> If new regions are created during the bulk load (are you pre-splitting ?),
> maybe try myTable.clearRegionCache() after the bulk load (or even after the
> pre-splitting if you do pre-split).
> This should clear the region cache. I needed to use this because I am
> pre-splitting my tables for bulk load.
> BTW I'm using HBase 0.94.2
> Good luck!
>
>
> On Fri, Jul 12, 2013 at 6:50 PM, Rohit Kelkar <rohitkelkar@gmail.com>
> wrote:
>
> > I am having problems while scanning a table created using HFile.
> > This is what I am doing -
> > Once Hfile is created I use following code to bulk load
> >
> > LoadIncrementalHFiles loadTool = new LoadIncrementalHFiles(conf);
> > HTable myTable = new HTable(conf, mytablename.getBytes());
> > loadTool.doBulkLoad(new Path(outputHFileBaseDir + "/" + mytablename),
> > mytableTable);
> >
> > Then scan the table using-
> >
> > HTable table = new HTable(conf, mytable);
> > Scan scan = new Scan();
> > scan.addColumn("cf".getBytes(), "q".getBytes());
> > ResultScanner scanner = table.getScanner(scan);
> > for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> > numRowsScanned += 1;
> > }
> >
> > This code crashes with following error - http://pastebin.com/SeKAeAST
> > If I remove the scan.addColumn from the code then the code works.
> >
> > Similarly on the hbase shell -
> > - A simple count 'mytable' in hbase shell gives the correct count.
> > - A scan 'mytable' gives correct results.
> > - get 'mytable', 'myrow', 'cf:q' crashes
> >
> > The hadoop dfs -ls /hbase/mytable shows the .tableinfo, .tmp, the
> directory
> > for region etc.
> >
> > Now if I do a major_compact 'mytable' and then execute my code with the
> > scan.addColumn statement then it works. Also the get 'mytable', 'myrow',
> > 'cf:q' works.
> >
> > My question is
> > What is major_compact doing to enable the scanner that the
> > LoadIncrementalFiles tool is not? I am sure I am missing a step after the
> > LoadIncrementalFiles.
> >
> > - R
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message