hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jianshi Huang <jianshi.hu...@gmail.com>
Subject Re: Completebulkload with namespace option?
Date Tue, 29 Jul 2014 06:52:28 GMT
I see why, looking at the source code of LoadIncrementalHFiles.java, it
seems the temporary path created for splitting will contain ':',

The error part should be this:
String uniqueName = getUniqueName(table.getName());
    HColumnDescriptor familyDesc =
table.getTableDescriptor().getFamily(item.family);
    Path botOut = new Path(tmpDir, uniqueName + ".bottom");
    Path topOut = new Path(tmpDir, uniqueName + ".top");
    splitStoreFile(getConf(), hfilePath, familyDesc, splitKey,
        botOut, topOut);

uniqueName will be "namespce:table" so new Path will fail.

A bug right?

Jianshi


On Tue, Jul 29, 2014 at 2:42 PM, Jianshi Huang <jianshi.huang@gmail.com>
wrote:

> I'm using hbase 0.98 with HDP 2.1.
>
>
> On Tue, Jul 29, 2014 at 2:39 PM, Jianshi Huang <jianshi.huang@gmail.com>
> wrote:
>
>> I'm using completebulkload to load 500GB of data to a table
>> (presplitted). However, it reports the following errors:
>>
>> Looks like completebulkload didn't recognize the namespace part
>> (namespace:table).
>>
>> Is there an option to do it? I can't find one in Google...
>>
>> Exception in thread "main" 14/07/28 23:32:19 INFO
>> mapreduce.LoadIncrementalHFiles: Trying to load
>> hfile=hdfs://xxx/vertices/PROP/f5cbf0965ff44cb8bdabd038e66485c3
>> first=dc595cfe#cust#1812199228741466242
>>  last=dc68cedc#cust#2251647837553603393
>> java.lang.reflect.InvocationTargetException
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>         at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:54)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>         at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>> Caused by: java.lang.IllegalStateException:
>> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
>> path in absolute URI: grapple:vertices,37.bottom
>>         at
>> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplitPhase(LoadIncrementalHFiles.java:421)
>>         at
>> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:291)
>>         at
>> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:825)
>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>         at
>> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.main(LoadIncrementalHFiles.java:831)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>         at
>> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
>>         at
>> org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:145)
>>         at
>> org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:153)
>>         ... 10 more
>> Caused by: java.lang.IllegalArgumentException:
>> java.net.URISyntaxException: Relative path in absolute URI:
>> grapple:vertices,37.bottom
>>         at org.apache.hadoop.fs.Path.initialize(Path.java:206)
>>         at org.apache.hadoop.fs.Path.<init>(Path.java:172)
>>         at org.apache.hadoop.fs.Path.<init>(Path.java:94)
>>         at
>> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.splitStoreFile(LoadIncrementalHFiles.java:450)
>>         at
>> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:516)
>>         at
>> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:400)
>>         at
>> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:398)
>>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>         at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>         at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>         at java.lang.Thread.run(Thread.java:724)
>> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
>> grapple:vertices,37.bottom
>>         at java.net.URI.checkPath(URI.java:1804)
>>         at java.net.URI.<init>(URI.java:752)
>>         at org.apache.hadoop.fs.Path.initialize(Path.java:203)
>>         ... 10 more
>>
>> --
>> Jianshi Huang
>>
>> LinkedIn: jianshi
>> Twitter: @jshuang
>> Github & Blog: http://huangjs.github.com/
>>
>
>
>
> --
> Jianshi Huang
>
> LinkedIn: jianshi
> Twitter: @jshuang
> Github & Blog: http://huangjs.github.com/
>



-- 
Jianshi Huang

LinkedIn: jianshi
Twitter: @jshuang
Github & Blog: http://huangjs.github.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message