hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <bus...@apache.org>
Subject Re: Hbase hbase-2.0.1, region server does not start on Hadoop 3.1
Date Mon, 02 Jul 2018 13:27:23 GMT
Hi Mich,

Please check out the section of our reference guide on Hadoop versions:

http://hbase.apache.org/book.html#hadoop

the short version is that there is not yet a Hadoop 3 version that the
HBase community considers appropriate for running HBase. if you'd like
to get into details and work arounds, please join the dev@hbase
mailing list and bring it up there.

Ted, please stop suggesting folks on the user list use anything other
than PMC sanctioned releases of HBase.

On Sun, Jul 1, 2018 at 1:09 AM, Mich Talebzadeh
<mich.talebzadeh@gmail.com> wrote:
> Hi,
>
> What is the ETA with version of Hbase that will work with Hadoop 3.1 and
> may not require HA setup for HDFS?
>
> Thanks
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Sun, 1 Jul 2018 at 00:26, Mich Talebzadeh <mich.talebzadeh@gmail.com>
> wrote:
>
>> Thanks Ted.
>>
>> Went back to hbase-1.2.6 that works OK with Hadoop 3.1
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
>> loss, damage or destruction of data or any other property which may arise
>> from relying on this email's technical content is explicitly disclaimed.
>> The author will in no case be liable for any monetary damages arising from
>> such loss, damage or destruction.
>>
>>
>>
>>
>> On Sun, 1 Jul 2018 at 00:15, Ted Yu <yuzhihong@gmail.com> wrote:
>>
>>> Have you tried setting the value for the config to filesystem ?
>>>
>>> Cheers
>>>
>>> On Sat, Jun 30, 2018 at 4:07 PM, Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com>
>>> wrote:
>>>
>>> > One way would be to set WAL outside of Hadoop environment. Will that
>>> work?
>>> >
>>> > The following did not work
>>> >
>>> > <property>
>>> >   <name>hbase.wal.provider</name>
>>> >   <value>multiwal</value>
>>> > </property>
>>> >
>>> >
>>> > Dr Mich Talebzadeh
>>> >
>>> >
>>> >
>>> > LinkedIn * https://www.linkedin.com/profile/view?id=
>>> > AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>> > <
>>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCd
>>> > OABUrV8Pw>*
>>> >
>>> >
>>> >
>>> > http://talebzadehmich.wordpress.com
>>> >
>>> >
>>> > *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any
>>> > loss, damage or destruction of data or any other property which may
>>> arise
>>> > from relying on this email's technical content is explicitly disclaimed.
>>> > The author will in no case be liable for any monetary damages arising
>>> from
>>> > such loss, damage or destruction.
>>> >
>>> >
>>> >
>>> >
>>> > On Sat, 30 Jun 2018 at 23:36, Ted Yu <yuzhihong@gmail.com> wrote:
>>> >
>>> > > Please read :
>>> > >
>>> > > http://hbase.apache.org/book.html#wal.providers
>>> > >
>>> > > On Sat, Jun 30, 2018 at 3:31 PM, Mich Talebzadeh <
>>> > > mich.talebzadeh@gmail.com>
>>> > > wrote:
>>> > >
>>> > > > Thanks
>>> > > >
>>> > > > In your point below
>>> > > >
>>> > > > …. or you can change default WAL to FSHLog.
>>> > > >
>>> > > > is there any configuration parameter to allow me to do so in
>>> > > > hbase-site.xml?
>>> > > >
>>> > > > Dr Mich Talebzadeh
>>> > > >
>>> > > >
>>> > > >
>>> > > > LinkedIn * https://www.linkedin.com/profile/view?id=
>>> > > > AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>> > > > <https://www.linkedin.com/profile/view?id=
>>> > AAEAAAAWh2gBxianrbJd6zP6AcPCCd
>>> > > > OABUrV8Pw>*
>>> > > >
>>> > > >
>>> > > >
>>> > > > http://talebzadehmich.wordpress.com
>>> > > >
>>> > > >
>>> > > > *Disclaimer:* Use it at your own risk. Any and all responsibility
>>> for
>>> > any
>>> > > > loss, damage or destruction of data or any other property which
may
>>> > arise
>>> > > > from relying on this email's technical content is explicitly
>>> > disclaimed.
>>> > > > The author will in no case be liable for any monetary damages
>>> arising
>>> > > from
>>> > > > such loss, damage or destruction.
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > > On Sat, 30 Jun 2018 at 23:25, Ted Yu <yuzhihong@gmail.com>
wrote:
>>> > > >
>>> > > > > Do you plan to deploy onto hadoop 3.1.x ?
>>> > > > >
>>> > > > > If so, you'd better build against hadoop 3.1.x yourself.
>>> > > > > You can either patch in HBASE-20244 and use asyncfswal.
>>> > > > > Or you can change default WAL to FSHLog.
>>> > > > >
>>> > > > > If you don't have to deploy onto hadoop 3.1.x, you can use
hbase
>>> > 2.0.1
>>> > > > >
>>> > > > > FYI
>>> > > > >
>>> > > > > On Sat, Jun 30, 2018 at 3:21 PM, Mich Talebzadeh <
>>> > > > > mich.talebzadeh@gmail.com>
>>> > > > > wrote:
>>> > > > >
>>> > > > > > so what options do I have her?. Is there any conf parameter
I
>>> can
>>> > set
>>> > > > in
>>> > > > > > hbase-site,xml to make this work? or shall I go back
to a more
>>> > stable
>>> > > > > > version of Hbase?
>>> > > > > >
>>> > > > > > cheers
>>> > > > > >
>>> > > > > > Dr Mich Talebzadeh
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > > LinkedIn * https://www.linkedin.com/profile/view?id=
>>> > > > > > AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>> > > > > > <https://www.linkedin.com/profile/view?id=
>>> > > > AAEAAAAWh2gBxianrbJd6zP6AcPCCd
>>> > > > > > OABUrV8Pw>*
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > > http://talebzadehmich.wordpress.com
>>> > > > > >
>>> > > > > >
>>> > > > > > *Disclaimer:* Use it at your own risk. Any and all
>>> responsibility
>>> > for
>>> > > > any
>>> > > > > > loss, damage or destruction of data or any other property
which
>>> may
>>> > > > arise
>>> > > > > > from relying on this email's technical content is explicitly
>>> > > > disclaimed.
>>> > > > > > The author will in no case be liable for any monetary
damages
>>> > arising
>>> > > > > from
>>> > > > > > such loss, damage or destruction.
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > > On Sat, 30 Jun 2018 at 23:15, Ted Yu <yuzhihong@gmail.com>
>>> wrote:
>>> > > > > >
>>> > > > > > > trunk version would correspond to hbase 3.0 which
has lot more
>>> > > > changes
>>> > > > > > > compared to hbase 2.
>>> > > > > > > The trunk build wouldn't serve you if your goal
is to run
>>> hbase
>>> > on
>>> > > > > hadoop
>>> > > > > > > 3.1 (see HBASE-20244)
>>> > > > > > >
>>> > > > > > > FYI
>>> > > > > > >
>>> > > > > > > On Sat, Jun 30, 2018 at 3:11 PM, Mich Talebzadeh
<
>>> > > > > > > mich.talebzadeh@gmail.com>
>>> > > > > > > wrote:
>>> > > > > > >
>>> > > > > > > > Thanks Ted.
>>> > > > > > > >
>>> > > > > > > > I downloaded the latest Hbase binary which
is 2.0.1
>>> 2018/06/19
>>> > > > > > > >
>>> > > > > > > > Is there any trunc version build for Hadoop
3.1 please and
>>> if
>>> > so
>>> > > > > where
>>> > > > > > > can
>>> > > > > > > > I download it?
>>> > > > > > > >
>>> > > > > > > > Regards,
>>> > > > > > > >
>>> > > > > > > > Dr Mich Talebzadeh
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > LinkedIn * https://www.linkedin.com/profile/view?id=
>>> > > > > > > > AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>> > > > > > > > <https://www.linkedin.com/profile/view?id=
>>> > > > > > AAEAAAAWh2gBxianrbJd6zP6AcPCCd
>>> > > > > > > > OABUrV8Pw>*
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > http://talebzadehmich.wordpress.com
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > *Disclaimer:* Use it at your own risk. Any
and all
>>> > responsibility
>>> > > > for
>>> > > > > > any
>>> > > > > > > > loss, damage or destruction of data or any
other property
>>> which
>>> > > may
>>> > > > > > arise
>>> > > > > > > > from relying on this email's technical content
is explicitly
>>> > > > > > disclaimed.
>>> > > > > > > > The author will in no case be liable for any
monetary
>>> damages
>>> > > > arising
>>> > > > > > > from
>>> > > > > > > > such loss, damage or destruction.
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > On Sat, 30 Jun 2018 at 22:52, Ted Yu <yuzhihong@gmail.com>
>>> > > wrote:
>>> > > > > > > >
>>> > > > > > > > > Which hadoop release was the 2.0.1 built
against ?
>>> > > > > > > > >
>>> > > > > > > > > In order to build hbase 2 against hadoop
3.0.1+ / 3.1.0+,
>>> you
>>> > > > will
>>> > > > > > need
>>> > > > > > > > > HBASE-20244.
>>> > > > > > > > >
>>> > > > > > > > > FYI
>>> > > > > > > > >
>>> > > > > > > > > On Sat, Jun 30, 2018 at 2:34 PM, Mich
Talebzadeh <
>>> > > > > > > > > mich.talebzadeh@gmail.com>
>>> > > > > > > > > wrote:
>>> > > > > > > > >
>>> > > > > > > > > > I am using the following hbase-site.xml
>>> > > > > > > > > >
>>> > > > > > > > > > <configuration>
>>> > > > > > > > > >   <property>
>>> > > > > > > > > >     <name>hbase.rootdir</name>
>>> > > > > > > > > >     <value>hdfs://rhes75:9000/hbase</value>
>>> > > > > > > > > >   </property>
>>> > > > > > > > > >   <property>
>>> > > > > > > > > >     <name>hbase.zookeeper.property.dataDir</name>
>>> > > > > > > > > >     <value>/home/hduser/zookeeper-3.4.6</value>
>>> > > > > > > > > >   </property>
>>> > > > > > > > > > <property>
>>> > > > > > > > > >     <name>hbase.master</name>
>>> > > > > > > > > >     <value>localhost:60000</value>
>>> > > > > > > > > > </property>
>>> > > > > > > > > > <property>
>>> > > > > > > > > >       <name>hbase.zookeeper.property.clientPort</name>
>>> > > > > > > > > >       <value>2181</value>
>>> > > > > > > > > >    </property>
>>> > > > > > > > > >   <property>
>>> > > > > > > > > >     <name>hbase.cluster.distributed</name>
>>> > > > > > > > > >     <value>true</value>
>>> > > > > > > > > >   </property>
>>> > > > > > > > > > <property>
>>> > > > > > > > > >      <name>hbase.defaults.for.version.skip</name>
>>> > > > > > > > > >      <value>true</value>
>>> > > > > > > > > > </property>
>>> > > > > > > > > > <property>
>>> > > > > > > > > >      <name>phoenix.query.dateFormatTimeZone</name>
>>> > > > > > > > > >      <value>Europe/London</value>
>>> > > > > > > > > > </property>
>>> > > > > > > > > > <property>
>>> > > > > > > > > >     <name>hbase.procedure.store.wal.use.hsync</name>
>>> > > > > > > > > >     <value>false</value>
>>> > > > > > > > > > </property>`
>>> > > > > > > > > > <property>
>>> > > > > > > > > >   <name>hbase.unsafe.stream.capability.enforce</name>
>>> > > > > > > > > >   <value>false</value>
>>> > > > > > > > > > </property>
>>> > > > > > > > > > </configuration>
>>> > > > > > > > > >
>>> > > > > > > > > > master starts OK but region server
throws some errors
>>> > > > > > > > > >
>>> > > > > > > > > > 2018-06-30 22:23:56,607 INFO
>>> [regionserver/rhes75:16020]
>>> > > > > > > > > > wal.AbstractFSWAL: WAL configuration:
blocksize=256 MB,
>>> > > > > > rollsize=128
>>> > > > > > > > MB,
>>> > > > > > > > > > prefix=rhes75%2C16020%2C1530393832024,
suffix=,
>>> > > > > > > > > > logDir=hdfs://rhes75:9000/hbase/WALs/rhes75,16020,153
>>> > > > > > > > > > 0393832024, archiveDir=hdfs://rhes75:9000/hbase/oldWALs
>>> > > > > > > > > > 2018-06-30 22:23:56,629 ERROR
>>> [regionserver/rhes75:16020]
>>> > > > > > > > > > regionserver.HRegionServer: ason:
>>> > > > > > > > > >     Type 'org/apache/hadoop/fs/ContentSummary'
(current
>>> > > frame,
>>> > > > > > > > stack[1])
>>> > > > > > > > > > is
>>> > > > > > > > > > not assignable to 'org/apache/hadoop/fs/QuotaUsage'
>>> > > > > > > > > >   Current Frame:
>>> > > > > > > > > >     bci: @105
>>> > > > > > > > > >     flags: { }
>>> > > > > > > > > >     locals: { 'org/apache/hadoop/fs/ContentSummary',
>>> > > > > > > > > > 'org/apache/hadoop/hdfs/protocol/proto/HdfsProtos$
>>> > > > > > > > > > ContentSummaryProto$Builder'
>>> > > > > > > > > > }
>>> > > > > > > > > >     stack: {
>>> > > > > > > > > > 'org/apache/hadoop/hdfs/protocol/proto/HdfsProtos$
>>> > > > > > > > > > ContentSummaryProto$Builder',
>>> > > > > > > > > > 'org/apache/hadoop/fs/ContentSummary'
}
>>> > > > > > > > > >   Bytecode:
>>> > > > > > > > > >     0x0000000: 2ac7 0005 01b0 b805
984c 2b2a b605 99b6
>>> > > > > > > > > >     0x0000010: 059a 2ab6 059b b605
9c2a b605 9db6 059e
>>> > > > > > > > > >     0x0000020: 2ab6 059f b605 a02a
b605 a1b6 05a2 2ab6
>>> > > > > > > > > >     0x0000030: 05a3 b605 a42a b605
a5b6 05a6 2ab6 05a7
>>> > > > > > > > > >     0x0000040: b605 a82a b605 a9b6
05aa 2ab6 05ab b605
>>> > > > > > > > > >     0x0000050: ac2a b605 adb6 05ae
572a b605 af9a 000a
>>> > > > > > > > > >     0x0000060: 2ab6 05b0 9900 0c2b
2ab8 0410 b605 b157
>>> > > > > > > > > >     0x0000070: 2bb6 05b2 b0
>>> > > > > > > > > >   Stackmap Table:
>>> > > > > > > > > >     same_frame(@6)
>>> > > > > > > > > >     append_frame(@103,Object[#2940])
>>> > > > > > > > > >     same_frame(@112)
>>> > > > > > > > > >  *****
>>> > > > > > > > > > java.lang.VerifyError: Bad type
on operand stack
>>> > > > > > > > > > Exception Details:
>>> > > > > > > > > >   Location:
>>> > > > > > > > > >
>>> > > > > > > > > > org/apache/hadoop/hdfs/protocolPB/PBHelperClient.
>>> > > > > > > > > >
>>> convert(Lorg/apache/hadoop/fs/ContentSummary;)Lorg/apache/
>>> > > > > > > > > >
>>> hadoop/hdfs/protocol/proto/HdfsProtos$ContentSummaryProto;
>>> > > > > > > > > > @105: invokestatic
>>> > > > > > > > > >   Reason:
>>> > > > > > > > > >     Type 'org/apache/hadoop/fs/ContentSummary'
(current
>>> > > frame,
>>> > > > > > > > stack[1])
>>> > > > > > > > > > is
>>> > > > > > > > > > not assignable to 'org/apache/hadoop/fs/QuotaUsage'
>>> > > > > > > > > >   Current Frame:
>>> > > > > > > > > >     bci: @105
>>> > > > > > > > > >     flags: { }
>>> > > > > > > > > >     locals: { 'org/apache/hadoop/fs/ContentSummary',
>>> > > > > > > > > > 'org/apache/hadoop/hdfs/protocol/proto/HdfsProtos$
>>> > > > > > > > > > ContentSummaryProto$Builder'
>>> > > > > > > > > > }
>>> > > > > > > > > >     stack: {
>>> > > > > > > > > > 'org/apache/hadoop/hdfs/protocol/proto/HdfsProtos$
>>> > > > > > > > > > ContentSummaryProto$Builder',
>>> > > > > > > > > > 'org/apache/hadoop/fs/ContentSummary'
}
>>> > > > > > > > > >   Bytecode:
>>> > > > > > > > > >     0x0000000: 2ac7 0005 01b0 b805
984c 2b2a b605 99b6
>>> > > > > > > > > >     0x0000010: 059a 2ab6 059b b605
9c2a b605 9db6 059e
>>> > > > > > > > > >     0x0000020: 2ab6 059f b605 a02a
b605 a1b6 05a2 2ab6
>>> > > > > > > > > >     0x0000030: 05a3 b605 a42a b605
a5b6 05a6 2ab6 05a7
>>> > > > > > > > > >     0x0000040: b605 a82a b605 a9b6
05aa 2ab6 05ab b605
>>> > > > > > > > > >     0x0000050: ac2a b605 adb6 05ae
572a b605 af9a 000a
>>> > > > > > > > > >     0x0000060: 2ab6 05b0 9900 0c2b
2ab8 0410 b605 b157
>>> > > > > > > > > >     0x0000070: 2bb6 05b2 b0
>>> > > > > > > > > >   Stackmap Table:
>>> > > > > > > > > >     same_frame(@6)
>>> > > > > > > > > >     append_frame(@103,Object[#2940])
>>> > > > > > > > > >     same_frame(@112)
>>> > > > > > > > > >
>>> > > > > > > > > > any ideas?
>>> > > > > > > > > >
>>> > > > > > > > > > thanks
>>> > > > > > > > > >
>>> > > > > > > > > > Dr Mich Talebzadeh
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > LinkedIn * https://www.linkedin.com/profile/view?id=
>>> > > > > > > > > > AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>> > > > > > > > > > <https://www.linkedin.com/profile/view?id=
>>> > > > > > > > AAEAAAAWh2gBxianrbJd6zP6AcPCCd
>>> > > > > > > > > > OABUrV8Pw>*
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > http://talebzadehmich.wordpress.com
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > *Disclaimer:* Use it at your own
risk. Any and all
>>> > > > responsibility
>>> > > > > > for
>>> > > > > > > > any
>>> > > > > > > > > > loss, damage or destruction of data
or any other
>>> property
>>> > > which
>>> > > > > may
>>> > > > > > > > arise
>>> > > > > > > > > > from relying on this email's technical
content is
>>> > explicitly
>>> > > > > > > > disclaimed.
>>> > > > > > > > > > The author will in no case be liable
for any monetary
>>> > damages
>>> > > > > > arising
>>> > > > > > > > > from
>>> > > > > > > > > > such loss, damage or destruction.
>>> > > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>

Mime
View raw message