spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Spark Standalone cluster job to connect Hbase is Stuck
Date Tue, 02 Feb 2016 01:16:06 GMT
>From your first email, it seems that you don't observer output from hbase
client.
Spark 1.2 was quite old, missing fixes for log4j such as SPARK-9826

Can you change the following line in HBase's conf/log4j.properties
from:
log4j.logger.org.apache.hadoop.hbase=INFO

to:
log4j.logger.org.apache.hadoop.hbase=DEBUG

on client and server sides (restart servers for the above to take effect)
to see if you would have more clue ?

On Mon, Feb 1, 2016 at 4:44 PM, sudhir patil <spatil.sudhir@gmail.com>
wrote:

> Thanks Ted for quick reply.
>
> I am using spark 1.2, exporting Hbase conf directory containing
> hbase-site.xml in HADOOP_CLASSPATH & SPARK_CLASSPATH. Do i need to do
> anything else?
>
> Issues in connecting to kerberos Hbase through spark yarn cluster is fixed
> spark 1.4+, so i am trying if it works in spark stand alone mode in spark
> 1.2, as i cannot upgrade cluster.
> https://issues.apache.org/jira/browse/SPARK-6918
>
>
>
> On Tue, Feb 2, 2016 at 8:31 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>
>> Is the hbase-site.xml on the classpath of the worker nodes ?
>>
>> Which Spark release are you using ?
>>
>> Cheers
>>
>> On Mon, Feb 1, 2016 at 4:25 PM, sudhir patil <spatil.sudhir@gmail.com>
>> wrote:
>>
>>> Spark job on Standalone cluster is Stuck, shows no logs after
>>> "util.AkkaUtils: Connecting to HeartbeatReceiver" on worker nodes and
>>> "storage.BlockmanagerInfo: Added broadcast..." on client driver side.
>>>
>>> Would be great, if you could clarify any of these ( or better all of
>>> these :)
>>> 1. Did anyone see similar issue? or any clues on what could be the
>>> reason?
>>> 2. How do you i increase debug or log level? to see whats actually
>>> happening?
>>> 3. Any clues or links on how to use kerborised Hbase in spark standalone?
>>>
>>
>>
>

Mime
View raw message