nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pierre Villard <pierre.villard...@gmail.com>
Subject Re: PutHiveStreaming Error : java.lang.RuntimeException: java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: rwx------
Date Fri, 10 Nov 2017 08:25:36 GMT
Hi Mohit,

As far as I know, the Hive Streaming API is directly writing into HDFS and,
based on the stack trace, is trying to write into /tmp/hive. Right now this
folder is not writable by your user and it is the reason why it's failing.
I would try changing the chmod permissions in HDFS.

Pierre

2017-11-10 5:56 GMT+01:00 <mohit.jain@open-insights.co.in>:

> Hi all,
>
>
>
> I am using PutHiveStreaming processor to ingest the data into a
> partitioned table. I have given all the permissions to /tmp/hive folder.
>
> Still, it prompts the error :
>
>
>
> 2017-11-09 11:48:06,849 ERROR [Timer-Driven Process Thread-8]
> o.a.n.processors.hive.PutHiveStreaming PutHiveStreaming[id=9b7b6af6-015f-1000-8772-6b70eb0a4841]
> PutHiveStreaming[id=9b7b6af6-015f-1000-8772-6b70eb0a4841] failed to
> process due to java.lang.RuntimeException: java.lang.RuntimeException: The
> root scratch dir: /tmp/hive on HDFS should be writable. Current permissions
> are: rwx------; rolling back session: {}
>
> java.lang.RuntimeException: java.lang.RuntimeException: The root scratch
> dir: /tmp/hive on HDFS should be writable. Current permissions are:
> rwx------
>
>         at org.apache.hadoop.hive.ql.session.SessionState.start(
> SessionState.java:523)
>
>         at org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.
> createPartitionIfNotExists(HiveEndPoint.java:454)
>
>         at org.apache.hive.hcatalog.streaming.HiveEndPoint$
> ConnectionImpl.<init>(HiveEndPoint.java:318)
>
>         at org.apache.hive.hcatalog.streaming.HiveEndPoint$
> ConnectionImpl.<init>(HiveEndPoint.java:278)
>
>         at org.apache.hive.hcatalog.streaming.HiveEndPoint.
> newConnectionImpl(HiveEndPoint.java:215)
>
>         at org.apache.hive.hcatalog.streaming.HiveEndPoint.
> newConnection(HiveEndPoint.java:192)
>
>         at org.apache.hive.hcatalog.streaming.HiveEndPoint.
> newConnection(HiveEndPoint.java:122)
>
>         at org.apache.nifi.util.hive.HiveWriter.lambda$
> newConnection$6(HiveWriter.java:237)
>
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1149)
>
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:624)
>
>         at java.lang.Thread.run(Thread.java:748)
>
> Caused by: java.lang.RuntimeException: The root scratch dir: /tmp/hive on
> HDFS should be writable. Current permissions are: rwx------
>
>         at org.apache.hadoop.hive.ql.session.SessionState.
> createRootHDFSDir(SessionState.java:613)
>
>         at org.apache.hadoop.hive.ql.session.SessionState.
> createSessionDirs(SessionState.java:555)
>
>         at org.apache.hadoop.hive.ql.session.SessionState.start(
> SessionState.java:509)
>
>         ... 11 common frames omitted
>
>
> I’m sharing the detail error in the screenshot.
>
> Please let me know if anyone is aware about this issue.
>
>
>
> Regards,
>
> Mohit Jain
>

Mime
View raw message