nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Wilson <wilson...@gmail.com>
Subject Re: PutHDFS Help
Date Wed, 16 Sep 2015 18:41:50 GMT
That's not the case.  I have a 4 node HDP cluster with 1 client node
running NiFi and the Hadoop client libraries.  I'm able to run NiFi as user
'nifi' and have created a '/users/nifi' directory in HDFS and that works.

I need to think my use case through and will re-post if needed.  I was
thinking I could "put as" a different user [other than the one I run NiFi
as], but need to sort the case out in my head first.

Thanks for the help.

-Chris

On Wed, Sep 16, 2015 at 2:31 PM, Matt Gilman <matt.c.gilman@gmail.com>
wrote:

> Awesome. I've never actually tried this myself, but it looks like you
> should be able to configure the Remote Owner and the Remote Group as
> properties of PutHDFS. Additionally, it states that this only works if NiFi
> is running as a user that has HDFS super user privileges. Can you confirm
> that is the case here? Thanks.
>
> Matt
>
> On Wed, Sep 16, 2015 at 2:19 PM, Christopher Wilson <wilsoncj1@gmail.com>
> wrote:
>
>> Thanks, I'm over that hurdle.  What about configuring PutHDFS then?
>> Still pretty lost here.
>>
>> -Chris
>>
>> On Wed, Sep 16, 2015 at 1:28 PM, Matt Gilman <matt.c.gilman@gmail.com>
>> wrote:
>>
>>> Does the user you specified have access to the directory structure where
>>> NiFi is installed? Specifically do they have access to
>>> lib/bootstrap/nifi-bootstrap-<version>.jar? Are there any more details
with
>>> that error message?
>>>
>>> Just to be sure, do they have access to all paths configured in
>>> conf/nifi.properties (assuming you've changed them from defaults)?
>>>
>>> Matt
>>>
>>>
>>> On Wed, Sep 16, 2015 at 1:21 PM, Christopher Wilson <wilsoncj1@gmail.com
>>> > wrote:
>>>
>>>> What version are you running?  I'm getting "Error: Could not find or
>>>> load class org.apache.nifi.bootstrap.RunNiFi" when I change to a different
>>>> user.
>>>>
>>>> -Chris
>>>>
>>>> On Wed, Sep 16, 2015 at 12:52 PM, Matt Gilman <matt.c.gilman@gmail.com>
>>>> wrote:
>>>>
>>>>> Chris,
>>>>>
>>>>> You shouldn't need to run NiFi as root. What is currently preventing
>>>>> you from doing so? Are you setting the run.as user in the
>>>>> conf/bootstrap.conf file?
>>>>>
>>>>> Matt
>>>>>
>>>>> On Wed, Sep 16, 2015 at 12:42 PM, Christopher Wilson <
>>>>> wilsoncj1@gmail.com> wrote:
>>>>>
>>>>>> I have a question regarding configuration of PutHDFS.  I'm running
>>>>>> NiFi as root (because I can't find how not to) and I've installed
NiFi on a
>>>>>> client node in my Hadoop cluster (HDP 2.3).  I can't find where to
tell
>>>>>> PutHDFS to put files as a specific user.  Not that strong on Hadoop
>>>>>> configuration so any help appreciated.
>>>>>>
>>>>>> Thanks in advance.
>>>>>>
>>>>>> Nifi version 0.2.1
>>>>>> Java version 1.7
>>>>>> RedHat 6.7
>>>>>>
>>>>>> PutHDFS[id=65cf6d5a-d5fb-4b77-82b5-f1d71a80ef72] Failed to write
to
>>>>>> HDFS due to org.apache.nifi.processor.exception.ProcessException:
>>>>>> IOException thrown from PutHDFS[id=65cf6d5a-d5fb-4b77-82b5-f1d71a80ef72]:
>>>>>> org.apache.hadoop.security.AccessControlException: Permission denied:
>>>>>> user=root, access=WRITE,
>>>>>> inode="/user/NOT-ROOT/.file.log":not-root:hdfs:drwxrwxr-x
>>>>>>
>>>>>> -Chris
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message