nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kumiko Yada <Kumiko.Y...@ds-iq.com>
Subject RE: Cannot Authenticate the Azure Datalake Store
Date Sat, 07 May 2016 01:14:01 GMT
It looks like there are recent changes in …\.m2\repository\com\microsoft\azure\azure-mgmt-datalake-store\1.0.0-SNAPSHOT\azure-mgmt-datalake-store-1.0.0-20160420.002135-2.jar!\com\microsoft\azure\management\datalake\store\DataLakeStoreAccountManagementClientImpl.class.
 I’m getting the same errors in my  sample java program now ☹

-Kumiko

From: Kumiko Yada [mailto:Kumiko.Yada@ds-iq.com]
Sent: Friday, May 6, 2016 10:45 AM
To: users@nifi.apache.org
Cc: Ki Kang <Ki.Kang@ds-iq.com>
Subject: RE: Cannot Authenticate the Azure Datalake Store

HI Bryan,

Here are the dependencies:

    <dependencies>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-processor-utils</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-mock</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.11</version>
            <scope>test</scope>
        </dependency>
                                        <dependency>
            <groupId>com.microsoft.azure</groupId>
            <artifactId>azure-client-authentication</artifactId>
            <version>1.0.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>com.microsoft.azure</groupId>
            <artifactId>azure-mgmt-datalake-store</artifactId>
            <version>1.0.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
           <artifactId>slf4j-api</artifactId>
            <version>1.7.5</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <version>1.6.4</version>
        </dependency>
    </dependencies>

Thanks
Kumiko

From: Bryan Bende [mailto:bbende@gmail.com]
Sent: Friday, May 6, 2016 7:04 AM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>
Cc: Ki Kang <Ki.Kang@ds-iq.com<mailto:Ki.Kang@ds-iq.com>>
Subject: Re: Cannot Authenticate the Azure Datalake Store

Hello,

It seems like maybe a wrong version of a library is being used, or maybe a JAR is missing
that needs to be included in your NAR.

Can you share what dependencies you have in the pom.xml of your processors project?

You can check under NIFI_HOME/work/nar/extensions/<YOUR_NAR>/META-INF/bundled-dependencies/
to see the JARs that are being included in your NAR.

-Bryan


On Fri, May 6, 2016 at 12:52 AM, Kumiko Yada <Kumiko.Yada@ds-iq.com<mailto:Kumiko.Yada@ds-iq.com>>
wrote:
Hello,

I’m writing the custom processor to create a file in the Azure Datalake Store.  I wrote
the sample java program to do using this sample code, https://azure.microsoft.com/en-us/documentation/articles/data-lake-store-get-started-java-sdk/.
 This is working fine.  However, when I wrote the code in the custom processor, it’s failing
if the Authenticate code with this this error when this custom processor is running.  I didn’t
get any build error when I built this custom processor.   Do you have any ideas why I’m
getting this error?  Is there any file I need to copy to the nifi folder other than nar file
for the custom processor?

2016-05-05 20:37:37,338 ERROR [Timer-Driven Process Thread-4] d.p.custom.CreateFileAzureDatalakeStore
CreateFileAzureDatalakeStore[id=f89e5860-1d99-4d20-be71-769f6bb775c7] CreateFileAzureDatalakeStore[id=f89e5860-1d99-4d20-be71-769f6bb775c7]
failed to process due to java.lang.NoSuchMethodError: com.microsoft.azure.AzureServiceClient:
method <init>()V not found; rolling back session: java.lang.NoSuchMethodError: com.microsoft.azure.AzureServiceClient:
method <init>()V not found
2016-05-05 20:37:37,342 INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService
Saved flow controller org.apache.nifi.controller.FlowController@3dcbc96c<mailto:org.apache.nifi.controller.FlowController@3dcbc96c>
// Another save pending = false
2016-05-05 20:37:37,342 ERROR [Timer-Driven Process Thread-4] d.p.custom.CreateFileAzureDatalakeStore
java.lang.NoSuchMethodError: com.microsoft.azure.AzureServiceClient: method <init>()V
not found
                    at com.microsoft.azure.management.datalake.store.DataLakeStoreAccountManagementClientImpl.<init>(DataLakeStoreAccountManagementClientImpl.java:176)
~[na:na]
                    at com.microsoft.azure.management.datalake.store.DataLakeStoreAccountManagementClientImpl.<init>(DataLakeStoreAccountManagementClientImpl.java:166)
~[na:na]
                    at dsiq.processors.custom.CreateFileAzureDatalakeStore.SetupClients(CreateFileAzureDatalakeStore.java:286)
~[na:na]
                    at dsiq.processors.custom.CreateFileAzureDatalakeStore.onTrigger(CreateFileAzureDatalakeStore.java:269)
~[na:na]
                    at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
~[nifi-api-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT]
                    at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1057)
[nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT]
                    at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
[nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT]
                    at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
[nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT]
                    at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:123)
[nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT]
                    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_77]
                    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_77]
                    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_77]
                    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_77]
                    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_77]
                    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_77]
                    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
2016-05-05 20:37:37,346 ERROR [Timer-Driven Process Thread-4] d.p.custom.CreateFileAzureDatalakeStore
CreateFileAzureDatalakeStore[id=f89e5860-1d99-4d20-be71-769f6bb775c7] CreateFileAzureDatalakeStore[id=f89e5860-1d99-4d20-be71-769f6bb775c7]
failed to process session due to java.lang.NoSuchMethodError: com.microsoft.azure.AzureServiceClient:
method <init>()V not found: java.lang.NoSuchMethodError: com.microsoft.azure.AzureServiceClient:
method <init>()V not found
2016-05-05 20:37:37,347 ERROR [Timer-Driven Process Thread-4] d.p.custom.CreateFileAzureDatalakeStore
java.lang.NoSuchMethodError: com.microsoft.azure.AzureServiceClient: method <init>()V
not found
                    at com.microsoft.azure.management.datalake.store.DataLakeStoreAccountManagementClientImpl.<init>(DataLakeStoreAccountManagementClientImpl.java:176)
~[na:na]
                    at com.microsoft.azure.management.datalake.store.DataLakeStoreAccountManagementClientImpl.<init>(DataLakeStoreAccountManagementClientImpl.java:166)
~[na:na]
                    at dsiq.processors.custom.CreateFileAzureDatalakeStore.SetupClients(CreateFileAzureDatalakeStore.java:286)
~[na:na]
                    at dsiq.processors.custom.CreateFileAzureDatalakeStore.onTrigger(CreateFileAzureDatalakeStore.java:269)
~[na:na]
                    at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
~[nifi-api-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT]
                    at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1057)
~[nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT]
                    at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
[nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT]
                    at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
[nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT]
                    at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:123)
[nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT]
                    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_77]
                    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_77]
                    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_77]
                    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_77]
                    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_77]
                    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_77]
                    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]



ApplicationTokenCredentials creds = new ApplicationTokenCredentials(_clientId, _tenantId,
_clientSecret, null);

 _adlsClient = new DataLakeStoreAccountManagementClientImpl(creds); ---------- failing in
this code
_adlsFileSystemClient = new DataLakeStoreFileSystemManagementClientImpl(creds);
_adlsClient.setSubscriptionId(_subId);

Thanks
Kumiko


Mime
View raw message