nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Austin Heyne <ahe...@ccri.com>
Subject GetHDFS from Azure Blob
Date Tue, 28 Mar 2017 17:15:37 GMT
Hi all,

Thanks for all the help you've given me so far. Today I'm trying to pull 
files from an Azure blob store. I've done some reading on this and from 
previous tickets [1] and guides [2] it seems the recommended approach is 
to place the required jars, to use the HDFS Azure protocol, in 
'Additional Classpath Resoures' and the hadoop core-site and hdfs-site 
configs into the 'Hadoop Configuration Resources'. I have my local HDFS 
properly configured to access wasb urls. I'm able to ls, copy to and 
from, etc with out problem. Using the same HDFS config files and trying 
both all the jars in my hadoop-client/lib directory (hdp) and using the 
jars recommend in [1] I'm still seeing the 
"java.lang.IllegalArgumentException: Wrong FS: " error in my NiFi logs 
and am unable to pull files from Azure blob storage.

Interestingly, it seems the processor is spinning up way to fast, the 
errors appear in the log as soon as I start the processor. I'm not sure 
how it could be loading all of those jars that quickly.

Does anyone have any experience with this or recommendations to try?

Thanks,
Austin

[1] https://issues.apache.org/jira/browse/NIFI-1922
[2] 
https://community.hortonworks.com/articles/71916/connecting-to-azure-data-lake-from-a-nifi-dataflow.html



Mime
View raw message