nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Boris Tyukin <bo...@boristyukin.com>
Subject Re: Fetch Contents of HDFS Directory as a Part of a Larger Flow
Date Thu, 03 May 2018 19:57:37 GMT
Shawn, I am not answering your question but I am curious how you would see
logistics of  HTTP Request and Response Processors. Is your intent to build
REST api entirely in NiFi? I asked a question here before as I had a
similar idea but was told it was not a good strategy as it would be
difficult to sync request/response.

On Thu, May 3, 2018 at 11:35 AM, Shawn Weeks <sweeks@weeksconsulting.us>
wrote:

> I'm building a rest service with the HTTP Request and Response
> Processors to support data extracts from Hive. Since some of the extracts
> can be quiet large using the SelectHiveQL Processor isn't a performant
> option and instead I'm trying to use on demand Hive Temporary Tables to do
> the heavy lifting via CTAS(Create Table as Select). Since GetHDFS doesn't
> support an incoming connection I'm trying to figure out another way to
> fetch the files Hive creates and return them as a download in the web
> service. Has anyone else worked out a good solution for fetching the
> contents of a directory from HDFS as a part of larger flow?
>
>
> Thanks
>
> Shawn
>

Mime
View raw message