nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koji Kawamura <ijokaruma...@gmail.com>
Subject Re: HDF NIfi - Does Nifi writes provenance/data on HDP Node ?
Date Thu, 15 Jun 2017 01:35:08 GMT
Hi Shashi,

Sorry for delayed response. I am not aware that NiFi writes any
provenance information on HDP nodes. But if your goal is to expose
NiFi provenance data to HDFS, Hive (or Spark) to analyze provenance
data using those services, then SiteToSiteProvenanceReportingTask
might be helpful.

SiteToSiteProvenanceReportingTask can sends provenance events in JSON
format. You can send it to a NiFi input port then pass those into HDFS
by PutHDFS processor.

If not, would you elaborate what you are trying to accomplish?

Thanks,
Koji

On Mon, Jun 12, 2017 at 6:25 AM, Shashi Vishwakarma
<shashi.vish123@gmail.com> wrote:
> Hi
>
> I have HDF cluster with 3 Nifi instance which lunches jobs(Hive/Spark) on
> HDP cluster. Usually nifi writes all information to different repositories
> available on local machine.
>
> My question is - Does nifi writes any data,provenance information or does
> spilling on HDP nodes (ex. data nodes in HDP cluster) while accessing
> HDFS,Hive or Spark services ?
>
> Thanks
>
> Shashi

Mime
View raw message