nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Santosh Pawar <sant...@open-insights.com>
Subject Ingestion for hive to mongodb
Date Tue, 11 Jun 2019 05:41:45 GMT
Hi Team,

I am ingesting data from hive to mongodb using flow as mentioned below :

SelectHiveQL - > ConvertRecord ->  SplitJson -> PutMongo

Is there any way to ingest data from hive to mongodb using nifi because I
used two processor to push data into mongodb PutMongoRecord and PutMongo
Proccessor but there are limitation with processors given below :

1. PutMongoRecord Processor : upsert mode not avaibalble

2. PutMongo Processor : There is upsert mode but it pushes single object at
a time

Question : Is there any way to insert ArrayJson directly into MongoDB,
because in current flow there is  performance issue which is happening &
becoming a bottleneck as number of files which are getting processed are
now becoming very large since array of json can not be inserted & splitjson
processor is used as overhead to split each array object from these files.

Thanks,
Santosh Pawar

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message