nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Olav Jordens <>
Subject RE: PutHiveStreaming has only one concurrent task
Date Mon, 07 Nov 2016 20:45:18 GMT
Hi Matt – Thanks for the prompt reply. This processor is perfect for our needs – great

From: Matt Burgess []
Sent: Tuesday, 8 November 2016 6:08 a.m.
Subject: Re: PutHiveStreaming has only one concurrent task


I contributed that processor, and for the life of me I can't remember why it is forcing serial
execution. My guess is that it was a limitation of the version of the Hive Streaming libraries.
I will try it with multiple concurrent tasks to see if it works, and if so, I will write a
Jira to support concurrency for this processor. In either case I will reply here with my findings.


On Sun, Nov 6, 2016 at 7:18 PM, Joe Witt <<>>

I'll let someone more familiar with that processor comment on the best usage pattern but will
say that for any processor which does not offer controls on the number of concurrent tasks
then it means that processor is meant to run serially by design.


On Sun, Nov 6, 2016 at 7:06 PM, Olav Jordens <<>>

I have noticed that it is not possible to change the number of concurrent tasks on the Scheduling
tab of the PutHiveStreaming processor from the default value of 1. Is there a reason for this,
or is it a bug? What would be the consequence if I try to get around this by using a DistributeLoad
processor followed by a sequence of PutHiveStreaming processors, which would all be streaming
into the same Hive table?


Olav Jordens
Senior ETL Developer
Two Degrees Mobile Limited
(M) 022 620 2429
(P) 09 919 7000<>
Two Degrees Mobile Limited | 47-49 George Street | Newmarket | Auckland | New Zealand |
PO Box 8355 | Symonds Street | Auckland 1150 | New Zealand | Fax +64 9 919 7001<tel:%2B64%209%20919%207001>

[cid:image002.png@01D239A4.CF82EED0] [cid:image003.png@01D239A4.CF82EED0]  [cid:image004.png@01D239A4.CF82EED0]


The e-mail and any files transmitted with it are confidential and may contain privileged or
copyright information. If you are not the intended recipient you must not copy, distribute,
or use this e-mail or the information contained in it for any purpose other than to notify
us of the error. If you have received this message in error, please notify the sender immediately,
by email or phone (+64 9 919 7000<tel:%28%2B64%209%20919%207000>) and delete this email
from your system. Any views expressed in this message are those of the individual sender,
except where the sender specifically states them to be the views of Two Degrees Mobile Limited.
We do not guarantee that this material is free from viruses or any other defects although
due care has been taken to minimize the risk

View raw message