nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Olav Jordens <Olav.Jord...@2degreesmobile.co.nz>
Subject RE: PutHiveStreaming has only one concurrent task
Date Mon, 07 Nov 2016 20:45:18 GMT
Hi Matt – Thanks for the prompt reply. This processor is perfect for our needs – great
job!
Olav

From: Matt Burgess [mailto:mattyb149@apache.org]
Sent: Tuesday, 8 November 2016 6:08 a.m.
To: users@nifi.apache.org
Subject: Re: PutHiveStreaming has only one concurrent task

Olav,

I contributed that processor, and for the life of me I can't remember why it is forcing serial
execution. My guess is that it was a limitation of the version of the Hive Streaming libraries.
I will try it with multiple concurrent tasks to see if it works, and if so, I will write a
Jira to support concurrency for this processor. In either case I will reply here with my findings.

Regards,
Matt

On Sun, Nov 6, 2016 at 7:18 PM, Joe Witt <joe.witt@gmail.com<mailto:joe.witt@gmail.com>>
wrote:
Olav,

I'll let someone more familiar with that processor comment on the best usage pattern but will
say that for any processor which does not offer controls on the number of concurrent tasks
then it means that processor is meant to run serially by design.

Thanks
Joe

On Sun, Nov 6, 2016 at 7:06 PM, Olav Jordens <Olav.Jordens@2degreesmobile.co.nz<mailto:Olav.Jordens@2degreesmobile.co.nz>>
wrote:
Hi,

I have noticed that it is not possible to change the number of concurrent tasks on the Scheduling
tab of the PutHiveStreaming processor from the default value of 1. Is there a reason for this,
or is it a bug? What would be the consequence if I try to get around this by using a DistributeLoad
processor followed by a sequence of PutHiveStreaming processors, which would all be streaming
into the same Hive table?

Thanks,
Olav



Olav Jordens
Senior ETL Developer
Two Degrees Mobile Limited
===========================
(M) 022 620 2429
(P) 09 919 7000
www.2degreesmobile.co.nz<http://www.2degreesmobile.co.nz>
[cid:image001.jpg@01D239A4.CF82EED0]
Two Degrees Mobile Limited | 47-49 George Street | Newmarket | Auckland | New Zealand |
PO Box 8355 | Symonds Street | Auckland 1150 | New Zealand | Fax +64 9 919 7001<tel:%2B64%209%20919%207001>

[cid:image002.png@01D239A4.CF82EED0] [cid:image003.png@01D239A4.CF82EED0]  [cid:image004.png@01D239A4.CF82EED0]
 [cid:image005.png@01D239A4.CF82EED0]

________________________________

Disclaimer
The e-mail and any files transmitted with it are confidential and may contain privileged or
copyright information. If you are not the intended recipient you must not copy, distribute,
or use this e-mail or the information contained in it for any purpose other than to notify
us of the error. If you have received this message in error, please notify the sender immediately,
by email or phone (+64 9 919 7000<tel:%28%2B64%209%20919%207000>) and delete this email
from your system. Any views expressed in this message are those of the individual sender,
except where the sender specifically states them to be the views of Two Degrees Mobile Limited.
We do not guarantee that this material is free from viruses or any other defects although
due care has been taken to minimize the risk




Mime
View raw message