nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Burgess <mattyb...@apache.org>
Subject Re: PutHiveStreaming has only one concurrent task
Date Mon, 07 Nov 2016 17:08:09 GMT
Olav,

I contributed that processor, and for the life of me I can't remember why
it is forcing serial execution. My guess is that it was a limitation of the
version of the Hive Streaming libraries. I will try it with multiple
concurrent tasks to see if it works, and if so, I will write a Jira to
support concurrency for this processor. In either case I will reply here
with my findings.

Regards,
Matt

On Sun, Nov 6, 2016 at 7:18 PM, Joe Witt <joe.witt@gmail.com> wrote:

> Olav,
>
> I'll let someone more familiar with that processor comment on the best
> usage pattern but will say that for any processor which does not offer
> controls on the number of concurrent tasks then it means that processor is
> meant to run serially by design.
>
> Thanks
> Joe
>
> On Sun, Nov 6, 2016 at 7:06 PM, Olav Jordens <Olav.Jordens@2degreesmobile.
> co.nz> wrote:
>
>> Hi,
>>
>>
>>
>> I have noticed that it is not possible to change the number of concurrent
>> tasks on the Scheduling tab of the PutHiveStreaming processor from the
>> default value of 1. Is there a reason for this, or is it a bug? What would
>> be the consequence if I try to get around this by using a DistributeLoad
>> processor followed by a sequence of PutHiveStreaming processors, which
>> would all be streaming into the same Hive table?
>>
>>
>>
>> Thanks,
>>
>> Olav
>>
>>
>>
>>
>> Olav Jordens
>> Senior ETL Developer
>> Two Degrees Mobile Limited
>> ===========================
>> (M) 022 620 2429
>> (P) 09 919 7000
>> www.2degreesmobile.co.nz
>>
>> Two Degrees Mobile Limited | 47-49 George Street | Newmarket | Auckland |
>> New Zealand |
>> PO Box 8355 | Symonds Street | Auckland 1150 | New Zealand | Fax +64 9
>> 919 7001
>>
>>
>>
>>
>> ------------------------------
>>
>> Disclaimer
>> The e-mail and any files transmitted with it are confidential and may
>> contain privileged or copyright information. If you are not the intended
>> recipient you must not copy, distribute, or use this e-mail or the
>> information contained in it for any purpose other than to notify us of the
>> error. If you have received this message in error, please notify the sender
>> immediately, by email or phone (+64 9 919 7000) and delete this email
>> from your system. Any views expressed in this message are those of the
>> individual sender, except where the sender specifically states them to be
>> the views of Two Degrees Mobile Limited. We do not guarantee that this
>> material is free from viruses or any other defects although due care has
>> been taken to minimize the risk
>>
>>
>>
>>
>

Mime
View raw message