nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koji Kawamura <ijokaruma...@gmail.com>
Subject Re: Execute multiple HQL statements in PutHiveQL or SelectHiveQL
Date Thu, 19 Apr 2018 06:54:36 GMT
Hello Dejan,

I tested SET property statements bundled with INSERT statement in a single
FlowFile passed to PutHiveQL.
The same warning message is logged as you reported. However, actual INSERT
was successful, I confirmed new rows were inserted.
Please let us know if not the case.

Although the warning is harmless, I agree it can be mis-leading.
I submitted a JIRA and a PR to suppress the warning logs for such SET
commands.
https://issues.apache.org/jira/browse/NIFI-5095

Thanks,
Koji


On Tue, Apr 17, 2018 at 5:03 PM, Krstic, Dejan <Dejan.Krstic@t-mobile.at>
wrote:

> Matt,
>
>
>
> No, there are no comments at all in this script.
>
>
>
> Br
>
> Dejan
>
>
>
> *Von:* Matt Burgess [mailto:mattyb149@apache.org]
> *Gesendet:* Montag, 16. April 2018 18:38
> *An:* users@nifi.apache.org
> *Betreff:* Re: Execute multiple HQL statements in PutHiveQL or
> SelectHiveQL
>
>
>
> Dejan,
>
>
>
> Are there any comments in your Hive "script"? If so I believe you need to
> remove them [1].  When I've seen that issue, it's always been because of
> comments before the SET line.
>
>
>
> Regards,
>
> Matt
>
>
>
> [1] https://issues.apache.org/jira/browse/HIVE-1166
>
>
>
>
>
> On Mon, Apr 16, 2018 at 12:29 PM, Krstic, Dejan <Dejan.Krstic@t-mobile.at>
> wrote:
>
> Hello,
>
>
>
> I have Hive table from which I am trying to load data into other Hive
> table (same server) and to do some transformations in between. The problem
> is, I need to have two SET statements before I actually do INSERT statement
> and if I try to separate everything with semicolons (as configured in
> processor) I am getting following error message:
>
>
>
> 2018-04-16 16:11:59,368 WARN [Timer-Driven Process Thread-1]
> o.apache.nifi.processors.hive.PutHiveQL PutHiveQL[id=9d9290d2-2196-3e7b-a522-ab0e2a46db11]
> Failed to parse hiveQL: SET hive.exec.dynamic.partition = true due to
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:4 missing KW_ROLE
> at 'hive' near 'hive'
>
> line 1:8 missing EOF at '.' near 'hive':
>
>
>
> The two SET statements at the beginning:
>
>
>
> Set hive.exec.dynamic.partition = true;
>
> set hive.exec.dynamic.partition.mode=nonstrict;
>
>
>
> I also tried the same thing with SelectHiveQL but it seems not to accept
> multiple queries. Which way would be the best to fix this problem and what
> processor is the best to use in this situation?
>
>
>
> Thank you.
>
>
>
> Best regards
>
> Dejan Krstic
>
>
>
>
>
> *T-Mobile Austria GmbH*
>
> Dejan Krstic
>
> Data Manager
>
>
>
> Rennweg 97-99
> <https://maps.google.com/?q=Rennweg+97-99+%0D%0AA-1030+Wien&entry=gmail&source=g>
> A-1030 Wien
>
> E-Mail: mailto:dejan.krstic@t-mobile.at <dejan.krstic@t-mobile.at>
> *www.t-mobile.at <http://www.t-mobile.at/>*
>
> *DAS VERBINDET UNS.*
>
>
>
> ___________________________________________________________________________________
>
> Notice: This e-mail and any attachments are confidential and may be privileged.
>
> If you are not the intended recipient, notify the sender immediately, destroy all
>
> copies from your system and do not disclose or use the information for any purpose.
>
> Diese E-Mail inklusive aller Anhaenge ist vertraulich und koennte bevorrechtigtem
>
> Schutz unterliegen. Wenn Sie nicht der beabsichtigte Adressat sind, informieren Sie
>
> bitte den Absender unverzueglich, loeschen Sie alle Kopien von Ihrem System und
>
> veroeffentlichen Sie oder nutzen Sie die Information keinesfalls, gleich zu welchem Zweck.
>
>
>
> Think before you print!
>
>
>
> T-Mobile Austria GmbH
>
> Geschaeftsfuehrung: Dr. Andreas Bierwirth (Vorsitzender), Aufsichtsrat: Dr. Rolf Nafziger
(Vorsitzender)
>
> Firmenbuch: Handelsgericht Wien, Sitz Wien, FN 171112k, UID ATU 45011703, DVR 0898295
>
> Konto: UniCredit Bank Austria AG IBAN: AT93 1200 0528 4407 2301, BIC: BKAUATWW
>
>
>
> T-Mobile - Das verbindet uns.
>
> ___________________________________________________________________________________
>
>
>
> ___________________________________________________________________________________
> Notice: This e-mail and any attachments are confidential and may be privileged.
> If you are not the intended recipient, notify the sender immediately, destroy all
> copies from your system and do not disclose or use the information for any purpose.
> Diese E-Mail inklusive aller Anhaenge ist vertraulich und koennte bevorrechtigtem
> Schutz unterliegen. Wenn Sie nicht der beabsichtigte Adressat sind, informieren Sie
> bitte den Absender unverzueglich, loeschen Sie alle Kopien von Ihrem System und
> veroeffentlichen Sie oder nutzen Sie die Information keinesfalls, gleich zu welchem Zweck.
>
> Think before you print!
>
> T-Mobile Austria GmbH
> Geschaeftsfuehrung: Dr. Andreas Bierwirth (Vorsitzender), Aufsichtsrat: Dr. Rolf Nafziger
(Vorsitzender)
> Firmenbuch: Handelsgericht Wien, Sitz Wien, FN 171112k, UID ATU 45011703, DVR 0898295
> Konto: UniCredit Bank Austria AG IBAN: AT93 1200 0528 4407 2301, BIC: BKAUATWW
>
> T-Mobile - Das verbindet uns.
> ___________________________________________________________________________________
>
>

Mime
View raw message