spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mostafa Alaa Mohamed <mohamedamost...@etisalat.ae>
Subject RE: Spark Hive Rejection
Date Tue, 04 Oct 2016 05:27:22 GMT
No, Sometime when you have table with column int and you insert in this column string job will
be fail some times


Best Regards,
Mostafa Alaa Mohamed,
Technical Expert Big Data,
M: +971506450787
Email: mohamedamostafa@etisalat.ae<mailto:mohamedamostafa@etisalat.ae>

From: Michael Segel [mailto:msegel_hadoop@hotmail.com]
Sent: Thursday, September 29, 2016 04:26 PM
To: Mostafa Alaa Mohamed
Cc: user@spark.apache.org
Subject: Re: Spark Hive Rejection

Correct me if I’m wrong but isn’t hive schema on read and not on write?
So you shouldn’t fail on write.


On Sep 29, 2016, at 1:25 AM, Mostafa Alaa Mohamed <mohamedamostafa@etisalat.ae<mailto:mohamedamostafa@etisalat.ae>>
wrote:

Dears,
I want to ask
•         What will happened if there are rejections rows when inserting dataframe into
hive?
o   Rejection will be for example table required integer into column and dataframe include
string.
o   Duplication rejection restriction from the table itself?
•         How can we specify the rejection directory?
If not avaiable do you recommend to open Jira issue?

Best Regards,
Mostafa Alaa Mohamed,
Technical Expert Big Data,
M: +971506450787
Email: mohamedamostafa@etisalat.ae<mailto:mohamedamostafa@etisalat.ae>

________________________________
The content of this email together with any attachments, statements and opinions expressed
herein contains information that is private and confidential are intended for the named addressee(s)
only. If you are not the addressee of this email you may not copy, forward, disclose or otherwise
use it or any part of it in any form whatsoever. If you have received this message in error
please notify postmaster@etisalat.ae<mailto:postmaster@etisalat.ae> by email immediately
and delete the message without making any copies.

________________________________
The content of this email together with any attachments, statements and opinions expressed
herein contains information that is private and confidential are intended for the named addressee(s)
only. If you are not the addressee of this email you may not copy, forward, disclose or otherwise
use it or any part of it in any form whatsoever. If you have received this message in error
please notify postmaster@etisalat.ae by email immediately and delete the message without making
any copies.
Mime
View raw message