spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vasu Bajaj (Jira)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-19811) sparksql 2.1 can not prune hive partition
Date Thu, 10 Oct 2019 08:01:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-19811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16948299#comment-16948299
] 

Vasu Bajaj edited comment on SPARK-19811 at 10/10/19 8:00 AM:
--------------------------------------------------------------

 the issue arises when you use your partition name in upper case. try using partition in
lower case, this solved the issue for me.


was (Author: vasubajaj):
It's too late, but the issue arises when you use your partition name in upper case. try using
partition in lower case, this solved the issue for me.

> sparksql 2.1 can not prune hive partition 
> ------------------------------------------
>
>                 Key: SPARK-19811
>                 URL: https://issues.apache.org/jira/browse/SPARK-19811
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.1.0
>            Reporter: sydt
>            Priority: Major
>
> When sparksql2.1 execute sql, it has error:
> java.lang.RuntimeException: Expected only partition pruning predicates: (isnotnull(DAY_ID#216)
&& (DAY_ID#216 = 20170212))  and the sql  sentence is 
> select PROD_INST_ID from CRM_DB.ITG_PROD_INST WHERE DAY_ID='20170212' AND  PROV_ID  ='842'
limit 10; where DAY_ID and PROVE_ID is partition in hive.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message