spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Or (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-15538) Truncate table does not work on data source table
Date Thu, 26 May 2016 00:51:12 GMT

     [ https://issues.apache.org/jira/browse/SPARK-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Or updated SPARK-15538:
------------------------------
    Description: 
Truncate table does not  seems to work on data source table. It returns success without any
error , but table is not truncated. 

Repro:
{code}
val df = Seq((1 , "john", "CA") ,(2,"Mike", "NY"), (3, "Robert", "CA")).toDF("id", "name",
"state")
df.write.format("parquet").partitionBy("state").saveAsTable("emp")

scala> sql("truncate table emp") 
res8: org.apache.spark.sql.DataFrame = []

scala> sql("select * from emp").show() // FileNotFoundException
{code} 

  was:
Truncate table does not  seems to work on data source table. It returns success without any
error , but table is not truncated. 

Repro:
{code}
val df = Seq((1 , "john", "CA") ,(2,"Mike", "NY"), (3, "Robert", "CA")).toDF("id", "name",
"state")
df.write.format("parquet").partitionBy("state").saveAsTable("emp")

scala> sql("truncate table emp") 
res8: org.apache.spark.sql.DataFrame = []

scala> sql("select * from emp").show ;
+---+------+-----+
| id|  name|state|
+---+------+-----+
|  3|Robert|   CA|
|  1|  john|   CA|
|  2|  Mike|   NY|
+---+------+-----+

{code} 

The select should have returned no results. 

By scanning through  the code  I found  some of the other DDL commands like LOAD DATA ,  and
SHOW PARTITIONS are not allowed for data source table and they raise error. 

It  Might be good to throw error until the truncate table works with  data source table also.
 


> Truncate table does not work on data source table
> -------------------------------------------------
>
>                 Key: SPARK-15538
>                 URL: https://issues.apache.org/jira/browse/SPARK-15538
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>            Reporter: Suresh Thalamati
>            Assignee: Andrew Or
>            Priority: Minor
>
> Truncate table does not  seems to work on data source table. It returns success without
any error , but table is not truncated. 
> Repro:
> {code}
> val df = Seq((1 , "john", "CA") ,(2,"Mike", "NY"), (3, "Robert", "CA")).toDF("id", "name",
"state")
> df.write.format("parquet").partitionBy("state").saveAsTable("emp")
> scala> sql("truncate table emp") 
> res8: org.apache.spark.sql.DataFrame = []
> scala> sql("select * from emp").show() // FileNotFoundException
> {code} 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message