spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From RK <prk...@yahoo.com.INVALID>
Subject Re: Does SparkSQL not support nested IF(1=1, 1, IF(2=2, 2, 3)) statements?
Date Mon, 05 Jan 2015 03:41:23 GMT
BTW, I am seeing this issue in Spark 1.1.1. 

     On Sunday, January 4, 2015 7:29 PM, RK <prk001@yahoo.com.INVALID> wrote:
   

 When I use a single IF statement like "select IF(col1 != "", col1+'$'+col3, col2+'$'+col3)
from my_table", it works fine.
However, when I use a nested IF like "select IF(col1 != "", col1+'$'+col3, IF(col2 != "", col2+'$'+col3,
'$')) from my_table", I am getting the following exception.
Exception in thread "main" org.apache.spark.sql.catalyst.errors.package$TreeNodeException:
Unresolved attributes: if (NOT (col1#1 = )) (CAST((CAST(col1#1, DoubleType) + CAST($, DoubleType)),
DoubleType) + CAST(col3#3, DoubleType)) else if (NOT (col2#2 = )) (CAST((CAST(col2#2, DoubleType)
+ CAST($, DoubleType)), DoubleType) + CAST(col3#3, DoubleType)) else $ AS c0#4, tree:Project
[if (NOT (col1#1 = )) (CAST((CAST(col1#1, DoubleType) + CAST($, DoubleType)), DoubleType)
+ CAST(col3#3, DoubleType)) else if (NOT (col2#2 = )) (CAST((CAST(col2#2, DoubleType) + CAST($,
DoubleType)), DoubleType) + CAST(col3#3, DoubleType)) else $ AS c0#4] Subquery my_table 
SparkLogicalPlan (ExistingRdd [DB#0,col1#1,col2#2,col3#3], MappedRDD[97] at getCallSite at
DStream.scala:294)
Does Spark SQL not support nested IF queries or is my query incorrect?
Thanks,RK 

   
Mime
View raw message