spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "1427357147@qq.com" <1427357...@qq.com>
Subject the issue about the + in column,can we support the string please?
Date Mon, 26 Mar 2018 03:36:15 GMT
Hi  all,

I have a table like below:

+---+---------+-----------+
| id|     name|sharding_id|
+---+---------+-----------+
|  1|leader us|      10000|
|  3|    mycat|      10000|
+---+---------+-----------+

My schema is :
root
 |-- id: integer (nullable = false)
 |-- name: string (nullable = true)
 |-- sharding_id: integer (nullable = false)

I want add a new column named newName. The new column is based on "name" and append "abc"
after it. My code looks like:

stud_scoreDF.withColumn("newName", stud_scoreDF.col("name") +  "abc"  ).show()
When I run the code, I got the reslult:
+---+---------+-----------+-------+
| id|     name|sharding_id|newName|
+---+---------+-----------+-------+
|  1|leader us|      10000|   null|
|  3|    mycat|      10000|   null|
+---+---------+-----------+-------+


I checked the code, the key code is  in arithmetic.scala. line 165.
It looks like:

override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = dataType match {
  case dt: DecimalType =>
    defineCodeGen(ctx, ev, (eval1, eval2) => s"$eval1.$$plus($eval2)")
  case ByteType | ShortType =>
    defineCodeGen(ctx, ev,
      (eval1, eval2) => s"(${ctx.javaType(dataType)})($eval1 $symbol $eval2)")
  case CalendarIntervalType =>
    defineCodeGen(ctx, ev, (eval1, eval2) => s"$eval1.add($eval2)")
  case _ =>
    defineCodeGen(ctx, ev, (eval1, eval2) => s"$eval1 $symbol $eval2")
}

My issue is:
Can we add case StringType in this class to support string append please?





1427357147@qq.com
Mime
View raw message