spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bas Harenslak <basharens...@gmail.com>
Subject Re: using replace function
Date Mon, 12 Dec 2016 13:48:12 GMT
There is no replace() function. You could use regexp_replace() for this:

SELECT regexp_replace(column_name, ’pattern’, ‘replacement’) FROM table_name

Documentation: https://spark.apache.org/docs/1.6.1/api/java/org/apache/spark/sql/functions.html#regexp_replace(org.apache.spark.sql.Column,%20java.lang.String,%20java.lang.String)
<https://spark.apache.org/docs/1.6.1/api/java/org/apache/spark/sql/functions.html#regexp_replace(org.apache.spark.sql.Column,
java.lang.String, java.lang.String)>

Rgrds Bas

> On 12 Dec 2016, at 08:54, Hitesh Goyal <hitesh.goyal@nlpcaptcha.com> wrote:
> 
> Hi team,
> I am using apache spark 1.6.1
> I want to access data from s3 using spark SQL.
> For this I need to use MySQL replace function. 
> DataFrame df=sqlcontext.sql(”Select replace(column_name,’ur’,’xy’) from table_name”);
>  
> But when I try to run this it is giving error that  function Replace is not defined.
> Exception in thread "main" org.apache.spark.sql.AnalysisException: undefined function
REPLACE;
>         at org.apache.spark.sql.catalyst.analysis.SimpleFunctionRegistry$$anonfun$2.apply(FunctionRegistry.scala:65)
>         at org.apache.spark.sql.catalyst.analysis.SimpleFunctionRegistry$$anonfun$2.apply(FunctionRegistry.scala:65)
>  
> Please specify how can I do this.
>  
> Regards,
> Hitesh Goyal
> Simpli5d Technologies
> Cont No.: 9996588220


Mime
View raw message