spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiangrui Meng (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-8345) Add an SQL node as a feature transformer
Date Thu, 13 Aug 2015 05:12:46 GMT

     [ https://issues.apache.org/jira/browse/SPARK-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Xiangrui Meng updated SPARK-8345:
---------------------------------
    Parent Issue: SPARK-9930  (was: SPARK-8521)

> Add an SQL node as a feature transformer
> ----------------------------------------
>
>                 Key: SPARK-8345
>                 URL: https://issues.apache.org/jira/browse/SPARK-8345
>             Project: Spark
>          Issue Type: Sub-task
>          Components: ML
>            Reporter: Xiangrui Meng
>            Assignee: Yanbo Liang
>             Fix For: 1.6.0
>
>
> Some simple feature transformations can take leverage on SQL operators. Users do not
need to create an ML transformer for each of them. We can have an SQL transformer that executes
an SQL command which operates on the input dataframe.
> {code}
> val sql = new SQL()
>   .setStatement("SELECT *, length(text) AS text_length FROM __THIS__")
> {code}
> where "__THIS__" will be replaced by a temp table that represents the DataFrame.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message