spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Owen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-16319) Non-linear (DAG) pipelines need better explanation
Date Fri, 15 Jul 2016 12:24:20 GMT

    [ https://issues.apache.org/jira/browse/SPARK-16319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15379314#comment-15379314
] 

Sean Owen commented on SPARK-16319:
-----------------------------------

[~mmoroz] are you going to propose a text change or close this?

> Non-linear (DAG) pipelines need better explanation
> --------------------------------------------------
>
>                 Key: SPARK-16319
>                 URL: https://issues.apache.org/jira/browse/SPARK-16319
>             Project: Spark
>          Issue Type: Documentation
>          Components: ML
>    Affects Versions: 2.0.0
>            Reporter: Max Moroz
>            Priority: Minor
>
> There's a [paragraph|http://spark.apache.org/docs/2.0.0-preview/ml-guide.html#details]
about non-linear pipeline in the ML docs, but it's not clear how DAG pipeline differs from
a linear pipeline, and in fact, it seems that a "DAG Pipeline" results in the behavior identical
to that of a regular linear pipeline (the stages are simply applied in the order provided
when the pipeline is created). In addition, no checks of input and output columns seem to
occur when the pipeline.fit() or pipeline.transform() is called.
> It would be better to clarify in the docs and/or remove that paragraph.
> I'd be happy to write it up, but I have no idea what the intention of this concept is
at this point.
> [Additional reference on SO|http://stackoverflow.com/questions/37541668/non-linear-dag-ml-pipelines-in-apache-spark]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message