spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: explain codegen
Date Mon, 04 Apr 2016 10:15:54 GMT
Could the error I encountered be due to missing import(s) of implicit ?

Thanks

On Sun, Apr 3, 2016 at 9:42 PM, Reynold Xin <rxin@databricks.com> wrote:

> Works for me on latest master.
>
>
>
> scala> sql("explain codegen select 'a' as a group by 1").head
> res3: org.apache.spark.sql.Row =
> [Found 2 WholeStageCodegen subtrees.
> == Subtree 1 / 2 ==
> WholeStageCodegen
> :  +- TungstenAggregate(key=[], functions=[], output=[a#10])
> :     +- INPUT
> +- Exchange SinglePartition, None
>    +- WholeStageCodegen
>       :  +- TungstenAggregate(key=[], functions=[], output=[])
>       :     +- INPUT
>       +- Scan OneRowRelation[]
>
> Generated code:
> /* 001 */ public Object generate(Object[] references) {
> /* 002 */   return new GeneratedIterator(references);
> /* 003 */ }
> /* 004 */
> /* 005 */ /** Codegened pipeline for:
> /* 006 */ * TungstenAggregate(key=[], functions=[], output=[a#10])
> /* 007 */ +- INPUT
> /* 008 */ */
> /* 009 */ final class GeneratedIterator extends
> org.apache.spark.sql.execution.BufferedRowIterator {
> /* 010 */   private Object[] references;
> /* 011 */   ...
>
>
> On Sun, Apr 3, 2016 at 9:38 PM, Jacek Laskowski <jacek@japila.pl> wrote:
>
>> Hi,
>>
>> Looks related to the recent commit...
>>
>> Repository: spark
>> Updated Branches:
>>   refs/heads/master 2262a9335 -> 1f0c5dceb
>>
>> [SPARK-14350][SQL] EXPLAIN output should be in a single cell
>>
>> Jacek
>> 03.04.2016 7:00 PM "Ted Yu" <yuzhihong@gmail.com> napisaƂ(a):
>>
>>> Hi,
>>> Based on master branch refreshed today, I issued 'git clean -fdx' first.
>>>
>>> Then this command:
>>> build/mvn clean -Phive -Phive-thriftserver -Pyarn -Phadoop-2.6
>>> -Dhadoop.version=2.7.0 package -DskipTests
>>>
>>> I got the following error:
>>>
>>> scala>  sql("explain codegen select 'a' as a group by 1").head
>>> org.apache.spark.sql.catalyst.parser.ParseException:
>>> extraneous input 'codegen' expecting {'(', 'SELECT', 'FROM', 'ADD',
>>> 'DESC', 'WITH', 'VALUES', 'CREATE', 'TABLE', 'INSERT', 'DELETE',
>>> 'DESCRIBE', 'EXPLAIN', 'LOGICAL', 'SHOW', 'USE', 'DROP', 'ALTER', 'MAP',
>>> 'SET', 'START', 'COMMIT', 'ROLLBACK', 'REDUCE', 'EXTENDED', 'REFRESH',
>>> 'CLEAR', 'CACHE', 'UNCACHE', 'FORMATTED', 'DFS', 'TRUNCATE', 'ANALYZE',
>>> 'REVOKE', 'GRANT', 'LOCK', 'UNLOCK', 'MSCK', 'EXPORT', 'IMPORT',
>>> 'LOAD'}(line 1, pos 8)
>>>
>>> == SQL ==
>>> explain codegen select 'a' as a group by 1
>>> --------^^^
>>>
>>> Can someone shed light ?
>>>
>>> Thanks
>>>
>>
>

Mime
View raw message