phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Soldatov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3771) Phoenix Storage Handler with Hive on Spark
Date Wed, 05 Jul 2017 16:54:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16075073#comment-16075073
] 

Sergey Soldatov commented on PHOENIX-3771:
------------------------------------------

[~jamestaylor] It's an enhancement. At the moment we supports only tez and mr engines. Honestly
speaking I'm not sure how good spark engine works with external storage handlers and whether
it works at all. Need to investigate. 

> Phoenix Storage Handler with Hive on Spark
> ------------------------------------------
>
>                 Key: PHOENIX-3771
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3771
>             Project: Phoenix
>          Issue Type: Improvement
>         Environment: Hadoop 2.7.3 HBase 1.1.4 Phoenix 4.10.0 Spark 1.6.3/2.1.0
>            Reporter: Sudhir Babu Pothineni
>             Fix For: 4.12.0
>
>
> We are working on join Hive table with Pheonix, Spark enabled. Right now its hit a wall
with following exception.
> org.apache.hive.service.cli.HiveSQLException: java.io.IOException: java.io.IOException:
spark execution engine unsupported yet.
>         at org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:352)
>         at org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:220)
>         at org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:685)
>         at sun.reflect.GeneratedMethodAccessor58.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
>         at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
>         at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:422)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>         at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
>         at com.sun.proxy.$Proxy22.fetchResults(Unknown Source)
>         at org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:454)
>         at org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:672)
>         at org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1553)
>         at org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1538)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>         at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
>         at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: java.io.IOException: spark execution engine unsupported
yet.
>         at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:507)
>         at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:414)
>         at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:140)
>         at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1670)
>         at org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:347)
>         ... 24 more
> Caused by: java.io.IOException: spark execution engine unsupported yet.
>         at org.apache.phoenix.hive.mapreduce.PhoenixInputFormat.getSplits(PhoenixInputFormat.java:128)
>         at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextSplits(FetchOperator.java:362)
>         at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:294)
>         at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:445)
>         ... 28 more



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message