drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Khurram Faraaz (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-5945) Streaming aggregate does not support schema changes
Date Wed, 08 Nov 2017 23:07:01 GMT
Khurram Faraaz created DRILL-5945:
-------------------------------------

             Summary: Streaming aggregate does not support schema changes
                 Key: DRILL-5945
                 URL: https://issues.apache.org/jira/browse/DRILL-5945
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - Flow
    Affects Versions: 1.11.0
            Reporter: Khurram Faraaz


There are two JSON files, one of the arrays "arr" in f2.json is empty, whereas array "arr"
in f1.json is not empty, a count on arr results in, "Streaming aggregate does not support
schema changes"

{noformat}
0: jdbc:drill:schema=dfs.tmp> select count(arr) from `f*.json`;
Error: UNSUPPORTED_OPERATION ERROR: Streaming aggregate does not support schema changes
Prior schema : 
BatchSchema [fields=[arr(VARCHAR:REPEATED) [$data$(VARCHAR:REQUIRED)]], selectionVector=NONE]
New schema : 
BatchSchema [fields=[arr(INT:REPEATED) [$data$(INT:REQUIRED)]], selectionVector=NONE]

Fragment 0:0

[Error Id: 9081b02f-0de8-486a-b58b-f28ec3b84d98 on centos-01.qa.lab:31010] (state=,code=0)

{noformat}

Stack trace from drillbit.log

{noformat}
2017-11-08 13:36:48,159 [25fc870f-e4d6-7107-c6cc-6e68ba92c4ba:frag:0:0] INFO  o.a.d.e.p.i.a.StreamingAggBatch
- User Error Occurred: Streaming aggregate does not support schema changes
Prior schema :
BatchSchema [fields=[arr(VARCHAR:REPEATED) [$data$(VARCHAR:REQUIRED)]], selectionVector=NONE]
New schema :
BatchSchema [fields=[arr(INT:REPEATED) [$data$(INT:REQUIRED)]], selectionVector=NONE]
org.apache.drill.common.exceptions.UserException: UNSUPPORTED_OPERATION ERROR: Streaming aggregate
does not support schema changes
Prior schema :
BatchSchema [fields=[arr(VARCHAR:REPEATED) [$data$(VARCHAR:REQUIRED)]], selectionVector=NONE]
New schema :
BatchSchema [fields=[arr(INT:REPEATED) [$data$(INT:REQUIRED)]], selectionVector=NONE]


[Error Id: 9081b02f-0de8-486a-b58b-f28ec3b84d98 ]
        at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:586)
~[drill-common-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.physical.impl.aggregate.StreamingAggBatch.innerNext(StreamingAggBatch.java:207)
[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:164)
[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:134)
[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:164)
[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:105) [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:95) [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:234)
[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:227)
[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at java.security.AccessController.doPrivileged(Native Method) [na:1.8.0_144]
        at javax.security.auth.Subject.doAs(Subject.java:422) [na:1.8.0_144]
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)
[hadoop-common-2.7.0-mapr-1607.jar:na]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:227)
[drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
        at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.11.0-mapr.jar:1.11.0-mapr]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[na:1.8.0_144]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[na:1.8.0_144]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_144]

{noformat}

Data used in the query

{noformat}

[root@centos-01 json_data]# cat f1.json 
{
"id":1,
"arr":["str1","str2","str3","str4"]
}
[root@centos-01 json_data]# cat f2.json 
{
"id":2,
"arr":[]
}

>From sqlline

OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in
8.0
apache drill 1.11.0-mapr 
"got drill?"
0: jdbc:drill:schema=dfs.tmp> select * from `f*.json`;
+-----+--------------------------------+
| id  |              arr               |
+-----+--------------------------------+
| 1   | ["str1","str2","str3","str4"]  |
| 2   | []                             |
+-----+--------------------------------+
2 rows selected (0.447 seconds)
0: jdbc:drill:schema=dfs.tmp> select * from `f1.json`;
+-----+--------------------------------+
| id  |              arr               |
+-----+--------------------------------+
| 1   | ["str1","str2","str3","str4"]  |
+-----+--------------------------------+
1 row selected (0.202 seconds)
0: jdbc:drill:schema=dfs.tmp> select * from `f2.json`;
+-----+------+
| id  | arr  |
+-----+------+
| 2   | []   |
+-----+------+
1 row selected (0.199 seconds)

{noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message