drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] sohami commented on a change in pull request #1459: DRILL-6731: Move the BFs aggregating work from the Foreman to the RuntimeFi…
Date Wed, 26 Sep 2018 20:26:17 GMT
sohami commented on a change in pull request #1459: DRILL-6731: Move the BFs aggregating work
from the Foreman to the RuntimeFi…
URL: https://github.com/apache/drill/pull/1459#discussion_r220710086
 
 

 ##########
 File path: exec/java-exec/src/main/java/org/apache/drill/exec/work/filter/RuntimeFilterSink.java
 ##########
 @@ -89,16 +142,27 @@ public void close() throws Exception {
 
     @Override
     public void run() {
-      while (running.get()) {
-        RuntimeFilterWritable toAggregate = rfQueue.poll();
-        if (toAggregate != null) {
-          if (aggregated != null) {
-            aggregated.aggregate(toAggregate);
-            currentBookId.incrementAndGet();
+      try {
 
 Review comment:
   I still see there are race conditions. 1) containOne() is checked outside the lock 2) Same
running flag is used by all 3 threads (netty thread, AsyncAggregateWorker and MinorFragment).
Consider a case when Netty thread checks the `running` state and it is true and was preempted,
then minor fragment thread came along and reset the running state and cleared up the queue.
After this the netty thread was scheduled and it add's the `RumtimeFilterWritable` inside
the queue. Now no-one will cleanup this newly added filter.
   
   Please see the top commit in this branch: https://github.com/sohami/drill/commits/DRILL-6731,
if you think it looks good to you then cherry-pick this commit in your PR and only squash
your original commit not the cherry-picked commit.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message