hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ferdinand Xu (JIRA)" <>
Subject [jira] [Updated] (HIVE-11297) Combine op trees for partition info generating tasks
Date Mon, 26 Jun 2017 02:59:00 GMT


Ferdinand Xu updated HIVE-11297:
       Resolution: Fixed
    Fix Version/s: 3.0.0
           Status: Resolved  (was: Patch Available)

Committed to the upstream. Thanks [~kellyzly] for the patch and [~csun] for the review.

> Combine op trees for partition info generating tasks
> ----------------------------------------------------
>                 Key: HIVE-11297
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>            Reporter: Chao Sun
>            Assignee: liyunzhang_intel
>             Fix For: 3.0.0
>         Attachments: HIVE-11297.1.patch, HIVE-11297.2.patch, HIVE-11297.3.patch, HIVE-11297.4.patch,
HIVE-11297.5.patch, HIVE-11297.6.patch, HIVE-11297.7.patch, HIVE-11297.8.patch, hive-site.xml
> Currently, for dynamic partition pruning in Spark, if a small table generates partition
info for more than one partition columns, multiple operator trees are created, which all start
from the same table scan op, but have different spark partition pruning sinks.
> As an optimization, we can combine these op trees and so don't have to do table scan
multiple times.

This message was sent by Atlassian JIRA

View raw message