hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-11405) Add early termination for recursion in StatsRulesProcFactory$FilterStatsRule.evaluateExpression for OR expression
Date Mon, 03 Aug 2015 02:40:04 GMT

     [ https://issues.apache.org/jira/browse/HIVE-11405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Prasanth Jayachandran updated HIVE-11405:
-----------------------------------------
    Attachment: HIVE-11405.2.patch

Fixed failing tests. This patch actually revealed a bug in previous stats annotation implementation
for OR conditions. 

Previous implementations:
{code}
(
  (
    (
      (
        (
          (
            ((key <> '302') and (key <> '311')) - 500 rows
            and
            ((value <> 'val_50') or (key > '1')) - 666 rows (This is wrong. this
should never exceed max number of rows)
          )
          and
          (key < '400') - 222 rows
        )
        and (key <> '305') - 222 rows
      )
      and (key <> '14') - 222 rows
    )
    and key is not null - 111 rows
  )
  and value is not null - 56 rows
)
{code}

Current implementation:
{code}
(
  (
    (
      (
        (
          (
            ((key <> '302') and (key <> '311')) - 500 rows
            and
            ((value <> 'val_50') or (key > '1')) - 500 rows
          )
          and
          (key < '400') - 166 rows
        )
        and (key <> '305') - 166 rows
      )
      and (key <> '14') - 166 rows
    )
    and key is not null - 83 rows
  )
  and value is not null - 42 rows
)
{code}

> Add early termination for recursion in StatsRulesProcFactory$FilterStatsRule.evaluateExpression
 for OR expression
> ------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-11405
>                 URL: https://issues.apache.org/jira/browse/HIVE-11405
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Hari Sankar Sivarama Subramaniyan
>            Assignee: Prasanth Jayachandran
>         Attachments: HIVE-11405.1.patch, HIVE-11405.2.patch, HIVE-11405.patch
>
>
> Thanks to [~gopalv] for uncovering this issue as part of HIVE-11330.  Quoting him,
> "The recursion protection works well with an AND expr, but it doesn't work against
> (OR a=1 (OR a=2 (OR a=3 (OR ...)
> since the for the rows will never be reduced during recursion due to the nature of the
OR.
> We need to execute a short-circuit to satisfy the OR properly - no case which matches
a=1 qualifies for the rest of the filters.
> Recursion should pass in the numRows - branch1Rows for the branch-2."



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message