phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (PHOENIX-3744) Support snapshot scanners for MR-based queries
Date Wed, 24 May 2017 17:22:04 GMT


ASF GitHub Bot commented on PHOENIX-3744:

Github user JamesRTaylor commented on a diff in the pull request:
    --- Diff: phoenix-core/src/main/java/org/apache/phoenix/iterate/
    @@ -0,0 +1,99 @@
    +package org.apache.phoenix.iterate;
    +import org.apache.hadoop.conf.Configuration;
    +import org.apache.hadoop.fs.FileSystem;
    +import org.apache.hadoop.fs.Path;
    +import org.apache.hadoop.hbase.HConstants;
    +import org.apache.hadoop.hbase.HRegionInfo;
    +import org.apache.hadoop.hbase.HRegionLocation;
    +import org.apache.hadoop.hbase.protobuf.generated.HBaseProtos;
    +import org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos;
    +import org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils;
    +import org.apache.hadoop.hbase.snapshot.SnapshotManifest;
    +import org.apache.phoenix.compile.StatementContext;
    +import org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil;
    +import java.sql.SQLException;
    +import java.util.List;
    +public class TableMRParallelScanGrouper extends MapReduceParallelScanGrouper {
    +  private static TableMRParallelScanGrouper INSTANCE = null;
    --- End diff --
    Same comment here - how can this be a singleton?

> Support snapshot scanners for MR-based queries
> ----------------------------------------------
>                 Key: PHOENIX-3744
>                 URL:
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: James Taylor
>            Assignee: Akshita Malhotra
>         Attachments: PHOENIX-3744.patch
> HBase support scanning over snapshots, with a SnapshotScanner that accesses the region
directly in HDFS. We should make sure that Phoenix can support that.
> Not sure how we'd want to decide when to run a query over a snapshot. Some ideas:
> - if there's an SCN set (i.e. the query is running at a point in time in the past)
> - if the memstore is empty
> - if the query is being run at a timestamp earlier than any memstore data
> - as a config option on the table
> - as a query hint
> - based on some kind of optimizer rule (i.e. based on estimated # of bytes that will
be scanned)
> Phoenix typically runs a query at the timestamp at which it was compiled. Any data committed
after this time should not be seen while a query is running.

This message was sent by Atlassian JIRA

View raw message