hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (HIVE-25344) Add a possibility to query Iceberg table snapshots based on the timestamp or the snapshot id
Date Tue, 27 Jul 2021 14:25:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-25344?focusedWorklogId=628497&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-628497
]

ASF GitHub Bot logged work on HIVE-25344:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 27/Jul/21 14:24
            Start Date: 27/Jul/21 14:24
    Worklog Time Spent: 10m 
      Work Description: marton-bod commented on a change in pull request #2512:
URL: https://github.com/apache/hive/pull/2512#discussion_r677502396



##########
File path: iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerWithEngine.java
##########
@@ -2271,6 +2274,116 @@ public void testStatWithPartitionedCTAS() {
     checkColStat("target", "dept");
   }
 
+  @Test
+  public void testAsOfTimestamp() throws IOException, InterruptedException {
+    Table table = prepareTableWithVersions(2);
+
+    List<Object[]> rows = shell.executeStatement(
+        "SELECT * FROM customers FOR SYSTEM_TIME AS OF '" + timestampAfterSnapshot(table,
0) + "'");
+
+    Assert.assertEquals(3, rows.size());
+
+    rows = shell.executeStatement(
+        "SELECT * FROM customers FOR SYSTEM_TIME AS OF '" + timestampAfterSnapshot(table,
1) + "'");
+
+    Assert.assertEquals(4, rows.size());
+
+    AssertHelpers.assertThrows("should throw exception", IllegalArgumentException.class,
+        "Cannot find a snapshot older than 1970-01-01 00:00:00", () -> {
+          shell.executeStatement("SELECT * FROM customers FOR SYSTEM_TIME AS OF '1970-01-01
00:00:00'");
+        });
+  }
+
+  @Test
+  public void testAsOfVersion() throws IOException, InterruptedException {
+    Table table = prepareTableWithVersions(2);
+
+    HistoryEntry first = table.history().get(0);
+    List<Object[]> rows =
+        shell.executeStatement("SELECT * FROM customers FOR SYSTEM_VERSION AS OF " + first.snapshotId());
+
+    Assert.assertEquals(3, rows.size());
+
+    HistoryEntry second = table.history().get(1);
+    rows = shell.executeStatement("SELECT * FROM customers FOR SYSTEM_VERSION AS OF " + second.snapshotId());
+
+    Assert.assertEquals(4, rows.size());
+
+    AssertHelpers.assertThrows("should throw exception", IllegalArgumentException.class,
+        "Cannot find snapshot with ID 1234", () -> {
+          shell.executeStatement("SELECT * FROM customers FOR SYSTEM_VERSION AS OF 1234");
+        });
+  }
+
+  @Test
+  public void testAsOfTimestampWithJoins() throws IOException, InterruptedException {
+    Table table = prepareTableWithVersions(4);
+
+    List<Object[]> rows = shell.executeStatement("SELECT * FROM " +
+        "customers FOR SYSTEM_TIME AS OF '" + timestampAfterSnapshot(table, 0) + "' fv, "
+
+        "customers FOR SYSTEM_TIME AS OF '" + timestampAfterSnapshot(table, 1) + "' sv "
+
+        "WHERE fv.first_name=sv.first_name");
+
+    Assert.assertEquals(4, rows.size());
+
+    rows = shell.executeStatement("SELECT * FROM " +
+         "customers FOR SYSTEM_TIME AS OF '" + timestampAfterSnapshot(table, 1) + "' sv,
" +
+         "customers FOR SYSTEM_TIME AS OF '" + timestampAfterSnapshot(table, 2) + "' tv "
+
+         "WHERE sv.first_name=tv.first_name");
+
+    Assert.assertEquals(8, rows.size());
+
+    rows = shell.executeStatement("SELECT * FROM " +
+        "customers FOR SYSTEM_TIME AS OF '" + timestampAfterSnapshot(table, 2) + "' sv, "
+
+        "customers lv " +
+        "WHERE sv.first_name=lv.first_name");
+
+    Assert.assertEquals(14, rows.size());
+  }
+
+  /**
+   * Creates the 'customers' table with the default records and creates extra snapshots by
inserting one more line
+   * into the table.
+   * @param versions The number of snapshots we want to create

Review comment:
       Yeah, I think there's a bit of a difference between snapshots and history. If you create
an empty table, it will have 1 snapshot, but 0 history - is that right? If that's true, maybe
we could reword the javadoc `The number of history elements we want to create`? (and the method
name)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 628497)
    Time Spent: 3h  (was: 2h 50m)

> Add a possibility to query Iceberg table snapshots based on the timestamp or the snapshot
id
> --------------------------------------------------------------------------------------------
>
>                 Key: HIVE-25344
>                 URL: https://issues.apache.org/jira/browse/HIVE-25344
>             Project: Hive
>          Issue Type: New Feature
>            Reporter: Peter Vary
>            Assignee: Peter Vary
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 3h
>  Remaining Estimate: 0h
>
> Implement the following commands:
> {code:java}
> SELECT * FROM t FOR SYSTEM_TIME AS OF <timestamp>;
> SELECT * FROM t FOR SYSTEM_VERSION AS OF <version>;{code}
> where SYSTEM_TIME is the Iceberg table state at the given timestamp (UTC), or SYSTEM_VERSION
is the Iceberg table snapshot id.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message