hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <>
Subject [jira] [Commented] (HIVE-21279) Avoid moving/rename operation in FileSink op for SELECT queries
Date Fri, 01 Mar 2019 06:56:00 GMT


Hive QA commented on HIVE-21279:

Here are the results of testing the latest attachment:

{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15823 tests executed
*Failed tests:*
org.apache.hadoop.hive.ql.TestTxnCommands.testSetClauseFakeColumn (batchId=327)

Test results:
Console output:
Test logs:

Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed

This message is automatically generated.

ATTACHMENT ID: 12960674 - PreCommit-HIVE-Build

> Avoid moving/rename operation in FileSink op for SELECT queries
> ---------------------------------------------------------------
>                 Key: HIVE-21279
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: Query Planning
>            Reporter: Vineet Garg
>            Assignee: Vineet Garg
>            Priority: Major
>             Fix For: 4.0.0
>         Attachments: HIVE-21279.1.patch, HIVE-21279.10.patch, HIVE-21279.11.patch, HIVE-21279.12.patch,
HIVE-21279.2.patch, HIVE-21279.3.patch, HIVE-21279.4.patch, HIVE-21279.5.patch, HIVE-21279.6.patch,
HIVE-21279.7.patch, HIVE-21279.8.patch, HIVE-21279.9.patch
> Currently at the end of a job FileSink operator moves/rename temp directory to another
directory from which FetchTask fetches result. This is done to avoid fetching potential partial/invalid
files by failed/runway tasks. This operation is expensive for cloud storage. It could be avoided
if FetchTask is passed on set of files to read from instead of whole directory.

This message was sent by Atlassian JIRA

View raw message