From issues-return-199781-apmail-hive-issues-archive=hive.apache.org@hive.apache.org Fri Oct 2 11:15:03 2020 Return-Path: X-Original-To: apmail-hive-issues-archive@locus.apache.org Delivered-To: apmail-hive-issues-archive@locus.apache.org Received: from mxout1-ec2-va.apache.org (mxout1-ec2-va.apache.org [3.227.148.255]) by minotaur.apache.org (Postfix) with ESMTP id B0BDF1A6D9 for ; Fri, 2 Oct 2020 11:15:02 +0000 (UTC) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-ec2-va.apache.org (ASF Mail Server at mxout1-ec2-va.apache.org) with SMTP id DAF0842A28 for ; Fri, 2 Oct 2020 11:15:01 +0000 (UTC) Received: (qmail 61519 invoked by uid 500); 2 Oct 2020 11:15:01 -0000 Delivered-To: apmail-hive-issues-archive@hive.apache.org Received: (qmail 61491 invoked by uid 500); 2 Oct 2020 11:15:01 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 61474 invoked by uid 99); 2 Oct 2020 11:15:01 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Oct 2020 11:15:01 +0000 Received: from jira-he-de.apache.org (static.172.67.40.188.clients.your-server.de [188.40.67.172]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id EC9D040F9C for ; Fri, 2 Oct 2020 11:15:00 +0000 (UTC) Received: from jira-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira-he-de.apache.org (ASF Mail Server at jira-he-de.apache.org) with ESMTP id 5EA0A7803A2 for ; Fri, 2 Oct 2020 11:15:00 +0000 (UTC) Date: Fri, 2 Oct 2020 11:15:00 +0000 (UTC) From: "Panagiotis Garefalakis (Jira)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Assigned] (HIVE-24224) Fix skipping header/footer for Hive on Tez on compressed files MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-24224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis reassigned HIVE-24224: --------------------------------------------- > Fix skipping header/footer for Hive on Tez on compressed files > -------------------------------------------------------------- > > Key: HIVE-24224 > URL: https://issues.apache.org/jira/browse/HIVE-24224 > Project: Hive > Issue Type: Bug > Reporter: Panagiotis Garefalakis > Assignee: Panagiotis Garefalakis > Priority: Major > > Compressed file with Hive on Tez returns header and footers - for both select * and select count ( * ): > {noformat} > printf "offset,id,other\n9,\"20200315 X00 1356\",123\n17,\"20200315 X00 1357\",123\nrst,rst,rst" > data.csv > hdfs dfs -put -f data.csv /apps/hive/warehouse/bz2test/bz2tbl1/ > bzip2 -f data.csv > hdfs dfs -put -f data.csv.bz2 /apps/hive/warehouse/bz2test/bz2tbl2/ > beeline -e "CREATE EXTERNAL TABLE default.bz2tst2 ( > sequence int, > id string, > other string) > ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde' > LOCATION '/apps/hive/warehouse/bz2test/bz2tbl2' > TBLPROPERTIES ( > 'skip.header.line.count'='1', > 'skip.footer.line.count'='1');" > beeline -e " > SET hive.fetch.task.conversion = none; > SELECT * FROM default.bz2tst2;" > +-------------------+--------------------+----------------+ > | bz2tst2.sequence | bz2tst2.id | bz2tst2.other | > +-------------------+--------------------+----------------+ > | offset | id | other | > | 9 | 20200315 X00 1356 | 123 | > | 17 | 20200315 X00 1357 | 123 | > | rst | rst | rst | > +-------------------+--------------------+----------------+ > {noformat} > PS: HIVE-22769 addressed the issue for Hive on LLAP. -- This message was sent by Atlassian Jira (v8.3.4#803005)