spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Owen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
Date Sat, 01 Aug 2015 18:00:04 GMT

    [ https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14650480#comment-14650480
] 

Sean Owen commented on SPARK-6873:
----------------------------------

[~rxin] [~lian cheng] It's still a problem. Yes I'm sure it's just a test issue, not a problem
with the code, but ideally the test must not rely on the ordering. Right now tests don't actually
pass in Java 8 because of things like ...

{code}
- show_create_table_serde *** FAILED ***
  Results do not match for show_create_table_serde:
  == Parsed Logical Plan ==
  HiveNativeCommand SHOW CREATE TABLE tmp_showcrt1
  
  == Analyzed Logical Plan ==
  result: string
  HiveNativeCommand SHOW CREATE TABLE tmp_showcrt1
  
  == Optimized Logical Plan ==
  HiveNativeCommand SHOW CREATE TABLE tmp_showcrt1
  
  == Physical Plan ==
  ExecutedCommand (HiveNativeCommand SHOW CREATE TABLE tmp_showcrt1)
  
  Code Generation: true
  == RDD ==
  result
  !== HIVE - 13 row(s) ==                                          == CATALYST - 13 row(s)
==
   CREATE EXTERNAL TABLE `tmp_showcrt1`(                           CREATE EXTERNAL TABLE `tmp_showcrt1`(
     `key` string,                                                   `key` string, 
     `value` boolean)                                                `value` boolean)
   ROW FORMAT SERDE                                                ROW FORMAT SERDE 
     'org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe'          'org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe'

   STORED BY                                                       STORED BY 
     'org.apache.hadoop.hive.ql.metadata.DefaultStorageHandler'      'org.apache.hadoop.hive.ql.metadata.DefaultStorageHandler'

   WITH SERDEPROPERTIES (                                          WITH SERDEPROPERTIES (

  !  'serialization.format'='$',                                     'field.delim'=',', 
  !  'field.delim'=',')                                              'serialization.format'='$')
   LOCATION                                                        LOCATION
     '<PATH>tmp_showcrt1'                                            '<PATH>tmp_showcrt1'
   TBLPROPERTIES (                                                 TBLPROPERTIES ( (HiveComparisonTest.scala:397)
{code}

I build with {{-Pyarn -Phive}} from master.

> Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
> ------------------------------------------------------------------------------------------
>
>                 Key: SPARK-6873
>                 URL: https://issues.apache.org/jira/browse/SPARK-6873
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL, Tests
>    Affects Versions: 1.3.1
>            Reporter: Sean Owen
>            Assignee: Cheng Lian
>            Priority: Minor
>
> As I mentioned, I've been seeing 4 test failures in Hive tests for a while, and actually
it still affects master. I think it's a superficial problem that only turns up when running
on Java 8, but still, would probably be an easy fix and good to fix.
> Specifically, here are four tests and the bit that fails the comparison, below. I tried
to diagnose this but had trouble even finding where some of this occurs, like the list of
synonyms?
> {code}
> - show_tblproperties *** FAILED ***
>   Results do not match for show_tblproperties:
> ...
>   !== HIVE - 2 row(s) ==   == CATALYST - 2 row(s) ==
>   !tmp	true                bar	bar value
>   !bar	bar value           tmp	true (HiveComparisonTest.scala:391)
> {code}
> {code}
> - show_create_table_serde *** FAILED ***
>   Results do not match for show_create_table_serde:
> ...
>    WITH SERDEPROPERTIES (                                          WITH SERDEPROPERTIES
( 
>   !  'serialization.format'='$',                                     'field.delim'=',',

>   !  'field.delim'=',')                                              'serialization.format'='$')
> {code}
> {code}
> - udf_std *** FAILED ***
>   Results do not match for udf_std:
> ...
>   !== HIVE - 2 row(s) ==                                         == CATALYST - 2 row(s)
==
>    std(x) - Returns the standard deviation of a set of numbers   std(x) - Returns the
standard deviation of a set of numbers
>   !Synonyms: stddev_pop, stddev                                  Synonyms: stddev, stddev_pop
(HiveComparisonTest.scala:391)
> {code}
> {code}
> - udf_stddev *** FAILED ***
>   Results do not match for udf_stddev:
> ...
>   !== HIVE - 2 row(s) ==                                            == CATALYST - 2 row(s)
==
>    stddev(x) - Returns the standard deviation of a set of numbers   stddev(x) - Returns
the standard deviation of a set of numbers
>   !Synonyms: stddev_pop, std                                        Synonyms: std, stddev_pop
(HiveComparisonTest.scala:391)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message