sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Attila Szabo <mau...@apache.org>
Subject Re: Review Request 58477: support DB2 xml data type when sqoop import with parquet
Date Wed, 17 May 2017 12:21:01 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58477/#review175237
-----------------------------------------------------------




src/java/org/apache/sqoop/manager/ConnManager.java
Line 211 (original), 211-212 (patched)
<https://reviews.apache.org/r/58477/#comment248691>

    Hey Angela,
    
    I'm a bit concerned here.
    According to the specs of java.sql.Types.OTHER
    
    """The constant in the Java programming language that indicates that the SQL type is database-specific
and gets mapped to a Java object that can be accessed via the methods getObject and setObject."""
    
    thus I'm a bit unsure, if it is the best thing to do representing it directly as a String
object/entity.
    
    Especially your change would like to fix something DB2 related, although your changes
affecting ConnManager, which is quite a central part of the whole Sqoop system (and this affecting
all the RDBMS implementations more or less).
    
    I have two requests for you:
    A. could you please double check this with someone from the Avro/Parquet team that every
XML/Unicode/UTF-8 character could be stored in the Type.STRING "column"?  One connection point
could be Gabor Szadovszky (https://github.com/gszadovszky). AFAIK he's been quite active in
the Avro/Parquet community recently, and working in the same hours as I do.
    B. could you please move this change to the DB2Manager.java (IMHO this change would be
better suited there, as right now the task is only about to handle XML types for Parquet).
    
    Thanks in advance!


- Attila Szabo


On May 8, 2017, 7:35 a.m., Ying Cao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58477/
> -----------------------------------------------------------
> 
> (Updated May 8, 2017, 7:35 a.m.)
> 
> 
> Review request for Sqoop, Boglarka Egyed, Attila Szabo, and Szabolcs Vasas.
> 
> 
> Bugs: SQOOP-3173
>     https://issues.apache.org/jira/browse/SQOOP-3173
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> SQOOP-3173 : support DB2 xml data type when sqoop import with parquet
> 
> 
> Diffs
> -----
> 
>   src/java/org/apache/sqoop/manager/ConnManager.java 1811ce0 
>   src/test/org/apache/sqoop/manager/db2/DB2XmlTypeParquetImportManualTest.java PRE-CREATION

> 
> 
> Diff: https://reviews.apache.org/r/58477/diff/3/
> 
> 
> Testing
> -------
> 
> UT passed by manually
> 
> This is third-part test, I ran it by :
> 
> ant clean test -Dthirdparty=true -Dsqoop.thirdparty.lib.dir=3rdparty -Dsqoop.test.db2.connectstring.host_url=connectstring
-Dsqoop.test.db2.connectstring.database=databasename -Dsqoop.test.db2.connectstring.username=username
-Dsqoop.test.db2.connectstring.password=password -Dtestcase=DB2XmlTypeParquetImportManualTest
> 
> 
> Thanks,
> 
> Ying Cao
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message