sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Leech (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-456) Generated Java class does not distinguish date and timestamp in Oracle DB
Date Tue, 02 Jul 2013 04:29:21 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13697496#comment-13697496
] 

Jonathan Leech commented on SQOOP-456:
--------------------------------------

Problem as I see it even with system properties / jdbc driver hacks / column mappings, is
that with splits on an Oracle date column, Sqoop's generated sql statements have something
like to_timestamp('format', 'the date of the split as text') and Oracle ain't never gonna
treat that as a date no matter what, at least for the purposes of using either a date in an
index or to go to a range of date partitions.

I may be missing something but my only way to avoid full scans on tables was to manually split
the data with multiple invocations of Sqoop with different output directories and --where
clauses, which worked but was annoying.

The different flavors of -D props for the Oracle driver would work if the splits queries used
a bound parameter with a java.sql.Timestamp instead of to_timestamp, with the added benefit
of a slight gain from Oracle not having to hard parse, and even crazy DBA stuff like query
baselining etc. could then be possible. For that matter another command-line arg --hints would
be pretty useful.

If you reopen this and let me loose on your codebase I will fix it for you. I would love to
make it that much easier for people to migrate from Oracle.
                
> Generated Java class does not distinguish date and timestamp in Oracle DB 
> --------------------------------------------------------------------------
>
>                 Key: SQOOP-456
>                 URL: https://issues.apache.org/jira/browse/SQOOP-456
>             Project: Sqoop
>          Issue Type: Bug
>          Components: codegen
>            Reporter: Cheolsoo Park
>            Priority: Minor
>
> In genereated Java class, both date and timestamp are considered as timestamps. For examples,
the following getter method is generated for a date column:
> {code:xml} 
> public void readFields(ResultSet __dbResults) throws SQLException {
>   this.__cur_result_set = __dbResults;
>   this.DATA_COL0 = JdbcWritableBridge.readTimestamp(1, __dbResults);
> }
> {code}
> This happens because both date and timestamp is labeled as Types.TIMESTAMP by ConnManager.
The fix seems straightforward. In ClassWriter, the new method getColTypeNamesForTable introduced
by Bilung at SQOOP-352 should replace getColTypes(). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message