sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abraham Elmahrek" <...@cloudera.com>
Subject Re: Review Request 24242: SQOOP-1405: Extra arg to enable SQL Server identity insert on export
Date Wed, 06 Aug 2014 23:18:07 GMT


> On Aug. 5, 2014, 11:27 p.m., Abraham Elmahrek wrote:
> > src/java/org/apache/sqoop/mapreduce/sqlserver/SqlServerExportBatchOutputFormat.java,
line 67
> > <https://reviews.apache.org/r/24242/diff/1/?file=650525#file650525line67>
> >
> >     Should this have a semicolon at the end of it?
> >     
> >     Also, does it make any sense to separate this out into its own statement?
> 
> Abraham Elmahrek wrote:
>     It seems like this one can be placed in the "SqlServerExportBatchRecordWriter" constructor.
Would that work?
> 
> Keegan Witt wrote:
>     This might be possible if we executed the SET IDENTITY_INSERT once per connection.
 When I tried this, however, it didn't seem to persist between PreparedStatements.  I also
tried doing a PreparedStatement.addBatch("SET IDENTITY_INSERT...") before the executeBatch()
is called.  This resulted in "com.microsoft.sqlserver.jdbc.SQLServerException: The method
addBatch() cannot take arguments on a PreparedStatement or CallableStatement."  Do you have
an idea about how this could be done specifically?

The JDBC connection should have a single session that would persist this setting. If that's
not working, then the fix you currently have makes sense.


> On Aug. 5, 2014, 11:27 p.m., Abraham Elmahrek wrote:
> > src/java/org/apache/sqoop/mapreduce/SQLServerExportDBExecThread.java, line 128
> > <https://reviews.apache.org/r/24242/diff/1/?file=650524#file650524line128>
> >
> >     Should this have a semicolon at the end of it?
> >     
> >     Also, does it make any sense to separate this out into its own statement?
> 
> Keegan Witt wrote:
>     I'm not sure I follow.  If I add a semicolon, this will be two statements in a single
PreparedStatement (which shouldn't be done).  I'd be OK breaking them into separate PreparedStatements,
but I didn't see where that would be done.  The AsyncSqlRecordWriter interface just has getPreparedStatement()
which returns a single PreparedStatement, not a collection of PreparedStatements to run. 
What am I missing?
> 
> Abraham Elmahrek wrote:
>     Good points.
>     
>     It seems like there is a "configureConnection" method that can be overriden in the
parent class "SQLServerAsyncDBExecThread". Would that work?
> 
> Keegan Witt wrote:
>     This could work if you could tell SQL Sever through a connection parameter passed
to SQLServerDriver or some property set on SQLServerConnection, but I didn't see a method
that would do that.  Suggestions?

You should be able to use separate statements. For example:
#1 Statement that executes: SET IDENTITY_INSERT ON
#2 Close statement
#3 Prepared Statement that executes batch stuff.
...

If the above doesn't work, then the solution you have is fine.


- Abraham


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24242/#review49657
-----------------------------------------------------------


On Aug. 4, 2014, 5:48 p.m., Keegan Witt wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24242/
> -----------------------------------------------------------
> 
> (Updated Aug. 4, 2014, 5:48 p.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> Adds an extra arg for SQL Server that runs SET IDENTITY_INSERT <table> ON before
exporting the data.
> 
> 
> Diffs
> -----
> 
>   src/docs/user/connectors.txt bba946a 
>   src/java/org/apache/sqoop/manager/SQLServerManager.java 534c0cc 
>   src/java/org/apache/sqoop/mapreduce/SQLServerExportDBExecThread.java 1810ba8 
>   src/java/org/apache/sqoop/mapreduce/sqlserver/SqlServerExportBatchOutputFormat.java
cc69d77 
> 
> Diff: https://reviews.apache.org/r/24242/diff/
> 
> 
> Testing
> -------
> 
> Tested that change enables inserts on tables with identity (with SQL Server 2005).
> 
> 
> Thanks,
> 
> Keegan Witt
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message