sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Richard (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-1400) Failed to import data using mysql-connector-java-5.1.17.jar
Date Thu, 24 Jul 2014 03:03:38 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14072733#comment-14072733
] 

Richard commented on SQOOP-1400:
--------------------------------

There is the function initOptionDefaults in MySQLManager.java
==========================================================
@Override
  protected void initOptionDefaults() {
    if (options.getFetchSize() == null) {
      LOG.info("Preparing to use a MySQL streaming resultset.");
      options.setFetchSize(Integer.MIN_VALUE);
    } else if (
        !options.getFetchSize().equals(Integer.MIN_VALUE)
        && !options.getFetchSize().equals(0)) {
      LOG.info("Argument '--fetch-size " + options.getFetchSize()
          + "' will probably get ignored by MySQL JDBC driver.");
      // see also
      // http://dev.mysql.com/doc/refman/5.5/en
      //                       /connector-j-reference-implementation-notes.html
    }
  }
==========================================================
If there is no fetch-size set in command line, the default fetch size will be set as Integer.MIN_VALUE
(0x80000000).
For Statement.java in rt.jar, the default value is 0, and the negative (Integer.MIN_VALUE)
is not satisfied, refer to the comment of setFetchSize.
==========================================================
/**
     * Gives the JDBC driver a hint as to the number of rows that should
     * be fetched from the database when more rows are needed for
     * <code>ResultSet</code> objects genrated by this <code>Statement</code>.
     * If the value specified is zero, then the hint is ignored.
     * The default value is zero.
     *
     * @param rows the number of rows to fetch
     * @exception SQLException if a database access error occurs,
     * this method is called on a closed <code>Statement</code> or the
     *        condition  <code>rows >= 0</code> is not satisfied.
     * @since 1.2
     * @see #getFetchSize
     */
    void setFetchSize(int rows) throws SQLException;
==========================================================

> Failed to import data using mysql-connector-java-5.1.17.jar
> -----------------------------------------------------------
>
>                 Key: SQOOP-1400
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1400
>             Project: Sqoop
>          Issue Type: Bug
>          Components: sqoop2-client
>    Affects Versions: 1.4.5
>         Environment: centos-6.4
>            Reporter: Richard
>
> sqoop 1.4.5 + mysql-connector-java-5.1.31.jar, which is the latest version (works fine)
> sqoop 1.4.5 + mysql-connector-java-5.1.17.jar, which is the default version on centos
6.4 installed by rpm (fails with error message showed below)
> ==========================================================
> 14/07/24 10:44:48 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `test1`
AS t LIMIT 1
> 14/07/24 10:44:48 ERROR manager.SqlManager: Error reading from database: java.sql.SQLException:
Streaming result set com.mysql.jdbc.RowDataDynamic@1cfabc3a is still active. No statements
may be issued when any streaming result sets are open and in use on a given connection. Ensure
that you have called .close() on any active streaming result sets before attempting more queries.
> java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@1cfabc3a is
still active. No statements may be issued when any streaming result sets are open and in use
on a given connection. Ensure that you have called .close() on any active streaming result
sets before attempting more queries.
> 	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:934)
> 	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:931)
> 	at com.mysql.jdbc.MysqlIO.checkForOutstandingStreamingData(MysqlIO.java:2735)
> 	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1899)
> 	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151)
> 	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2619)
> 	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2569)
> 	at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1521)
> 	at com.mysql.jdbc.ConnectionImpl.getMaxBytesPerChar(ConnectionImpl.java:3003)
> 	at com.mysql.jdbc.Field.getMaxBytesPerCharacter(Field.java:602)
> 	at com.mysql.jdbc.ResultSetMetaData.getPrecision(ResultSetMetaData.java:445)
> 	at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:285)
> 	at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
> 	at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:226)
> 	at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
> 	at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1773)
> 	at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1578)
> 	at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
> 	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
> 	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:601)
> 	at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> 	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
> 	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
> 	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> 	at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> 14/07/24 10:44:48 ERROR tool.ImportTool: Encountered IOException running import job:
java.io.IOException: No columns to generate for ClassWriter
> 	at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1584)
> 	at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
> 	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
> 	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:601)
> 	at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> 	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
> 	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
> 	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> 	at org.apache.sqoop.Sqoop.main(Sqoop.java:236)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message