beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Innocent (JIRA)" <>
Subject [jira] [Assigned] (BEAM-3714) should create a forward-only, read-only result set
Date Thu, 01 Mar 2018 00:45:00 GMT


Innocent reassigned BEAM-3714:

    Assignee: Innocent  (was: Jean-Baptiste Onofré)

> should create a forward-only, read-only result set
> ----------------------------------------------------------------
>                 Key: BEAM-3714
>                 URL:
>             Project: Beam
>          Issue Type: Bug
>          Components: io-java-jdbc
>            Reporter: Eugene Kirpichov
>            Assignee: Innocent
>            Priority: Major
> [] -
a user is trying to load a large table from MySQL, and the MySQL JDBC driver requires special
measures when loading large result sets.
> JdbcIO currently calls simply "connection.prepareStatement(query)"
- it should specify type TYPE_FORWARD_ONLY and concurrency CONCUR_READ_ONLY - these values
should always be used.
> Seems that different databases have different requirements for streaming result sets.
> E.g. MySQL requires setting fetch size; PostgreSQL says "The Connection must not be in
autocommit mode."
. Oracle, I think, doesn't have any special requirements but I don't know. Fetch size should
probably still be set to a reasonably large value.
> Seems that the common denominator of these requirements is: set fetch size to a reasonably
large but not maximum value; disable autocommit (there's nothing to commit in read() anyway).

This message was sent by Atlassian JIRA

View raw message