drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rahul Raj <rahul....@option3consulting.com>
Subject Drill session and jdbc connections
Date Wed, 13 Dec 2017 10:17:09 GMT
Hi,

How is a drill session related to a drill jdbc connection instance? What
happens in a pool of connections when one connection changes the
store.format? I am seeing some mix-ups where a parquet row is written as an
array of multiple records(rather than multiple columns) when another thread
tries to create a csv file. This happens only during the race condition
between CSV and parquet formats.

Scenario:

Thread 1 for CSV creation:

Connection conn = getConnection();
conn.execute("ALTER SESSION SET `store.format`='csv'")
conn.execute("CREATE TABLE someparquet AS ...")
conn.execute("ALTER SESSION SET `store.format`='parquet'")

Thread 2 for parquet creation:

Connection conn = getConnection();
conn.execute("CREATE TABLE somecsv AS ...")

In thread 2, the parquet gets written as an ARRAY with all the fields
because of the side effect of Thread 1 setting format as CSV when they
execute in parallel.

Is it possible to have session isolation in this situation?

Regards,
Rahul

-- 
**** This email and any files transmitted with it are confidential and 
intended solely for the use of the individual or entity to whom it is 
addressed. If you are not the named addressee then you should not 
disseminate, distribute or copy this e-mail. Please notify the sender 
immediately and delete this e-mail from your system.****

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message