spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From linkpatrickliu <>
Subject RE: SparkSQL 1.1 hang when "DROP" or "LOAD"
Date Tue, 16 Sep 2014 07:17:06 GMT
Hi, Hao Cheng.

I have done other tests. And the result shows the thriftServer can connect
to Zookeeper.

However, I found some more interesting things. And I think I have found a

Test procedure:
(0) Use beeline to connect to thriftServer.
(1) Switch database "use dw_op1"; (OK)
The logs show that the thriftServer connected with Zookeeper and acquired

(2) Drop table "drop table src;" (Blocked)
The logs show that the thriftServer is "acquireReadWriteLocks".

The reason why I cannot drop table src is because the first SQL "use dw_op1"
have left locks in Zookeeper  unsuccessfully released.
So when the second SQL is acquiring locks in Zookeeper, it will block.

Restart thriftServer.
Instead of switching to another database, I just drop the table in the
default database;
(0) Restart thriftServer & use beeline to connect to thriftServer.
(1) Drop table "drop table src"; (OK)
Amazing! Succeed!
(2) Drop again!  "drop table src2;" (Blocked)
Same error: the thriftServer is blocked in the "acquireReadWriteLocks"

As you can see. 
Only the first SQL requiring locks can succeed.
So I think the reason is that the thriftServer cannot release locks
correctly in Zookeeper.

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message