spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kohki Nishio <tarop...@gmail.com>
Subject Lock issue with SQLConf.getConf
Date Sat, 11 Sep 2021 03:20:59 GMT
Hello,
I'm running spark in local mode and seeing multiple threads showing like
below, anybody knows why it's not using a concurrent hash map  ?

---
"request-handler-dispatcher-19785" #107532 prio=5 os_prio=0
tid=0x00007fbd78036000 nid=0x4ebf runnable [0x00007fc6e83af000]
   java.lang.Thread.State: RUNNABLE
at java.util.Collections$SynchronizedMap.get(Collections.java:2586)
- locked <0x00007fc901c7d9f8> (a java.util.Collections$SynchronizedMap)
at org.apache.spark.sql.internal.SQLConf.getConf(SQLConf.scala:3750)
at
org.apache.spark.sql.internal.SQLConf.constraintPropagationEnabled(SQLConf.scala:3346)

"Executor task launch worker for task 23.0 in stage 279783.0 (TID 449746)"
#107929 daemon prio=5 os_prio=0 tid=0x00007fbe2c005000 nid=0x55f6 waiting
for monitor entry [0x00007fc6e4037000]
   java.lang.Thread.State: BLOCKED (on object monitor)
at java.util.Collections$SynchronizedMap.get(Collections.java:2586)
- waiting to lock <0x00007fc901c7d9f8> (a
java.util.Collections$SynchronizedMap)
at org.apache.spark.sql.internal.SQLConf.getConf(SQLConf.scala:3750)
at
org.apache.spark.sql.internal.SQLConf.methodSplitThreshold(SQLConf.scala:3330)

"Executor task launch worker for task 16.0 in stage 279883.0 (TID 449728)"
#107690 daemon prio=5 os_prio=0 tid=0x00007fbc60056800 nid=0x53c6 waiting
for monitor entry [0x00007fc65a3a8000]
   java.lang.Thread.State: BLOCKED (on object monitor)
at java.util.Collections$SynchronizedMap.get(Collections.java:2586)
- waiting to lock <0x00007fc901c7d9f8> (a
java.util.Collections$SynchronizedMap)
at org.apache.spark.sql.internal.SQLConf.getConf(SQLConf.scala:3750)
at
org.apache.spark.sql.internal.SQLConf.planChangeLogLevel(SQLConf.scala:3160)
at
org.apache.spark.sql.catalyst.rules.PlanChangeLogger.<init>(RuleExecutor.scala:49)

---


-- 
Kohki Nishio

Mime
View raw message