phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zz d <zzd7...@gmail.com>
Subject Phoenix Client create too many threads
Date Sat, 19 Sep 2015 13:32:52 GMT
Version: phoenix-4.5.0-HBase-0.98

Program Exception:

```
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:714)
        at
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
        at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1360)
        at
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:132)
        at
org.apache.hadoop.hbase.client.HTable.coprocessorService(HTable.java:1625)
        at
org.apache.hadoop.hbase.client.HTable.coprocessorService(HTable.java:1598)
        at
org.apache.phoenix.cache.ServerCacheClient.removeServerCache(ServerCacheClient.java:308)
        at
org.apache.phoenix.cache.ServerCacheClient.access$000(ServerCacheClient.java:82)
```

I found that the program had created too many threads.

I read the HBase code and found the max threads number is determined by
`hbase.htable.threads.max`

```
public static ThreadPoolExecutor getDefaultExecutor(Configuration conf) {
    int maxThreads = conf.getInt("hbase.htable.threads.max",
Integer.MAX_VALUE);
    if (maxThreads == 0) {
      maxThreads = 1; // is there a better default?
    }
    long keepAliveTime = conf.getLong("hbase.htable.threads.keepalivetime",
60);

    // Using the "direct handoff" approach, new threads will only be created
    // if it is necessary and will grow unbounded. This could be bad but in
HCM
    // we only create as many Runnables as there are region servers. It
means
    // it also scales when new region servers are added.
    ThreadPoolExecutor pool = new ThreadPoolExecutor(1, maxThreads,
keepAliveTime, TimeUnit.SECONDS,
        new SynchronousQueue<Runnable>(),
Threads.newDaemonThreadFactory("htable"));
    ((ThreadPoolExecutor) pool).allowCoreThreadTimeOut(true);
    return pool;
  }
```

This parameter can be found in phoenix code and its document here:
https://phoenix.apache.org/secondary_indexingha.html

I set the parameter in `hbase-site.xml` and restart the hbase. I also use
the `hbase-site.xml` in client side, but the threads number in my client do
not reduce.

How can I control the threads in client?

Thanks !

Mime
View raw message