phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zz d <zzd7...@gmail.com>
Subject Re: Phoenix Client create too many threads
Date Sat, 19 Sep 2015 15:34:15 GMT
Thanks.

Yes, I am sure.

2015-09-19 23:23 GMT+08:00 anil gupta <anilgupta84@gmail.com>:

> Please make sure that hbase-site.xml is in classpath of your app.
>
> On Sat, Sep 19, 2015 at 6:32 AM, zz d <zzd7zzd@gmail.com> wrote:
>
>> Version: phoenix-4.5.0-HBase-0.98
>>
>> Program Exception:
>>
>> ```
>> java.lang.OutOfMemoryError: unable to create new native thread
>>         at java.lang.Thread.start0(Native Method)
>>         at java.lang.Thread.start(Thread.java:714)
>>         at
>> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
>>         at
>> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1360)
>>         at
>> java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:132)
>>         at
>> org.apache.hadoop.hbase.client.HTable.coprocessorService(HTable.java:1625)
>>         at
>> org.apache.hadoop.hbase.client.HTable.coprocessorService(HTable.java:1598)
>>         at
>> org.apache.phoenix.cache.ServerCacheClient.removeServerCache(ServerCacheClient.java:308)
>>         at
>> org.apache.phoenix.cache.ServerCacheClient.access$000(ServerCacheClient.java:82)
>> ```
>>
>> I found that the program had created too many threads.
>>
>> I read the HBase code and found the max threads number is determined by
>> `hbase.htable.threads.max`
>>
>> ```
>> public static ThreadPoolExecutor getDefaultExecutor(Configuration conf) {
>>     int maxThreads = conf.getInt("hbase.htable.threads.max",
>> Integer.MAX_VALUE);
>>     if (maxThreads == 0) {
>>       maxThreads = 1; // is there a better default?
>>     }
>>     long keepAliveTime =
>> conf.getLong("hbase.htable.threads.keepalivetime", 60);
>>
>>     // Using the "direct handoff" approach, new threads will only be
>> created
>>     // if it is necessary and will grow unbounded. This could be bad but
>> in HCM
>>     // we only create as many Runnables as there are region servers. It
>> means
>>     // it also scales when new region servers are added.
>>     ThreadPoolExecutor pool = new ThreadPoolExecutor(1, maxThreads,
>> keepAliveTime, TimeUnit.SECONDS,
>>         new SynchronousQueue<Runnable>(),
>> Threads.newDaemonThreadFactory("htable"));
>>     ((ThreadPoolExecutor) pool).allowCoreThreadTimeOut(true);
>>     return pool;
>>   }
>> ```
>>
>> This parameter can be found in phoenix code and its document here:
>> https://phoenix.apache.org/secondary_indexingha.html
>>
>> I set the parameter in `hbase-site.xml` and restart the hbase. I also use
>> the `hbase-site.xml` in client side, but the threads number in my client do
>> not reduce.
>>
>> How can I control the threads in client?
>>
>> Thanks !
>>
>
>
>
> --
> Thanks & Regards,
> Anil Gupta
>

Mime
View raw message