storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sai Dilip Reddy Kiralam <dkira...@aadhya-analytics.com>
Subject Re: Storm topology using all the Max connections of db
Date Wed, 04 May 2016 05:52:28 GMT
Hi,

I just added

a statement
* // config.setMaximumPoolSize(50);*
    public synchronized void prepare() {
        if(dataSource == null) {
            Properties properties = new Properties();
            properties.putAll(configMap);
            HikariConfig config = new HikariConfig(properties);
          *  config.setMaximumPoolSize(50);*
            this.dataSource = new HikariDataSource(config);
            this.dataSource.setAutoCommit(false);
        }
    }


and I'm getting the following error

java.lang.RuntimeException: Fail-fast during pool initialization at
com.zaxxer.hikari.pool.HikariPool.fillPool(HikariPool.java:475) at
com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:159) at
com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:112) at
com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:78) at
com.aail.config.MConnectionProvider.prepare(MConnectionProvider.java:27) at
com.aail.storm.bolts.AbstractJdbcBolt.prepare(AbstractJdbcBolt.java:34) at
com.aail.storm.bolts.Inserts.prepare(Inserts.java:290) at
backtype.storm.daemon.executor$fn__6647$fn__6659.invoke(executor.clj:692)
at backtype.storm.util$async_loop$fn__459.invoke(util.clj:461) at
clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:745)
Caused by: org.postgresql.util.PSQLException: Connection attempt timed out.
at org.postgresql.Driver$ConnectThread.getResult(Driver.java:372) at
org.postgresql.Driver.connect(Driver.java:284) at
java.sql.DriverManager.getConnection(DriverManager.java:664) at
java.sql.DriverManager.getConnection(DriverManager.java:247) at
org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:99)
at
org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:82)
at com.zaxxer.hikari.pool.HikariPool.addConnection(HikariPool.java:398) at
com.zaxxer.hikari.pool.HikariPool.fillPool(HikariPool.java:474) ... 10 more




*Best regards,*

*K.Sai Dilip Reddy.*

On Wed, May 4, 2016 at 9:53 AM, Sai Dilip Reddy Kiralam <
dkiralam@aadhya-analytics.com> wrote:

> Hi,
>
> Thank you Spico Florin.
>
>
>
> *Best regards,*
>
> *K.Sai Dilip Reddy.*
>
> On Wed, May 4, 2016 at 1:18 AM, Spico Florin <spicoflorin@gmail.com>
> wrote:
>
>> hi!
>>   please have a look
>> https://github.com/apache/storm/tree/master/external/storm-jdbc where
>> you hve to implemnt thw interface connectionprovider interface. you have to
>> dins a third paty librry that provides conection pooloing for postgresql
>> and use that library to implement the interface connection provider. i hope
>> that other community member will give you some other ideas.
>>  i hope that these help.
>> regards,
>>  florin
>>
>> On Tuesday, May 3, 2016, Sai Dilip Reddy Kiralam <
>> dkiralam@aadhya-analytics.com> wrote:
>>
>>> to implemI'm using storm jdbc for connecting to db.how can I use pooling
>>> on codes on topology.please share information on connection pooling used on
>>> storm topologies.
>>>
>>>
>>>
>>> *Best regards,*
>>>
>>> *K.Sai Dilip Reddy.*
>>>
>>> On Thu, Apr 28, 2016 at 12:28 PM, Sai Dilip Reddy Kiralam <
>>> dkiralam@aadhya-analytics.com> wrote:
>>>
>>>> Hi Spico,
>>>>
>>>> I use 9 bolts with parallelism of 1 and with 1 task for each
>>>> bolt(default) and I'm not using any ConnectionPool for connecting to
>>>> postgres.Just using jdbc classe examples here is source example
>>>> http://storm.apache.org/releases/2.0.0-SNAPSHOT/storm-jdbc.html.
>>>> I will check by using the ConnectionPool.
>>>>
>>>> Thank you.
>>>>
>>>>
>>>>
>>>>
>>>> *Best regards,*
>>>>
>>>> *K.Sai Dilip Reddy.*
>>>>
>>>> On Thu, Apr 28, 2016 at 12:19 PM, Spico Florin <spicoflorin@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello!
>>>>>    How many tasks do you have for inserting the data to your database?
>>>>> Are you using ConnectionPool for connecting to Postgres? If your number
of
>>>>> task superseeds the number of max connections provided in connection
pool
>>>>> then your have a problem.
>>>>> Please also check the number of max connections that your db accepts.
>>>>> I hope that these help.
>>>>> Regards,
>>>>>   Florin
>>>>>
>>>>> On Thu, Apr 28, 2016 at 7:09 AM, Sai Dilip Reddy Kiralam <
>>>>> dkiralam@aadhya-analytics.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I have topology that make connection with postgresdb and insert the
>>>>>> fields into tables of a test db.my topology is working fine but when
I
>>>>>> submit the topology it is establishing all the connections of db.but
I
>>>>>> don’t know why it is taking all the max connections.
>>>>>>
>>>>>>
>>>>>> below attached the pics of pg_stat_activity.
>>>>>>
>>>>>>
>>>>>> *Best regards,*
>>>>>>
>>>>>> *K.Sai Dilip Reddy.*
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>

Mime
View raw message