nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <bbe...@gmail.com>
Subject Re: RedisConnectionPoolService - Incorrectly Attempts to Connect to localhost
Date Sat, 26 Jan 2019 14:37:32 GMT
Hello,

This should be fixed in master, just hasn’t been released yet.

https://issues.apache.org/jira/browse/NIFI-5830

Bryan

On Sat, Jan 26, 2019 at 8:04 AM <Josef.Zahner1@swisscom.com> wrote:

> FYI
>
>
>
> I’ve just created a bugreport – I don’t think this is expected behavior.
> https://issues.apache.org/jira/browse/NIFI-5976
>
>
>
> I’ve also described a workaround in there.
>
>
>
> Cheers Josef
>
>
>
>
>
> *From: *"Zahner Josef, GSB-LR-TRW-LI" <Josef.Zahner1@swisscom.com>
> *Date: *Saturday, 26 January 2019 at 13:07
> *To: *"users@nifi.apache.org" <users@nifi.apache.org>
> *Subject: *Re: RedisConnectionPoolService - Incorrectly Attempts to
> Connect to localhost
>
>
>
> Hi Jim
>
>
>
> Did you solve the issue? I have exactly the same behavior… I wanted to try
> Redis but ended up with this error.
>
>
>
> Cheers Josef
>
>
>
>
>
> On 2018/11/29 21:54:25, "Williams, Jim" <j...@alertlogic.com> wrote:
>
> > Hello,>
>
> >
>
> >  >
>
> >
>
> > I'm trying to set up the>
>
> > RedisConnectionPoolService/RedisDistributedMapCacheClientService.>
>
> >
>
> >  >
>
> >
>
> > Some basic observations:>
>
> >
>
> > *          This is a standalone Nifi 1.8.0 server>
>
> > *          SELinux is disabled on the server>
>
> > *          There are no iptables rules configured for blocking on the
> server>
>
> > *          I am able to resolve the hostname of the Redis server to an
> IP>
>
> > address on the Nifi server>
>
> > *          I can connect to the Redis server to the Nifi server using
> telnet>
>
> >
>
> >  >
>
> >
>
> > The stack trace I see when the services are started is:>
>
> >
>
> > 2018-11-29 21:16:03,527 WARN [Timer-Driven Process Thread-8]>
>
> > o.a.n.controller.tasks.ConnectableTask Administratively Yielding>
>
> > PutDistributedMapCache[id=0167105c-4a54-1adf-cb8d-1b45de7f0c99] due to>
>
> > uncaught Exception: org.springframework.data.redis.RedisConnection>
>
> >
>
> > FailureException: Cannot get Jedis connection; nested exception is>
>
> > redis.clients.jedis.exceptions.JedisConnectionException: Could not get
> a>
>
> > resource from the pool>
>
> >
>
> > org.springframework.data.redis.RedisConnectionFailureException: Cannot
> get>
>
> > Jedis connection; nested exception is>
>
> > redis.clients.jedis.exceptions.JedisConnectionException: Could not get
> a>
>
> > resource from the pool>
>
> >
>
> >         at>
>
> >
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetch>
>
>
> > JedisConnector(JedisConnectionFactory.java:281)>
>
> >
>
> >         at>
>
> >
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getCo>
>
>
> > nnection(JedisConnectionFactory.java:464)>
>
> >
>
> >         at>
>
> >
> org.apache.nifi.redis.service.RedisConnectionPoolService.getConnection(Redis>
>
>
> > ConnectionPoolService.java:89)>
>
> >
>
> >         at sun.reflect.GeneratedMethodAccessor580.invoke(Unknown
> Source)>
>
> >
>
> >         at>
>
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl>
>
>
> > .java:43)>
>
> >
>
> >         at java.lang.reflect.Method.invoke(Method.java:498)>
>
> >
>
> >         at>
>
> >
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandle>
>
>
> > r.invoke(StandardControllerServiceInvocationHandler.java:84)>
>
> >
>
> >         at com.sun.proxy.$Proxy98.getConnection(Unknown Source)>
>
> >
>
> >         at>
>
> >
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.withConn>
>
>
> > ection(RedisDistributedMapCacheClientService.java:343)>
>
> >
>
> >         at>
>
> >
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.put(Redi>
>
>
> > sDistributedMapCacheClientService.java:189)>
>
> >
>
> >         at sun.reflect.GeneratedMethodAccessor579.invoke(Unknown
> Source)>
>
> >
>
> >         at>
>
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl>
>
>
> > .java:43)>
>
> >
>
> >         at java.lang.reflect.Method.invoke(Method.java:498)>
>
> >
>
> >         at>
>
> >
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandle>
>
>
> > r.invoke(StandardControllerServiceInvocationHandler.java:84)>
>
> >
>
> >         at com.sun.proxy.$Proxy96.put(Unknown Source)>
>
> >
>
> >         at>
>
> >
> org.apache.nifi.processors.standard.PutDistributedMapCache.onTrigger(PutDist>
>
>
> > ributedMapCache.java:202)>
>
> >
>
> >         at>
>
> >
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java>
>
>
> > :27)>
>
> >
>
> >         at>
>
> >
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessor>
>
>
> > Node.java:1165)>
>
> >
>
> >         at>
>
> >
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java>
>
>
> > :203)>
>
> >
>
> >         at>
>
> >
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(Timer>
>
>
> > DrivenSchedulingAgent.java:117)>
>
> >
>
> >         at>
>
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)>
>
> >
>
> >         at
> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)>
>
> >
>
> >         at>
>
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$>
>
>
> > 301(ScheduledThreadPoolExecutor.java:180)>
>
> >
>
> >         at>
>
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Sch>
>
>
> > eduledThreadPoolExecutor.java:294)>
>
> >
>
> >         at>
>
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11>
>
>
> > 49)>
>
> >
>
> >         at>
>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6>
>
>
> > 24)>
>
> >
>
> >         at java.lang.Thread.run(Thread.java:748)>
>
> >
>
> > Caused by: redis.clients.jedis.exceptions.JedisConnectionException:
> Could>
>
> > not get a resource from the pool>
>
> >
>
> >         at redis.clients.util.Pool.getResource(Pool.java:53)>
>
> >
>
> >         at
> redis.clients.jedis.JedisPool.getResource(JedisPool.java:226)>
>
> >
>
> >         at redis.clients.jedis.JedisPool.getResource(JedisPool.java:16)>
>
> >
>
> >         at>
>
> >
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetch>
>
>
> > JedisConnector(JedisConnectionFactory.java:271)>
>
> >
>
> >         ... 26 common frames omitted>
>
> >
>
> > Caused by: redis.clients.jedis.exceptions.JedisConnectionException:>
>
> > java.net.ConnectException: Connection refused (Connection refused)>
>
> >
>
> >         at redis.clients.jedis.Connection.connect(Connection.java:207)>
>
> >
>
> >         at
> redis.clients.jedis.BinaryClient.connect(BinaryClient.java:93)>
>
> >
>
> >         at
> redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1767)>
>
> >
>
> >         at>
>
> > redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:106)>
>
> >
>
> >         at>
>
> >
> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.jav>
>
>
> > a:868)>
>
> >
>
> >         at>
>
> >
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPo>
>
>
> > ol.java:435)>
>
> >
>
> >         at>
>
> >
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPo>
>
>
> > ol.java:363)>
>
> >
>
> >         at redis.clients.util.Pool.getResource(Pool.java:49)>
>
> >
>
> >         ... 29 common frames omitted>
>
> >
>
> > Caused by: java.net.ConnectException: Connection refused (Connection>
>
> > refused)>
>
> >
>
> >         at java.net.PlainSocketImpl.socketConnect(Native Method)>
>
> >
>
> >         at>
>
> >
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)>
>
>
> >
>
> >         at>
>
> >
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.ja>
>
>
> > va:206)>
>
> >
>
> >         at>
>
> >
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)>
>
> >
>
> >         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)>
>
> >
>
> >         at java.net.Socket.connect(Socket.java:589)>
>
> >
>
> >         at redis.clients.jedis.Connection.connect(Connection.java:184)>
>
> >
>
> >         ... 36 common frames omitted>
>
> >
>
> >  >
>
> >
>
> > A tcpdump shows an attempt to connect to localhost:6379 rather than the>
>
> > Redis server ip address, with the result being a TCP RST.  (Thus the>
>
> > "Connection refused" seen in the stack dump.)>
>
> >
>
> >  >
>
> >
>
> > [root@REDACTED logs]# tcpdump -i any -s 1500 -X port 6379>
>
> >
>
> > tcpdump: verbose output suppressed, use -v or -vv for full protocol
> decode>
>
> >
>
> > listening on any, link-type LINUX_SLL (Linux cooked), capture size 1500>
>
> > bytes>
>
> >
>
> > 21:16:01.523226 IP localhost.39300 > localhost.6379: Flags [S], seq>
>
> > 2608543115, win 43690, options [mss 65495,sackOK,TS val 2641616382 ecr>
>
> > 0,nop,wscale 7], length 0>
>
> >
>
> >      0x0000:  4500 003c 2df9 4000 ff06 4fc0 7f00 0001  E..<-.@...O.....>
>
> >
>
> >      0x0010:  7f00 0001 9984 18eb 9b7b 358b 0000 0000  .........{5.....>
>
> >
>
> >      0x0020:  a002 aaaa fe30 0000 0204 ffd7 0402 080a  .....0..........>
>
> >
>
> >      0x0030:  9d73 ddfe 0000 0000 0103 0307 0000 0000  .s..............>
>
> >
>
> >      0x0040:  0000 0000 0000 0000 0000 0000            ............>
>
> >
>
> > 21:16:01.523234 IP localhost.6379 > localhost.39300: Flags [R.], seq 0,
> ack>
>
> > 2608543116, win 0, length 0>
>
> >
>
> >      0x0000:  4500 0028 0000 4000 ff06 7dcd 7f00 0001  E..(..@...}.....>
>
> >
>
> >      0x0010:  7f00 0001 18eb 9984 0000 0000 9b7b 358c  .............{5.>
>
> >
>
> >      0x0020:  5014 0000 2e57 0000 0000 0000 0000 0000  P....W..........>
>
> >
>
> >      0x0030:  0000 0000 0000 0000                      ........>
>
> >
>
> >  >
>
> >
>
> > This is the configuration for the RedisConnectionPoolService:>
>
> >
>
> >  >
>
> >
>
> >     <controllerService>>
>
> >
>
> >       <id>01671039-4a54-1adf-4b56-4ab9dcfaebb6</id>>
>
> >
>
> >       <name>RedisConnectionPoolService</name>>
>
> >
>
> >       <comment/>>
>
> >
>
> >  >
>
> > <class>org.apache.nifi.redis.service.RedisConnectionPoolService</class>>
>
> >
>
> >       <bundle>>
>
> >
>
> >         <group>org.apache.nifi</group>>
>
> >
>
> >         <artifact>nifi-redis-nar</artifact>>
>
> >
>
> >         <version>1.8.0</version>>
>
> >
>
> >       </bundle>>
>
> >
>
> >       <enabled>true</enabled>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Redis Mode</name>>
>
> >
>
> >         <value>Standalone</value>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Connection String</name>>
>
> >
>
> >         <value>REDACTED.use1.cache.amazonaws.com:6379</value>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Database Index</name>>
>
> >
>
> >         <value>0</value>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Communication Timeout</name>>
>
> >
>
> >         <value>10 seconds</value>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Cluster Max Redirects</name>>
>
> >
>
> >         <value>5</value>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Sentinel Master</name>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Password</name>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Pool - Max Total</name>>
>
> >
>
> >         <value>4</value>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Pool - Max Idle</name>>
>
> >
>
> >         <value>2</value>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Pool - Min Idle</name>>
>
> >
>
> >         <value>1</value>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Pool - Block When Exhausted</name>>
>
> >
>
> >         <value>true</value>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Pool - Max Wait Time</name>>
>
> >
>
> >         <value>10 seconds</value>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Pool - Min Evictable Idle Time</name>>
>
> >
>
> >         <value>60 seconds</value>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Pool - Time Between Eviction Runs</name>>
>
> >
>
> >         <value>30 seconds</value>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Pool - Num Tests Per Eviction Run</name>>
>
> >
>
> >         <value>-1</value>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Pool - Test On Create</name>>
>
> >
>
> >         <value>true</value>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Pool - Test On Borrow</name>>
>
> >
>
> >         <value>false</value>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Pool - Test On Return</name>>
>
> >
>
> >         <value>false</value>>
>
> >
>
> >       </property>>
>
> >
>
> >       <property>>
>
> >
>
> >         <name>Pool - Test While Idle</name>>
>
> >
>
> >         <value>true</value>>
>
> >
>
> >       </property>>
>
> >
>
> >     </controllerService>>
>
> >
>
> >  >
>
> >
>
> >  >
>
> >
>
> > I've seen where other folks have RedisDistributedMapCacheClientService>
>
> > working in their environments, so this seems to be something particular
> to>
>
> > this environment and not necessarily Nifi.  Does anyone have an idea as
> to>
>
> > what this might be or other items to check?>
>
> >
>
> >  >
>
> >
>
> >  >
>
> >
>
> > Warm regards,>
>
> >
>
> >  >
>
> >
>
> >  >
>
> >
>
> >
>
> >  <https://www.alertlogic.com/> >
>
> >
>
> > Jim Williams | Principal Database Developer>
>
> >
>
> >
>
> > O: +1 713.341.7812 | C: +1 919.523.8767 | jwilliams@alertlogic.com |>
>
> > <http://www.alertlogic.com/> alertlogic.com>
>
> > <https://twitter.com/alertlogic>>
>
> > <https://www.linkedin.com/company/alert-logic> >
>
> >
>
> >
>
> >  >
>
> >
>
> >
>
> >
>
> >  >
>
> >
>
> >
>
-- 
Sent from Gmail Mobile

Mime
View raw message