falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Baptiste Onofré ...@nanthrax.net>
Subject Re: [HEADS UP] Support of ActiveMQ HA/Scalability
Date Wed, 11 Feb 2015 20:07:53 GMT
By the way, who are the clients which use stomp (maybe python as it's 
the only protocol supported by python) ? Openwire should be used as 
default transport connector, at least for JMS and CMS.

Regards
JB

On 02/11/2015 08:59 PM, Idris Ali wrote:
> Yeah sure, no hurry.
>
> I think I figured it out, seems to be the memory config issue in the new
> installation.
>
> 2015-02-11 17:03:44,843 | ERROR | Could not accept connection :
> java.lang.Exception: java.lang.OutOfMemoryError: Java heap space |
> org.apache.activemq.broker.TransportConnector | Ac
>
> tiveMQ Transport Server Thread Handler: stomp://localhost:61613
>
> 2015-02-11 17:02:32,440 | ERROR | Could not accept connection :
> java.lang.Exception: java.lang.OutOfMemoryError: Java heap space |
> org.apache.activemq.broker.TransportConnector | Ac
>
> tiveMQ Transport Server Thread Handler: tcp://0.0.0.0:61616
>
>
>
> 2015-02-11 17:02:26,140 | DEBUG | Unregistering MBean
> org.apache.activemq:type=Broker,brokerName=localhost,connector=clientConnectors,connectorName=openwire,connectionViewType=clien
>
> tId,connectionName=ID_tzns4008.grid.uj1.inmobi.com-48815-1422442061155-2_1
> | org.apache.activemq.broker.jmx.ManagementContext | ActiveMQ Transport:
> tcp:///IP:40234@61616
>
> 2015-02-11 17:01:45,533 | WARN  | Failed to remove producer:
> ID:machine-48831-1423672843718-0:1:1:1 |
> org.apache.activemq.broker.TransportConnection | ActiveMQ
>
> Transport: tcp:///IP:55499@61616
>
> *java.lang.OutOfMemoryError: Java heap space   *
>
>
>
>          at
> java.util.concurrent.CopyOnWriteArrayList.remove(CopyOnWriteArrayList.java:515)
>
>
>          at
> java.util.concurrent.CopyOnWriteArraySet.remove(CopyOnWriteArraySet.java:232)
>
> Followed by lot of this messages:
>
> java.lang.OutOfMemoryError: Java heap space
>
>
> Thanks again,
>
> -Idris
>
> On Thu, Feb 12, 2015 at 1:04 AM, Jean-Baptiste Onofré <jb@nanthrax.net>
> wrote:
>
>> Hey Idris,
>>
>> if you don't mind, I would like to work on this tomorrow (my time).
>>
>> Regards
>> JB
>>
>> On 02/11/2015 07:36 PM, Idris Ali wrote:
>>
>>> Hi JB,
>>>
>>> Slightly off from this discussion, we recently upgraded ActiveMQ version
>>> to
>>> 5.9.1 from 5.4.3 to work with Falcon.
>>>
>>> With the default configs, however we see wire format negotiation timedout
>>> issue pretty often and the job fails.
>>>
>>> Failing Oozie Launcher, Main class
>>> [org.apache.oozie.action.hadoop.JavaMain], main() threw exception,
>>> javax.jms.JMSException: Wire format negotiation timeout: peer did not
>>> send his wire format.
>>> org.apache.oozie.action.hadoop.JavaMainException:
>>> javax.jms.JMSException: Wire format negotiation timeout: peer did not
>>> send his wire format.
>>>          at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:60)
>>>          at org.apache.oozie.action.hadoop.LauncherMain.run(
>>> LauncherMain.java:42)
>>>          at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:37)
>>>          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>          at sun.reflect.NativeMethodAccessorImpl.invoke(
>>> NativeMethodAccessorImpl.java:57)
>>>          at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>> DelegatingMethodAccessorImpl.java:43)
>>>          at java.lang.reflect.Method.invoke(Method.java:606)
>>>          at org.apache.oozie.action.hadoop.LauncherMapper.map(
>>> LauncherMapper.java:293)
>>>          at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>>>
>>>
>>> Initial investigation suggests something to do with Log4j deadlock.
>>> Please let me know if this is a known issue in 5.9.1 ?
>>>
>>> Thanks,
>>> -Idris
>>>
>>>
>>>
>>> On Wed, Feb 11, 2015 at 9:20 AM, Pallavi Rao <pallavi.rao@inmobi.com>
>>> wrote:
>>>
>>>   JB,
>>>> We are working on Falcon talking to ActiveMQ HA setup too.  Our initial
>>>> hunch is that as long as the right URL (with fail over scheme) is
>>>> specified
>>>> in the configuration, startup properties or cluster definition, it should
>>>> work without any code changes. Plan to run a quick test soon. Will keep
>>>> you
>>>> posted.
>>>>
>>>> Regards,
>>>> Pallavi
>>>> On Feb 10, 2015 7:51 PM, "Jean-Baptiste Onofré" <jb@nanthrax.net>
wrote:
>>>>
>>>>   Hi all,
>>>>>
>>>>> I'm working on the ActiveMQ update (to 5.10.1) and improvements on the
>>>>> broker (configuration flexibility, tuning, etc).
>>>>>
>>>>> I plan also to improve the support of some ActiveMQ features,
>>>>> especially:
>>>>> 1/ master/slave support with kahadb and JDBC locking backend
>>>>> 2/ network of brokers support to be able to have Falcon instances
>>>>> producing on one broker which can dispatch the messages (including
>>>>>
>>>> conduit
>>>>
>>>>> subscriptions support) to other brokers where Falcon instances can get
>>>>>
>>>> the
>>>>
>>>>> messages.
>>>>>
>>>>> I will send more details and patches very soon.
>>>>>
>>>>> Thanks,
>>>>> Regards
>>>>> JB
>>>>> --
>>>>> Jean-Baptiste Onofré
>>>>> jbonofre@apache.org
>>>>> http://blog.nanthrax.net
>>>>> Talend - http://www.talend.com
>>>>>
>>>>>
>>>> --
>>>> _____________________________________________________________
>>>> The information contained in this communication is intended solely for
>>>> the
>>>> use of the individual or entity to whom it is addressed and others
>>>> authorized to receive it. It may contain confidential or legally
>>>> privileged
>>>> information. If you are not the intended recipient you are hereby
>>>> notified
>>>> that any disclosure, copying, distribution or taking any action in
>>>> reliance
>>>> on the contents of this information is strictly prohibited and may be
>>>> unlawful. If you have received this communication in error, please notify
>>>> us immediately by responding to this email and then delete it from your
>>>> system. The firm is neither liable for the proper and complete
>>>> transmission
>>>> of the information contained in this communication nor for any delay in
>>>> its
>>>> receipt.
>>>>
>>>>
>>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>

-- 
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Mime
View raw message