stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gayan Gunarathne <gay...@wso2.com>
Subject Re: Messaging protocol with Stratos python cartridge agent
Date Tue, 02 Sep 2014 08:44:17 GMT
Hi Paul,

Please find the example of existing message header and body.AFAIK
"event-class-name" is the header value used for distinguish the messages.

header: key expires , value 0
header: key timestamp , value 1409646637558
header: key destination , value /topic/topology
header: key persistent , value true
header: key priority , value 4
header: key event-class-name , value
org.apache.stratos.messaging.event.topology.CompleteTopologyEvent

Message body:

 {"topology":{"serviceMap":{"gateway":{"serviceName":"gateway","serviceType":"SingleTenant","clusterIdClusterMap":{"gatewaygateway.am.wso2.com.dom":{"serviceName":"gateway","clusterId":"gatewaygateway.am.wso2.com.dom","autoscalePolicyName":"auto-scl-pol1","deploymentPolicyName":"
dep-pol1","hostNames":["foo.org"],"tenantRange":"1-*","isLbCluster":false,"memberMap":{"gatewaygateway.am.wso2.com.dom-member-2":{"serviceName":"gateway","clusterId":"gatewaygateway.am.wso2.com.dom","networkPartitionId":"network-partition1","partitionId":"cloud-partition1","memberId":"gatewaygateway.am.wso2.com.dom-member-2","portMap":{"9080":{"protocol":"http","value":8080,"proxy":9080}},"memberPublicIp":"127.0.0.1
public","status":"Activated","memberIp":"127.0.0.1
private","properties":{"mykey1xxxxxxxxx":"myvalweccccccccccccccc"}},"gatewaygateway.am.wso2.com.dom-member-1":{"serviceName":"gateway","clusterId":"gatewaygateway.am.wso2.com.dom","networkPartitionId":"network-partition1","partitionId":"cloud-partition1","memberId":"gatewaygateway.am.wso2.com.dom-member-1","portMap":{"9080":{"protocol":"http","value":8080,"proxy":9080}},"memberPublicIp":"127.0.0.1
public","status":"Activated","memberIp":"127.0.0.1
private","properties":{"mykey1xxxxxxxxx":"myvalweccccccccccccccc"}},"gatewaygateway.am.wso2.com.dom5863cf8d-2746-4be1-91e9-da98c2bcb5a9":{"serviceName":"gateway","clusterId":"gatewaygateway.am.wso2.com.dom","networkPartitionId":"network-partition1","partitionId":"cloud-partition1","memberId":"gatewaygateway.am.wso2.com.dom5863cf8d-2746-4be1-91e9-da98c2bcb5a9","portMap":{},"memberPublicIp":"public
IP","memberIp":"private
IP"},"lb-member-id-1":{"serviceName":"lb","clusterId":"lbisuruh.lk.domain","networkPartitionId":"network-partition1","partitionId":"cloud-partition1","memberId":"lb-member-id-1","portMap":{},"memberPublicIp":"public
IP of apistore","memberIp":"private IP of
apistore"}},"properties":{"mykey1weeee":"myvalweeeeeeeee1","mykey1tttttttt":"myvalweetttttttttttt1"}}},"portMap":{"9080":{"protocol":"http","value":8080,"proxy":9080}},"properties":{"mykey1weeee":"myvalweeeeeeeee1","mykey1tttttttt":"myvalweetttttttttttt1"}},"apistore":{"serviceName":"apistore","serviceType":"SingleTenant","clusterIdClusterMap":{"apistorestore.am.wso2.com.doma":{"serviceName":"apistore","clusterId":"apistorestore.am.wso2.com.doma","autoscalePolicyName":"auto-scl-pol1","deploymentPolicyName":"
dep-pol1","hostNames":["foo.org"],"tenantRange":"1-*","isLbCluster":false,"memberMap":{"apistorestore.am.wso2.com.doma298e2959-a3b4-45a0-a11f-7671606158ed":{"serviceName":"apistore","clusterId":"apistorestore.am.wso2.com.doma","networkPartitionId":"network-partition1","partitionId":"cloud-partition1","memberId":"apistorestore.am.wso2.com.doma298e2959-a3b4-45a0-a11f-7671606158ed","portMap":{},"memberPublicIp":"pub
ip","status":"Activated","memberIp":"ip","lbClusterId":"lbmongolb594224131.mongolb.dom"}}}},"portMap":{"9080":{"protocol":"http","value":8080,"proxy":9080}}},"lb":{"serviceName":"lb","serviceType":"SingleTenant","clusterIdClusterMap":{"lbisuruh.lk.domain":{"serviceName":"lb","clusterId":"lbisuruh.lk.domain","autoscalePolicyName":"auto-scl-pol1","deploymentPolicyName":"
dep-pol1","hostNames":["foo.org
"],"tenantRange":"1-*","isLbCluster":false,"memberMap":{}}},"portMap":{"9080":{"protocol":"http","value":8080,"proxy":9080}}}},"initialized":false}}


Thanks,
Gayan



On Tue, Sep 2, 2014 at 1:49 PM, Paul Fremantle <paul@wso2.com> wrote:

> Can you post an example of the headers and body?
>
> Paul
>
>
> On 2 September 2014 09:17, Gayan Gunarathne <gayang@wso2.com> wrote:
>
>> Hi Paul,
>>
>> Currently we have existing java based message publisher with the headers.
>> Here we try to use those published messages with the python client. If we
>> use JSON structured message with  headers enclosed we have to change all
>> other current implementations as well. AFAIK We will distinguished the
>> messages with their header value.So my point is if we used this sort of
>> JSON structure for messages, we need to change all the current messaging
>> model.
>>
>> OTOH with the Stomp protocol with python client we were able to get those
>> header values and used those messages as it is.
>>
>> Thanks,
>> Gayan
>>
>>
>> On Tue, Sep 2, 2014 at 1:04 PM, Paul Fremantle <paul@wso2.com> wrote:
>>
>>> What headers do you need to pass?
>>>
>>> The way of doing headers with MQTT is to use a structured message (e.g.
>>> JSON) and put some headers into the JSON.
>>>
>>> Paul
>>>
>>>
>>> On 2 September 2014 07:58, Gayan Gunarathne <gayang@wso2.com> wrote:
>>>
>>>>
>>>> We did some analysis regarding the MQTT protocol as well.Following are
>>>> our findings regarding the MQTT.
>>>>
>>>> MQTT is low bandwidth consumption messaging protocol which
>>>> supports pub/sub protocol with the following limitations.
>>>>
>>>>    - It has fixed header (2 bytes) . Specifications mentioning that it
>>>>    can be extended.But AFAIU clients[1] didn't support for this header
>>>>    extension. We have checked the MQTT with the messages published with
>>>>    headers and we didn't receive the header on those.Only receive the message
>>>>    body.
>>>>    - The payload size for a MQTT message is 256MB.I guess design point
>>>>    for MQTT usage is targeted at more constrained devices.[3]. MQTT is better
>>>>    at a high volume of low size messages.
>>>>    - It is supports only topic.This will not be a issue currently in
>>>>    our scenario as we used only topic currently AFAIK.[4]
>>>>
>>>>
>>>> I guess it build with above limitations and it is very good choice for
>>>> the mobile and small application as it is very lightweight and low
>>>> power.[5] WDYT?
>>>>
>>>> Please share your thoughts on this.
>>>>
>>>> [1] https://pypi.python.org/pypi/paho-mqtt
>>>> [2[ https://github.com/iwanbk/nyamuk
>>>> [3] http://stephendnicholas.com/archives/1217
>>>> [4] http://www.mqtechconference.com/sessions_v2013/MQTC_v2013_MQTT.pdf
>>>> [5]
>>>> http://blogs.vmware.com/vfabric/2013/02/choosing-your-messaging-protocol-amqp-mqtt-or-stomp.html
>>>>
>>>>
>>>> Thanks,
>>>> Gayan
>>>>
>>>>
>>>> On Mon, Sep 1, 2014 at 4:32 PM, Gayan Gunarathne <gayang@wso2.com>
>>>> wrote:
>>>>
>>>>> Thanks Paul.
>>>>>
>>>>> We will check on this paho-mgtt python library as well.
>>>>>
>>>>> Thanks,
>>>>> Gayan
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Sep 1, 2014 at 4:17 PM, Paul Fremantle <paul@wso2.com>
wrote:
>>>>>
>>>>>> Python has excellent pure python MQTT support.
>>>>>>
>>>>>> > pip install paho-mqtt
>>>>>>
>>>>>> https://pypi.python.org/pypi/paho-mqtt
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>>
>>>>>> On 1 September 2014 11:40, Gayan Gunarathne <gayang@wso2.com>
wrote:
>>>>>>
>>>>>>> Hi Paul,
>>>>>>>
>>>>>>> Here we try to connect the our existing Apache ActiveMQ with
the
>>>>>>> pure python.(We try to replace our java cartridge agent with
the
>>>>>>> python as after that we can remove the java dependency from the
cartridge
>>>>>>> itself).
>>>>>>> In the initial research we didn't notice such a pure python
>>>>>>> libraries with the MQTT  which can connect with the ActiveMQ.
Also with the
>>>>>>> initial reading I guess they are recommend it for the mobile
applications
>>>>>>> as its low bandwidth consumption.Also I am noticed lot of web
socket based
>>>>>>> applications are developed through MQTT.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Gayan
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Sep 1, 2014 at 3:42 PM, Paul Fremantle <paul@wso2.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Have you looked at mqtt?
>>>>>>>>
>>>>>>>> Paul
>>>>>>>> On 1 Sep 2014 12:10, "Gayan Gunarathne" <gayang@wso2.com>
wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> As per our initial research on the python based messaging
>>>>>>>>> implementation of python cartridge agent and discussion[1]
we can summarize
>>>>>>>>> our findings as following.
>>>>>>>>>
>>>>>>>>> *1.AMQP*
>>>>>>>>>
>>>>>>>>> Apache ActiveMQ used the AMQP 1.0 protocol. We found
C based
>>>>>>>>> client library (proton) [2] which can be used with python.
>>>>>>>>> But it is a platform dependent library and every time
we need to
>>>>>>>>> install the libraries based on the architecture and OS
version.IMO this
>>>>>>>>> will be difficult us to manage as we need to install
the libraries based on
>>>>>>>>> the platform.
>>>>>>>>>
>>>>>>>>> Other python libraries(txAMQP ,pika etc) supports AMQP
0.9 which
>>>>>>>>> fails to connect with the ActiveMQ.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *2.STOMP*
>>>>>>>>>
>>>>>>>>> We can connect to the ActiveMQ with the STOMP protocol
by using
>>>>>>>>> the STOMP python client library[3].IMO we can used STOMP
with ActiveMQ
>>>>>>>>> without worry about the platform based library installation.By
considering
>>>>>>>>> our scope of the cartridge agent I think it is better
to have a platform
>>>>>>>>> independence cartridge as far as we can. Then, IMO STOMP
based messaging
>>>>>>>>> queue implementation  will be more manageable when we
increase our number
>>>>>>>>> of platform supported.
>>>>>>>>> WDYT?
>>>>>>>>>
>>>>>>>>> When comparing the performance of theses two protocols
it looks
>>>>>>>>> like there are no such huge difference[4].I guess the
performance also
>>>>>>>>> tightly coupled with the message broker you are using
as well.OTOH Apache
>>>>>>>>> ActiveMQ recommended way also to used the STOMP with
the python[5].
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Please share your thoughts on this.
>>>>>>>>>
>>>>>>>>> [1] Stratos Python Cartridge Agent
>>>>>>>>> [2] https://github.com/apache/qpid-proton
>>>>>>>>> [3] https://github.com/jasonrbriggs/stomp.py
>>>>>>>>> [4]
>>>>>>>>> http://blogs.vmware.com/vfabric/2013/02/choosing-your-messaging-protocol-amqp-mqtt-or-stomp.html
>>>>>>>>> [5] http://activemq.apache.org/python.html
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Gayan
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Gayan Gunarathne
>>>>>>>>> Technical Lead
>>>>>>>>> WSO2 Inc. (http://wso2.com)
>>>>>>>>> email  : gayang@wso2.com  | mobile : +94 766819985
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best Regards,
>>>>>>>
>>>>>>> Gayan Gunarathne
>>>>>>> Technical Lead
>>>>>>> WSO2 Inc. (http://wso2.com)
>>>>>>>  email  : gayang@wso2.com  | mobile : +94 766819985
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Paul Fremantle
>>>>>> CTO and Co-Founder, WSO2
>>>>>> OASIS WS-RX TC Co-chair, Apache Member
>>>>>>
>>>>>> UK: +44 207 096 0336
>>>>>>
>>>>>> blog: http://pzf.fremantle.org
>>>>>> twitter.com/pzfreo
>>>>>> paul@wso2.com
>>>>>>
>>>>>> wso2.com Lean Enterprise Middleware
>>>>>>
>>>>>> Disclaimer: This communication may contain privileged or other
>>>>>> confidential information and is intended exclusively for the addressee/s.
>>>>>> If you are not the intended recipient/s, or believe that you may
have
>>>>>> received this communication in error, please reply to the sender
indicating
>>>>>> that fact and delete the copy you received and in addition, you should
not
>>>>>> print, copy, retransmit, disseminate, or otherwise use the information
>>>>>> contained in this communication. Internet communications cannot be
>>>>>> guaranteed to be timely, secure, error or virus-free. The sender
does not
>>>>>> accept liability for any errors or omissions.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>>
>>>>> Gayan Gunarathne
>>>>> Technical Lead
>>>>> WSO2 Inc. (http://wso2.com)
>>>>> email  : gayang@wso2.com  | mobile : +94 766819985
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards,
>>>>
>>>> Gayan Gunarathne
>>>> Technical Lead
>>>> WSO2 Inc. (http://wso2.com)
>>>> email  : gayang@wso2.com  | mobile : +94 766819985
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Paul Fremantle
>>> CTO and Co-Founder, WSO2
>>> OASIS WS-RX TC Co-chair, Apache Member
>>>
>>> UK: +44 207 096 0336
>>>
>>> blog: http://pzf.fremantle.org
>>> twitter.com/pzfreo
>>> paul@wso2.com
>>>
>>> wso2.com Lean Enterprise Middleware
>>>
>>> Disclaimer: This communication may contain privileged or other
>>> confidential information and is intended exclusively for the addressee/s.
>>> If you are not the intended recipient/s, or believe that you may have
>>> received this communication in error, please reply to the sender indicating
>>> that fact and delete the copy you received and in addition, you should not
>>> print, copy, retransmit, disseminate, or otherwise use the information
>>> contained in this communication. Internet communications cannot be
>>> guaranteed to be timely, secure, error or virus-free. The sender does not
>>> accept liability for any errors or omissions.
>>>
>>
>>
>>
>> --
>> Best Regards,
>>
>> Gayan Gunarathne
>> Technical Lead
>> WSO2 Inc. (http://wso2.com)
>> email  : gayang@wso2.com  | mobile : +94 766819985
>>
>>
>
>
>
> --
> Paul Fremantle
> CTO and Co-Founder, WSO2
> OASIS WS-RX TC Co-chair, Apache Member
>
> UK: +44 207 096 0336
>
> blog: http://pzf.fremantle.org
> twitter.com/pzfreo
> paul@wso2.com
>
> wso2.com Lean Enterprise Middleware
>
> Disclaimer: This communication may contain privileged or other
> confidential information and is intended exclusively for the addressee/s.
> If you are not the intended recipient/s, or believe that you may have
> received this communication in error, please reply to the sender indicating
> that fact and delete the copy you received and in addition, you should not
> print, copy, retransmit, disseminate, or otherwise use the information
> contained in this communication. Internet communications cannot be
> guaranteed to be timely, secure, error or virus-free. The sender does not
> accept liability for any errors or omissions.
>



-- 
Best Regards,

Gayan Gunarathne
Technical Lead
WSO2 Inc. (http://wso2.com)
email  : gayang@wso2.com  | mobile : +94 766819985

Mime
View raw message