cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Niclas Lindblom <niclas_lindb...@icloud.com.INVALID>
Subject Re: Help with publishing and consuming events using rabbitmq
Date Wed, 18 Nov 2020 23:34:59 GMT
Thanks,

I actually got it working, the problem I had was the part on creating the routing key filter
in RabbitMQ which isn’t too well documented at the moment. However, there is a bug currently
in the string being send by CS not being formatted correctly which has been logged as issue
#4468 .

Nclas

> On 19 Nov 2020, at 00:52, Gabriel Beims Bräscher <gabrascher@gmail.com> wrote:
> 
> I did play a bit with Rabbit MQ + CloudStack some time ago. Here follows
> some details regarding it.
> 
> Note that this example is really simplified and bound to steps for a "proof
> of concept".
> 
> *I. Configuring RabbitMQ + CloudStack:*
> 
> 1. create file
> /etc/cloudstack/management/META-INF/cloudstack/core/spring-event-bus-context.xml
> 2. edit spring-event-bus-context.xml to contain the following data:
> 2.1 the server that is running the RabbitMQ: localhost
> 2.2 the port on which RabbitMQ server is running: 5672
> 2.3 username associated with the account to access the RabbitMQ server:
> guest
> 2.4 password associated with the username of the account to access the
> RabbitMQ server: guest
> 2.5 The exchange name on the RabbitMQ server where CloudStack events are
> published: cloudstack-events
> 
> - - spring-event-bus-context.xml:
> 
> *<beans xmlns="http://www.springframework.org/schema/beans"
> <http://www.springframework.org/schema/beans%22>*
> 
> *xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> <http://www.w3.org/2001/XMLSchema-instance%22>*
> 
> *xmlns:context="http://www.springframework.org/schema/context"
> <http://www.springframework.org/schema/context%22>*
> 
> *xmlns:aop="http://www.springframework.org/schema/aop"
> <http://www.springframework.org/schema/aop%22>*
> 
> *xsi:schemaLocation="http://www.springframework.org/schema/beans
> <http://www.springframework.org/schema/beans>*
> 
> *http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
> <http://www.springframework.org/schema/beans/spring-beans-3.0.xsd>*
> 
> *http://www.springframework.org/schema/aop
> <http://www.springframework.org/schema/aop>http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
> <http://www.springframework.org/schema/aop/spring-aop-3.0.xsd>*
> 
> *http://www.springframework.org/schema/context
> <http://www.springframework.org/schema/context>*
> 
> *http://www.springframework.org/schema/context/spring-context-3.0.xsd"
> <http://www.springframework.org/schema/context/spring-context-3.0.xsd%22>>*
> 
> *<bean id="eventNotificationBus"
> class="org.apache.cloudstack.mom.rabbitmq.RabbitMQEventBus">*
> 
> *<property name="name" value="eventNotificationBus"/>*
> 
> *<property name="server" value="127.0.0.1"/>*
> 
> *<property name="port" value="5672"/>*
> 
> *<property name="username" value="guest"/>*
> 
> *<property name="password" value="guest"/>*
> 
> *<property name="exchange" value="cloudstack-events"/>*
> 
> *</bean>*
> 
> *</beans>*
> 
> 3. Enable the rabbitmq_management
> *rabbitmq-plugins enable rabbitmq_management*
> 
> 4. restart rabbitmq and cloudstack services
> 
> 
> *systemctl restart rabbitmq-server.servicesystemctl restart
> cloudstack-management.service*
> 
> 5. connected on the management server tunneling the localhost with port
> 15672
> ssh -L 15672:localhost:15672 user@cloudstack-management-host
> 
> 6. In this example the UI would be available at http://localhost:15672
> 
> *II. Binding the exchange ‘cloudstack-events’ with a queue*
> 
> CloudStack creates the exchange ‘cloudstack-events’ which will receive
> messages containing CloudStack events; however, there are no queues yet.
> To create a queue and bind with cloudstack-events the following steps are
> needed:
> 
> 1. Go to Queues tab and add a queue, e.g. 'cloudstack-queue’
> 2. Go to Exchanges tab and Bind to queue cloudstack-queue with the desired
> ‘Routing key’.
> 
> 
> *III. Routing keys*
> 
> The routing key is a list of words, delimited by a period (".").
> CloudStack builds routing keys according to each event type, some examples
> are:
> a)
> /management-server.ActionEvent.ACCOUNT-CREATE.Account.b9117aa2-9432-4dc4-a055-fee45c428239/
> b)
> /management-server.UsageEvent.VOLUME-CREATE.com-cloud-storage-Volume.1232e3e6-2576-4983-bde5-b904eba9e4cb/
> c)
> /management-server.UsageEvent.VM-CREATE.com-cloud-vm-VirtualMachine.1232e3e-9432-4dc4-a055-fee45c428239/
> 
> Some example of routing keys that match CloudStack events:
> a) A pound symbol (“#”) indicates a match on zero or more words; thus, it
> will match any possible set of words;
> b) Asterisk (“*”) matching any word and the period (“.”) delimiting:
> ‘*.*.*.*.*’;
> c) expressions to filter a specific set of events, e.g. matching VM-CREATE
> or UsageEvent: ‘management-server.UsageEvent.VM-CREATE.#’ or
> ‘management-server.UsageEvent.#’.
> 
> Cheers,
> Gabriel.
> 
> Em seg., 9 de nov. de 2020 às 13:38, Rakesh v <www.rakeshv.com@gmail.com>
> escreveu:
> 
>> I did it in a different way but not using rabbitmq.
>> 
>> I wrote a separate application which calls listEvents every minutes and
>> collects all the events which can later fed into another rabbitmq or Kafka
>> or elk stack.
>> 
>> Sent from my iPhone
>> 
>>> On 09-Nov-2020, at 1:15 PM, Niclas Lindblom <niclas_lindblom@icloud.com.invalid>
>> wrote:
>>> 
>>> Hi,
>>> 
>>> I would appreciate some help with setting up events to be consumed using
>> AQMP. I have limited knowledge with rabbitmq and have got to the following
>> point
>>> 
>>> 1. I have 2 cloudstack management servers (this is not a production
>> environment)
>>> 
>>> 2. I have configured both with the spring-event-bus-context.xml as per
>> the 4.14 documentation and my understanding is that there is no further
>> software to be installed.
>>> 
>>> 3. I have a rabbitmq server on a separate server and I can see the
>> cloudstack-events Exchange being created once I restart the management
>> service
>>> 
>>> 4. I have recreated the exchange as fanout and bound it to a queue
>>> 
>>> I can’t see any events coming through when setting up a consumer using
>> the basic python code from rabbitmq’s tutorial, but if I publish an event
>> using the same example code to the same queue it does come through.
>>> 
>>> 
>>> Questions
>>> 
>>> 1 Am I missing something in my configuration ?
>>> 
>>> 2. What should I set the routing key to, with fanout out I believe the
>> routing key is ignored but for reference.
>>> 
>>> 2. How can I confirm that messages are being sent from the server, the
>> only evidence I have is the exchange being created once the management
>> service was restart
>>> 
>>> 3. I can’t see anything pointing towards messages being sent in the
>> management log, should I ?
>>> 
>>> The endgame here is to consume these events with a Stackstorm sensor to
>> trigger automation, any pointers would be appreciated
>>> 
>>> Thanks
>>> 
>>> Niclas
>>> 
>> 


Mime
View raw message