synapse-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hiranya Jayathilaka <hiranya...@gmail.com>
Subject Re: Synapse: tcp-listener: handling of multiple syslog messages in one TCP message?
Date Fri, 09 Aug 2013 19:15:49 GMT

On Aug 9, 2013, at 4:57 AM, "Wright, Peter" <Peter.Wright@six-group.com> wrote:

> Hi Hiranja,
> 
> Great, thanks a lot, now appears to be working correctly!!

That's great. 

I think it would be good to have some syslog mediation support built into Synapse. The correct
solution is probably to implement a syslog message builder and a formatter. There's a long
standing jura issue regarding this as well. We'll consider this feature for a future release.

Thanks,
Hiranya

> I changed my class mediator to generate the XML as you suggested, and then had to
> include the namespace in the iterate definition and include it in the xpath expression
> before it worked correctly.
> 
> Best regards,
> Peter
> 
> =================================
> 
> 1. XML message returned by my class mediator
> 
> <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><text
xmlns="http://ws.apache.org/commons/ns/payload">This is msg number 1</text><text
xmlns="http://ws.apache.org/commons/ns/payload">This is msg number 2</text><text
xmlns="http://ws.apache.org/commons/ns/payload">This is msg number 3</text></soapenv:Body></soapenv:Envelope>
> 
> 2. synapse.xml fragment (iterator definition)
> 
> <iterate id="syslogInterator" preservePayload="false" sequential="true" xmlns:m0="http://ws.apache.org/commons/ns/payload"
expression="//m0:text">
>  <target>
>    <sequence>
>      <send>
>       <endpoint>
>          <address .../>
>        </endpoint>
>      </send>
>    </sequence>
>  </target>
> </iterate>
> 
> -----Original Message-----
> From: Hiranya Jayathilaka [mailto:hiranya911@gmail.com] 
> Sent: Freitag, 9. August 2013 01:54
> To: user@synapse.apache.org
> Subject: Re: Synapse: tcp-listener: handling of multiple syslog messages in one TCP message?
> 
> 
> On Aug 7, 2013, at 4:53 AM, "Wright, Peter" <Peter.Wright@six-group.com> wrote:
> 
>> Hi Hiranja,
>> 
>> I have now implemented a class mediator that parses the (multiple)
>> syslog plain text messages and makes an XML payload from them.
>> I've also got an iterate mediator that splits the messages and
>> sends them individually as JMS messages to the messaging provider.
>> So far so good...
>> 
>> However, the individual JMS messages are now in XML form rather
>> than plain text (with all the XML headers that I don't want).
>> How can I get just the plain text payload to be sent?
>> 
>> Thanks
>> Peter
>> 
>> =====================
>> 
>> 1. Input: Dummy (syslog) message sent to TCP listener (individual messages separated
by NL)
>> 20 This is msg number 1
>> 20 This is msg number 2
>> 20 This is msg number 3
>> 
>> 2. SOAP message logged by synapse
>> <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><text
xmlns="http://ws.apache.org/commons/ns/payload">20 This is msg number 1
>> 20 This is msg number 2
>> 20 This is msg number 3
>> </text></soapenv:Body></soapenv:Envelope>
>> 
>> 3. XML message returned by my class mediator
>> <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><syslogMsgs><msg>This
is msg number 1</msg><msg>This is msg number 2</msg><msg>This is msg
number 3</msg></syslogMsgs></soapenv:Body></soapenv:Envelope>
> 
> This should create a message with multiple <text> elements. See the output from
step 2 above. It has a single <text> element with all the content. You should break
it down to multiple <text> elements. The plain text serializer of Axis2 looks for this
specific <text> element when sending plain text messages.
> 
> Pay attention to the namespace (http://ws.apache.org/commons/ns/payload) of the <text>
element too. It's also required.
> 
> Thanks,
> Hiranya
> 
>> 
>> 4. JMS messages sent to the messaging provider
>> <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><msg>This
is msg number 1</msg></soapenv:Body></soapenv:Envelope>
>> <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><msg>This
is msg number 2</msg></soapenv:Body></soapenv:Envelope>
>> <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><msg>This
is msg number 3</msg></soapenv:Body></soapenv:Envelope>
>> 
>> 5. JMS messages that I want to be sent
>> This is msg number 1
>> This is msg number 2
>> This is msg number 3
>> 
>> 6. synapse.xml
>> <definitions xmlns="http://ws.apache.org/ns/synapse">
>>   <proxy name="proxyTcp2Jms" transports="tcp">
>>       <target>
>>           <inSequence>
>>               <!-- Define TCP listener expects plain text (syslog) messages -->
>>               <property name="messageType" value="text/plain" scope="axis2"/>
>>               <property name="OUT_ONLY" value="true"/>
>>               <property name="TRANSPORT_HEADERS" scope="axis2" action="remove"/>
>>               <log level="full"/>
>> 
>>               <!-- Split any multiple syslog messages for the iterator -->
>>               <!-- and return as XML with child messages               -->
>>               <class name="com.sixtelekurs.classMediators.syslogMsgBuilder">
>>                   <log level="full"/>
>>               </class>
>> 
>>               <!-- Iterate over any multiple messages -->
>>               <iterate id="syslogInterator" preservePayload="false" sequential="true"
expression="//msg">
>>                   <log level="full"/>
>>                   <target>
>>                       <sequence>
>>                           <send>
>>                               <endpoint>
>>                                   <address uri="jms:/cn=sed.finesb.syslog?java.naming.factory.initial=com.sun.jndi.ldap.LdapCtxFactory&amp;java.naming.provider.url=MY_LDAP_URL&amp;transport.jms.ConnectionFactoryJNDIName=cn=MY_TCP&amp;transport.jms.DestinationType=topic&amp;java.naming.security.principal=MY_DN&amp;java.naming.security.credentials=MY_PW"/>
>>                               </endpoint>
>>                           </send>
>>                       </sequence>
>>                   </target>
>>               </iterate>
>>           </inSequence>
>>           <outSequence/>
>>           <faultSequence>
>>               <log level="full" category="ERROR" separator=","/>
>>           </faultSequence>
>>       </target>
>>       <parameter name="transport.tcp.port">6060</parameter>
>>       <parameter name="transport.tcp.contentType">text/plain</parameter>
>>   </proxy>
>> 
>> </definitions>
>> 
>> 
>> 
>> -----Original Message-----
>> From: Hiranya Jayathilaka [mailto:hiranya911@gmail.com] 
>> Sent: Freitag, 26. Juli 2013 20:05
>> To: user@synapse.apache.org
>> Subject: Re: Synapse: tcp-listener: handling of multiple syslog messages in one TCP
message?
>> 
>> You should probably apply some kind of a content transformation to transform this
payload into a XML payload with multiple "text" elements (one per each syslog message). But
I'm not sure whether traditional techniques like XSLT and XQuery support that kind of transformations.
After all you're trying to transform based on XML text and not the XML nodes. That's almost
always very difficult.
>> 
>> IMO you're better off writing a custom mediator to do this transformation. Then you
can use the iterate mediator to split the message into its constituents and send to the JMS
queue. An even more elegant solution would be to write a custom message builder that knows
how to parse the incoming batch requests and construct a proper XML payload out of it.
>> 
>> Thanks,
>> Hiranya
>> 
>> On Jul 26, 2013, at 6:10 AM, "Wright, Peter" <Peter.Wright@six-group.com> wrote:
>> 
>>> Hi,
>>> 
>>> I got synapse working with a TCP-Listener accepting plain text messages
>>> as input, and the messages being sent as JMS messages to IBM Websphere MQ.
>>> 
>>> Now we're in the test phase with "real" syslog (RFC-5424) data, and I'm having
a problem
>>> because the TCP listener is receiving multiple syslog messages in one TCP message.
>>> The multiple messages are being sent on as one JMS message. The individual
>>> syslog messages are separated by NL and have the message length at the start.
>>> 
>>> Here an example message from my log file with 8 syslog messages:
>>> 
>>> 2013-07-25T16:12:20,INFO ,mdzhsfesb01,MSGMED,MediatorLog.java:99,To: , WSAction:
urn:mediate, SOAPAction: urn:mediate, MessageID: urn:uuid:1BC21F33F4BAA449111374761227027,
Direction: request, Envelope:
>>> <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><text
xmlns="http://ws.apache.org/commons/ns/payload">
>>> 186 &lt;174>1 2013-07-25T16:08:00+02:00 mdzhldos02 DaMon.DAWD 12355 -
[meta sequenceId="1"] [HEARTBEAT@7410 Type="Heartbeat" timestamp="1374761220" telegrams="5522922"
BKs="632" LineIDs="441"]
>>> 400 &lt;170>1 2013-07-25T16:09:00+02:00 mdzhldos02 DaMon.DAWD 12355 -
[meta sequenceId="2"] [Ticket_605@7410 Type="Feed Outage" Status="New" BC="462" BK="842" Occured="1374760680"
Reported="1374761340" Summary="Outage of BC 462 at 2013-07-25 13:58:00 UTC (DAWD on mdzhldos02)"
Details="Outage of BC 462; BK 842; No Bid/Asks for 11 minutes; Occured: 2013-07-25 13:58:00
UTC; Reported by DAWD on mdzhldos02"]
>>> 186 &lt;174>1 2013-07-25T16:09:00+02:00 mdzhldos02 DaMon.DAWD 12355 -
[meta sequenceId="3"] [HEARTBEAT@7410 Type="Heartbeat" timestamp="1374761280" telegrams="6001019"
BKs="665" LineIDs="441"]
>>> 186 &lt;174>1 2013-07-25T16:10:00+02:00 mdzhldos02 DaMon.DAWD 12355 -
[meta sequenceId="4"] [HEARTBEAT@7410 Type="Heartbeat" timestamp="1374761340" telegrams="4908793"
BKs="612" LineIDs="440"]
>>> 119 &lt;174>1 2013-07-25T16:10:40+02:00 mdzhldos02 DaMon.DAWD 24438 -
[meta sequenceId="5"] [HEARTBEAT@7410 type="Heartbeat"]
>>> 362 &lt;170>1 2013-07-25T16:11:00+02:00 mdzhldos02 DaMon.DAWD 12355 -
[meta sequenceId="6"] [Ticket_605@7410 Status="Close" BC="462" BK="(null)" Occured="1374761340"
Reported="1374761460" Summary="Problem with BC 462 disappeared at 2013-07-25 14:09:00 UTC
(DAWD on mdzhldos02)" Details="Problem with BC 462 disappeared at 2013-07-25 14:09:00 UTC
(DAWD on mdzhldos02)"]
>>> 186 &lt;174>1 2013-07-25T16:11:00+02:00 mdzhldos02 DaMon.DAWD 12355 -
[meta sequenceId="7"] [HEARTBEAT@7410 Type="Heartbeat" timestamp="1374761400" telegrams="6099578"
BKs="657" LineIDs="440"]
>>> 186 &lt;174>1 2013-07-25T16:12:00+02:00 mdzhldos02 DaMon.DAWD 12355 -
[meta sequenceId="8"] [HEARTBEAT@7410 Type="Heartbeat" timestamp="1374761460" telegrams="5570590"
BKs="613" LineIDs="446"]
>>> </text></soapenv:Body></soapenv:Envelope>
>>> 
>>> What is the recommended way of handling this? Use the Iterate-Mediator to
>>> split the messages and send them individually? Do I have to use Xpath here,
>>> or is it possible to use e.g. javascript? Any examples would be appreciated.
>>> 
>>> Thanks for any help,
>>> Peter
>>> 
>>> 
>>> The content of this e-mail is intended only for the confidential use of the person
addressed. 
>>> If you are not the intended recipient, please notify the sender and delete this
email immediately.
>>> Thank you.
>> 
>> --
>> Hiranya Jayathilaka
>> Mayhem Lab/RACE Lab;
>> Dept. of Computer Science, UCSB;  http://cs.ucsb.edu
>> E-mail: hiranya@cs.ucsb.edu;  Mobile: +1 (805) 895-7443
>> Blog: http://techfeast-hiranya.blogspot.com
>> 
>> The content of this e-mail is intended only for the confidential use of the person
addressed. 
>> If you are not the intended recipient, please notify the sender and delete this email
immediately.
>> Thank you.
> 
> --
> Hiranya Jayathilaka
> Mayhem Lab/RACE Lab;
> Dept. of Computer Science, UCSB;  http://cs.ucsb.edu
> E-mail: hiranya@cs.ucsb.edu;  Mobile: +1 (805) 895-7443
> Blog: http://techfeast-hiranya.blogspot.com
> 
> The content of this e-mail is intended only for the confidential use of the person addressed.

> If you are not the intended recipient, please notify the sender and delete this email
immediately.
> Thank you.

--
Hiranya Jayathilaka
Mayhem Lab/RACE Lab;
Dept. of Computer Science, UCSB;  http://cs.ucsb.edu
E-mail: hiranya@cs.ucsb.edu;  Mobile: +1 (805) 895-7443
Blog: http://techfeast-hiranya.blogspot.com


Mime
View raw message