synapse-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andreas Veithen <andreas.veit...@gmail.com>
Subject Re: Data Loss Issue VFS
Date Thu, 14 May 2009 08:19:01 GMT
Kim,

I did a quick test, but I'm unable to reproduce this. To investigate
this, we will definitely need an example file that reproduces this
problem.

Andreas

On Tue, May 12, 2009 at 09:21, kimhorn <kim.horn@icsglobal.net> wrote:
>
> The synapse.xml is in the original post. We have tried synapse with very
> large files and other types of data but have never succeeded with this data
> type. The data contains many spaces, carriage returns, line feeds etc. I
> must admit the results I am getting are weird, but I can't see what I am
> doing wrong. I gave up on this a month ago and now have some time to look
> into it again.
>
> Currently our existing systems (and our clients) must use CData to wrap this
> data for sends and receives but unfortunately I cannot even get this far to
> test this with Synapse. Hence why I added the "*" to see where the
> whitespace are disappearing in the data.
>
> I will also attach a data file, when I can,  but can't do that at the moment
> as all the data has private info in it and it is not easy to sanitise these
> types of files.
>
> Thanks
> Kim
>
>
>
> pzfreo wrote:
>>
>> Kim
>>
>> Can you please post a simple Synapse.xml and a file that demonstrates
>> this. I'm quite surprised because I have had successful use of Synapse
>> with similar record oriented data to yours.
>>
>> Paul
>>
>> On Tue, May 12, 2009 at 6:53 AM, kimhorn <kim.horn@icsglobal.net> wrote:
>>>
>>> I should also mention that not only the data at the end of the file is
>>> missing but data in the middle is also gone in a number of places.
>>>
>>>
>>> kimhorn wrote:
>>>>
>>>> In testing the CDATA issues (another stream) we have come across a more
>>>> serious problem.
>>>> We have found that synapse is loosing parts of files that have imbedded
>>>> CR
>>>> and spaces. These are fixed format files. The script at the bottom is
>>>> used
>>>> to reproduce the problem. Logging the property extracted shows the loss
>>>> of
>>>> data. As the files are not small (96KB) and are full of spaces it is
>>>> hard
>>>> to demonstrate or shows the full logs. Here is the start and end of the
>>>> data file only; it has 100's of records in between. The data has a "*"
>>>> at
>>>> the beginning and at the end.
>>>>
>>>> <Data Start>
>>>> *000000141973601M95202YBA000000014197                 341316840
>>>>
>>>>     010402030000010
>>>> 000190000000000000000000000000000000000000000000000000000000000
>>>>
>>>>  006770000000000000000000000000000000000000000000000020000000000
>>>>
>>>>                                                  
    * <Data END>
>>>>
>>>>
>>>> Looking at the log the last part of the file is gone when Property DATA
>>>> is
>>>> logged.
>>>> -----------------------------------------------------------------------------
>>>>
>>>> 2009-05-12 15:23:50,825 [-] [vfs-Worker-5]  INFO LogMediator DATA =
>>>> *000000141973601M95202YBA000000014197                 341316840
>>>>
>>>>     010402030000010
>>>> 000190000000000000000000000000000000000000000000000000000000000
>>>>
>>>>
>>>> 2009-05-12 15:23:54,387 [-] [vfs-Worker-5] DEBUG LogMediator End : Log
>>>> mediator
>>>>
>>>> Looking at the XML created we get:
>>>> ----------------------------------
>>>>
>>>> <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope
>>>> xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><data>*000000141973601M95202YBA000000014197
>>>>
>>>> 000190000000000000000000000000000000000000000000000000000000000</data></soapenv:Body></soapenv:Envelope>
>>>>
>>>>
>>>> Clearly the last part of the file has gone. I thought the error could be
>>>> in the Javascript but it appears the data is already lost at the
>>>> property
>>>> when extracted from the VFS payload. Any suggestions on whats happening
>>>> here ?
>>>>
>>>> Could this be an issue with the XPATH property setting code?
>>>>
>>>> Version of Synapse is one of recent Stable Snapshots.
>>>>
>>>> Thanks
>>>> Kim
>>>>
>>>> <definitions xmlns="http://ws.apache.org/ns/synapse">
>>>>   <proxy name="FileCheckProxyPRS" transports="vfs">
>>>>     <parameter
>>>> name="transport.vfs.FileURI">file:///C:/work/sftpagent/testdata/prs/ndc</parameter>
>>>>     <parameter name="transport.vfs.ContentType">text/plain</parameter>
>>>>     <parameter name="transport.vfs.FileNamePattern">.*req</parameter>
>>>>     <parameter name="transport.PollInterval">15</parameter>
>>>>     <parameter
>>>> name="transport.vfs.MoveAfterProcess">file:///C:/work/sftpagent/testdata/prs/ndc/archive</parameter>
>>>>     <parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
>>>>     <target inSequence="inSequencePRS"/>
>>>>   </proxy>
>>>>   <sequence name="inSequencePRS">
>>>>     <log level="custom">
>>>>       <property name="MSG" value="&gt;&gt;&gt;&gt;&gt;&gt;&gt;
IN"/>
>>>>     </log>
>>>>     <log level="custom">
>>>>       <property xmlns:axis2ns="http://ws.apache.org/commons/ns/payload"
>>>> name="DATA" expression="//axis2ns:text"/>
>>>>     </log>
>>>>     <property xmlns:axis2ns="http://ws.apache.org/commons/ns/payload"
>>>> name="claimData" expression="//axis2ns:text"/>
>>>>     <log level="full"/>
>>>>     <script language="js"><![CDATA[ var claimData =
>>>> mc.getProperty("claimData").toString();
>>>>                   mc.setPayloadXML(<data>{claimData}</data>);
>>>>                   ]]></script>
>>>>     <send>
>>>>       <endpoint>
>>>>         <address uri="vfs:file:///C:/work/sftpagent/testdata/prs"/>
>>>>       </endpoint>
>>>>     </send>
>>>>   </sequence>
>>>> </definitions>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Data-Loss-Issue-VFS-tp23496565p23496604.html
>>> Sent from the Synapse - Dev mailing list archive at Nabble.com.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
>>> For additional commands, e-mail: dev-help@synapse.apache.org
>>>
>>>
>>
>>
>>
>> --
>> Paul Fremantle
>> Co-Founder and CTO, WSO2
>> Apache Synapse PMC Chair
>> OASIS WS-RX TC Co-chair
>>
>> blog: http://pzf.fremantle.org
>> paul@wso2.com
>>
>> "Oxygenating the Web Service Platform", www.wso2.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
>> For additional commands, e-mail: dev-help@synapse.apache.org
>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Data-Loss-Issue-VFS-tp23496565p23497440.html
> Sent from the Synapse - Dev mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
> For additional commands, e-mail: dev-help@synapse.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
For additional commands, e-mail: dev-help@synapse.apache.org


Mime
View raw message