axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shameera Rathnayaka <shameerai...@gmail.com>
Subject Re: [GSoC 2012] XML to JSON convention using XMLStreamWriter API and Schemas with google-gson
Date Wed, 08 Aug 2012 06:47:13 GMT
HI devs,

I finished the implementation and did a performance test with existing SOAP
and implemented JSON. Here is the summary of the test results.
For this i used same service and send the same request in both SOAP and
JSON ways, size of JSON message is lower than SOAP message but requests are
exactly same.

Environment :
OS - Ubuntu 11.10
Axis2 1.7(current trunk)
Tool - Java benchmark

*num of requests* : 50000
*concurrency level*: 10 (Total num of requests are 50000*10 = 500000 )
*Time taken for test* :  with SOAP  *178* sec and with JSON *140* sec
*Total transferred* : with SOAP *2371000000* bytes and with JSON
*674000000*bytes
*Requests per second*: with SOAP * 2,801.76* [#/sec] (mean)  and with
JSON *3,568.89
* [#/sec] (mean)

will do a performance test with concurrency level 50 and num of requests
500000 and give the summary of results.

Thanks,
Shameera.



On Tue, Jul 31, 2012 at 10:50 AM, Shameera Rathnayaka <
shameerainfo@gmail.com> wrote:

> HI devs,
>
> I have implemented XMLStreamWriter as well as XMLStreamReader to convert
> XML <----> JSON with XMLSchema,
> Basically it works fine, but still need few test to be done after that I
> am willing to write performance test with implemented JSON support with
> existing SOAP support.
>
> Thanks,
> Shameera.
>
>
>
> On Mon, Jul 16, 2012 at 10:43 AM, Amila Suriarachchi <
> amilasuriarachchi@gmail.com> wrote:
>
>>
>>
>> On Sun, Jul 15, 2012 at 10:53 PM, Shameera Rathnayaka <
>> shameerainfo@gmail.com> wrote:
>>
>>> Hi devs,
>>>
>>> I implemented this feature completely to convert XML to JSON with the
>>> help of XmlSchema of that XML document. I tested this with different
>>> Schemas, it is working fine. I implemented this as a separate project, so i
>>> will integrate this with axis2 trunk and try with actual scenario.
>>>
>>
>> Can you use this technique at the reader level as well to support
>> namespaces?
>>
>> Then do a performance comparison using a POJO service with the exiting
>> json support and based the new gson based XMLstream API. According to the
>> result we need to optimise the later to have better performance.
>>
>> thanks,
>> Amila.
>>
>>
>>>
>>> Yes, This may be a standard way to convert XML to JSON conversion as
>>> other implementations don't convert JSON message correctly when it has one
>>> value array, unless implementation supports special XML structure where
>>> Json-lib does.
>>>
>>> Definitely we can improve the performance of this implementation. May be
>>> in the way that Sagara had suggested in his reply or in another way.
>>>
>>> Thanks,
>>> Shameera.
>>>
>>> On Sun, Jul 15, 2012 at 6:56 PM, Amila Suriarachchi <
>>> amilasuriarachchi@gmail.com> wrote:
>>>
>>>>
>>>>
>>>> On Sun, Jul 15, 2012 at 2:19 PM, Sagara Gunathunga <
>>>> sagara.gunathunga@gmail.com> wrote:
>>>>
>>>>> On Sun, Jul 15, 2012 at 1:27 PM, Amila Suriarachchi
>>>>> <amilasuriarachchi@gmail.com> wrote:
>>>>> >
>>>>> >
>>>>> > On Sat, Jul 14, 2012 at 1:14 PM, Sagara Gunathunga
>>>>> > <sagara.gunathunga@gmail.com> wrote:
>>>>> >>
>>>>> >> On Sat, Jul 14, 2012 at 12:42 PM, Shameera Rathnayaka
>>>>> >> <shameerainfo@gmail.com> wrote:
>>>>> >> > Hi devs,
>>>>> >> >
>>>>> >> > As Amila(Project mentor) suggested in previous thread[0]
, I have
>>>>> >> > implemented
>>>>> >> > XMLStreamWriter API to use schema of the processing xml
and gson
>>>>> >> > together
>>>>> >> > to convert XML ---> JSON. I have made my progress up
to support
>>>>> xml
>>>>> >> > which
>>>>> >> > don't have complex type elements with maxOccur >1, 
but may have
>>>>> simple
>>>>> >> > type elements with maxOccur >=1 . That means In the
JSON point of
>>>>> view,
>>>>> >> > my
>>>>> >> > present implementation supports XML --> JSON where there
is not
>>>>> any JSON
>>>>> >> > Arrays which has JSON objects in the converted JSON String.
>>>>> >> >
>>>>> >> > Here is a summary of the implementation.
>>>>> >> >
>>>>> >> > GsonXmlStreamWriter which implements XMLStreamWriter has
a
>>>>> constructor
>>>>> >> > to get JsonWriter and XmlSchema object. Then it process
XmlSchema
>>>>> and
>>>>> >>
>>>>> >> Are you plan to process  XmlSchema object with every response
? If
>>>>> so
>>>>> >> processing  XmlSchema object will add some overhead and result
into
>>>>> >> performance.  Axis2 already has JSON support main objective
of this
>>>>> >> project is to provide a high performance JSON implementation,
>>>>> >> processing  XmlSchema object with every response does not help
to
>>>>> meet
>>>>> >> your expectations. Instead I suggest to process  XmlSchema object
>>>>> only
>>>>> >> one-time during the deployment after the AxisService creation
and
>>>>> >> store result of  XmlSchema processing  with AxisService object
as  a
>>>>> >> light-weight object ( Let's say a map)   and use this light
weight
>>>>> map
>>>>> >> with response. In this approach overhead if relatively low.
>>>>> >
>>>>> >
>>>>> > This is a POC level approach to convert xml stream to json using
xml
>>>>> schema.
>>>>> > For an example there are some techniques like
>>>>> > bagger fish and others to address this problem. But those things
>>>>> complicated
>>>>> > the produced json stream.
>>>>> >
>>>>> > Once it proves at the POC level we can think of improving
>>>>> performance.
>>>>> >
>>>>> > If you look at the current axis2 json support does that properly
>>>>> address the
>>>>> > problem shameera mentioned earlier? i.e serialising the
>>>>> > array elements when there is on element. And also can that be used
>>>>> with some
>>>>> > thing like ADB or WSO2 Data Services,rules etc..
>>>>> > Idea here to develop a technique to address those problems properly
>>>>> while
>>>>> > converting an xml stream to json stream.
>>>>>
>>>>> I don't have any issue with the technique proposed here (i.e.
>>>>> processing XMLSchema to identify structure ). My only concern is why
>>>>> we should process XMLSchema with every response because in each and
>>>>> every response we end up getting same result, why shouldn't we process
>>>>> XMLSchema only one time and reuse results ? May be as last step of
>>>>> deployment or with very first response processing ?
>>>>>
>>>>> Of course we can keep it as future improvement for the moment but
>>>>> Axis2 having good history of forgetting those improvement  after
>>>>> initial development every time when I dig into code I found couple of
>>>>> such TODOs.
>>>>>
>>>>
>>>> :) I am not telling to do it after GSoc project. But first we need to
>>>> have a proper POC.
>>>>
>>>> thanks,
>>>> Amila.
>>>>
>>>>>
>>>>> Thanks !
>>>>>
>>>>>
>>>>>
>>>>> >
>>>>> > thanks,
>>>>> > Amila.
>>>>> >
>>>>> >>
>>>>> >>
>>>>> >> Thanks !
>>>>> >>
>>>>> >> > use a queue to keep the schema structure. At the moment
i have
>>>>> only
>>>>> >> > implemented
>>>>> >> > writeStartElement(String localName) , writeCharaters(String
text)
>>>>> ,
>>>>> >> > writeEndElement()
>>>>> >> > and writeEndDocument() functions. I am processing the queue
which
>>>>> has
>>>>> >> > the schema structure and put it to another stack to identify
the
>>>>> scope
>>>>> >> > of
>>>>> >> > end elements.
>>>>> >> >
>>>>> >> > I am now working on, improving this to support xml which
has
>>>>> complex
>>>>> >> > type
>>>>> >> > elements
>>>>> >> >  with maxOccour > 1 . I'll keep updating my progress
mean while.
>>>>> >> >
>>>>> >> > Thanks,
>>>>> >> > Shameera.
>>>>> >> >
>>>>> >> > --
>>>>> >> > Shameera Rathnayaka
>>>>> >> > Undergraduate
>>>>> >> > Department of Computer Science and Engineering
>>>>> >> > University of Moratuwa.
>>>>> >> > Sri Lanka.
>>>>> >> >
>>>>> >> > Blog : http://shameerarathnayaka.blogspot.com/
>>>>> >> >
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> --
>>>>> >> Sagara Gunathunga
>>>>> >>
>>>>> >> Blog      - http://ssagara.blogspot.com
>>>>> >> Web      - http://people.apache.org/~sagara/
>>>>> >> LinkedIn - http://www.linkedin.com/in/ssagara
>>>>> >>
>>>>> >>
>>>>> ---------------------------------------------------------------------
>>>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>> >>
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Amila Suriarachchi
>>>>> > WSO2 Inc.
>>>>> > blog: http://amilachinthaka.blogspot.com/
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sagara Gunathunga
>>>>>
>>>>> Blog      - http://ssagara.blogspot.com
>>>>> Web      - http://people.apache.org/~sagara/
>>>>> LinkedIn - http://www.linkedin.com/in/ssagara
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Amila Suriarachchi
>>>> WSO2 Inc.
>>>> blog: http://amilachinthaka.blogspot.com/
>>>>
>>>
>>>
>>>
>>> --
>>> Shameera Rathnayaka
>>> Undergraduate
>>> Department of Computer Science and Engineering
>>> University of Moratuwa.
>>> Sri Lanka.
>>>
>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>
>>>
>>
>>
>> --
>> Amila Suriarachchi
>> WSO2 Inc.
>> blog: http://amilachinthaka.blogspot.com/
>>
>
>
>
> --
> Shameera Rathnayaka
> Undergraduate
> Department of Computer Science and Engineering
> University of Moratuwa.
> Sri Lanka.
>
> Blog : http://shameerarathnayaka.blogspot.com/
>
>


-- 
Shameera Rathnayaka
Undergraduate
Department of Computer Science and Engineering
University of Moratuwa.
Sri Lanka.

Blog : http://shameerarathnayaka.blogspot.com/

Mime
View raw message