ode-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tiger Gui <tigergui1...@gmail.com>
Subject Re: Draft version of "Refactor OModel and provide intelligent migration feature for Apache ODE" project proposal
Date Thu, 05 Apr 2012 09:07:20 GMT
Hi Tammo,

I have update the "introduction" section of the proposal like this:

== Introduction ==

Apache ODE (Orchestration Director Engine) executes business processes
written following the WS-BPEL standard. It talks to web services, sending
and receiving messages, handling data manipulation and error recovery as
described by your process definition. It supports both long and short
living process executions to orchestrate all the services that are part of
your application.

Current ODE's OModel has a static structure and is serialized using Java's
built-in mechanisms, which are not very tolerant against class changes and
could even struggle with JDK upgrades.

The backward compatibility issue is currently hindering ODE team from
adding new features and making releases more often. Once OModel changed,
old version .cbp file will fail to re-serialize back to OModel classes,
even ODE team has no good solution to migrate old version .cbp file to new
version. This is sucks for end-users. In real-world software development
scenarios, people hate this words - "Not compatible".

It is really cool if we solve this problem,  provide intelligent migration
feature for ODE, make OModel more dynamic, ODE will be more dynamic in the
future. Our project solve real-world software development issues, i think
this is fit to Google Summer of Code's goals, it will be a good GSoC
project.

Details here
http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/tigergui/41002#

Regards & thanks

2012/4/5 Tiger Gui <tigergui1990@gmail.com>

> Hi Tammo,
>
> This is absolutely what in my mind. I think we should re-contract another
> copy of OModel classes to confirm they can be easily serialize and
> deserialize by Jackson. I have updated my proposal "schedule" part
>
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/tigergui/41002
>
> Now, it may be more clear about this part. Please have a review of it.
>
> Yeah, once you finish pushing the experiment code into Github, please tell
> me. I will take a look into it and find useful parts to discuss with you :-)
>
> 2012/4/5 Tammo van Lessen <tvanlessen@gmail.com>
>
>> Hi Tiger,
>>
>> serializing worked for me after defining some mixed-in classes for
>> Jackson with some annotations. I have it on a different laptop
>> currently, I try to fetch it and share it with you. Perhaps I just
>> push my test code to github. So at least it serialized the whole tree,
>> but I'm not entirely sure if really all elements and attributes were
>> included.
>>
>> Okay, yes. I think we're on the same page. The important step is: a
>> new version of the OModel (e.g. in a different package), that has a
>> Map for unknown fields and that has private default constructors so
>> that Jackson can easily serialize and deserialize them. This OModel
>> needs then to be included into compiler and to be used by the runtime.
>> The next step is to provide means to read an old cbp file and migrate
>> its contents to the new OModel. And we both believe that Jackson can
>> ease this job as well, since we could serialize the old OModel to
>> JSON, tweak the output a "little" and the deserialize it to the new
>> OModel. I tried this in my experiments as well and simply used
>> String.replace to change the package name. I'll write you a second
>> mail when I pushed that stuff to github.
>>
>> HTH,
>>  Tammo
>>
>> On Wed, Apr 4, 2012 at 17:54, Tiger Gui <tigergui1990@gmail.com> wrote:
>> > Hi Tammo,
>> >
>> > I mean, if we use Jackson to serialize OProcess directly, such as
>> following
>> > code:
>> >
>> >         JsonGenerator jsonGenerator = null;
>> > ObjectMapper objectMapper = null;
>> > try {
>> > objectMapper = new ObjectMapper();
>> >             jsonGenerator =
>> > objectMapper.getJsonFactory().createJsonGenerator(System.out,
>> > JsonEncoding.UTF8);
>> >             jsonGenerator.writeObject(OProcess instance here);
>> > } catch (IOException e) {
>> >             e.printStackTrace();
>> >         }
>> >
>> > We will get some unsupport exception, right? Did you caught them in your
>> > experiment ?
>> >
>> > So, if we want to use Jackson to serialize the whole OProcess object
>> > successfully, we should extend Jackson and
>> > create another class (call it OProcessB) which store all the
>> information of
>> > the original OProcess object by key-value mode.
>> >
>> > This OProcessB may be similar with OProcess, but they are two different
>> > classes. I mean the transfrom job from OProcess -> OProcessB is the
>> > "refactor" job.
>> >
>> > Yeah, i miss deserialization part in the proposal, i will add this
>> part. But
>> > when i talked about the "Refactor OModel
>> > classes Oxxxx to confirm jackson can serialize them", i really mean :
>> >
>> > We should do some other Jackson extension job (even little OModel class
>> > refactor job) to confirm we can serialize and de-serialize these OModel
>> > classes to/from Json format file.
>> >
>> > Your suggestions? Thank you
>> >
>> >
>> > 2012/4/4 Tammo van Lessen <tvanlessen@gmail.com>
>> >>
>> >> Hi Tiger,
>> >>
>> >> thanks, I'll have a look. One thing I found is "Refactor OModel
>> >> classes Oxxxx to confirm jackson can serialize them.". Serialization
>> >> seems to be much easier than deserialization. In my experiments, the
>> >> OModel could be serialized without significant changes. Also,
>> >> refactoring the original OModel is difficult because we need to keep
>> >> the binary compatibility to the cbp file. So we can only refactor the
>> >> duplicated, new OModel and make sure that this serializes and
>> >> deserializes correctly. I guess this is what you meant, but it could
>> >> be more clear in the proposal.
>> >>
>> >> Thanks,
>> >>  Tammo
>> >>
>> >> On Wed, Apr 4, 2012 at 10:35, Tiger Gui <tigergui1990@gmail.com>
>> wrote:
>> >> > Hi Tammo,
>> >> >
>> >> > I have added project schedule and about me sections for my proposal
>> and
>> >> > submit it GSoC melange system, you can find the proposal details
>> >> > here[1],
>> >> > and we can keep improving it before April 6th, only 2 days left. If
>> you
>> >> > have
>> >> > any suggestions about it, please tell me, hope we can get an
>> acceptable
>> >> > proposal :-)
>> >> >
>> >> > Thank you
>> >> >
>> >> >
>> >> > [1]
>> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/tigergui/41002
>> >> >
>> >> > 2012/4/4 Tammo van Lessen <tvanlessen@gmail.com>
>> >> >>
>> >> >> Hi Tiger,
>> >> >>
>> >> >> thanks for your proposal. I'll need to read it more thoroughly,
I'll
>> >> >> come back to you tomorrow. But what is missing is a rough schedule
>> >> >> with defined milestones. This is something other applications
>> provide
>> >> >> and are helpful and necessary for the ranking and also for measuring
>> >> >> your progress. It would be great if you could add a project plan.
>> >> >>
>> >> >> Thanks,
>> >> >>  Tammo
>> >> >>
>> >> >> On Fri, Mar 30, 2012 at 10:11, Tiger Gui <tigergui1990@gmail.com>
>> >> >> wrote:
>> >> >> > Hi all,
>> >> >> >
>> >> >> > I have drawed up draft version of GSoC 2012 project "Refactor
>> OModel
>> >> >> > and
>> >> >> > provide intelligent migration feature for Apache ODE" project
>> >> >> > proposal,
>> >> >> > you
>> >> >> > guys can find the details here[1].
>> >> >> >
>> >> >> > This is only initial idea of Tammo and me, if you guys have
any
>> good
>> >> >> > advises
>> >> >> > about this project, let's discuss here, and i will update
the
>> >> >> > proposal
>> >> >> > according to the discussion result. As GSoC's student application
>> >> >> > period
>> >> >> > will close on April 6, we have still a week to improve the
>> proposal.
>> >> >> > Thank
>> >> >> > you all :-)
>> >> >> >
>> >> >> > [1]
>> >> >> >
>> >> >> >
>> >> >> >
>> http://code.google.com/p/tigergui-proposal-temp/wiki/Dynamic_OModel?ts=1333094366&updated=Dynamic_OModel
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > ===============Refactor OModel and provide intelligent migration
>> >> >> > feature
>> >> >> > for
>> >> >> > Apache ODE====================
>> >> >> >
>> >> >> > Introduction
>> >> >> >
>> >> >> > Apache ODE (Orchestration Director Engine) executes business
>> >> >> > processes
>> >> >> > written following the WS-BPEL standard. It talks to web services,
>> >> >> > sending
>> >> >> > and receiving messages, handling data manipulation and error
>> recovery
>> >> >> > as
>> >> >> > described by your process definition. It supports both long
and
>> short
>> >> >> > living
>> >> >> > process executions to orchestrate all the services that are
part
>> of
>> >> >> > your
>> >> >> > application.
>> >> >> >
>> >> >> > Current ODE's OModel has a static structure and is serialized
>> using
>> >> >> > Java's
>> >> >> > built-in mechanisms, which are not very tolerant against class
>> >> >> > changes
>> >> >> > and
>> >> >> > could even struggle with JDK upgrades. We want to create a
GSoC
>> >> >> > project
>> >> >> > to
>> >> >> > refactor OModel to provide intelligent migration feature for
ODE.
>> >> >> >
>> >> >> > Details Description
>> >> >> >
>> >> >> > Current ODE OModel Design
>> >> >> >
>> >> >> > Let's begin with Apache ODE's architectural:
>> >> >> >
>> >> >> > When a BPEL process is deployed to ODE, it first runs through
the
>> >> >> > compiler,
>> >> >> > which parses the BPEL file into a BPEL Object Model (BOM),
which
>> >> >> > basically
>> >> >> > aims at abstracting from different BPEL versions (1.1, 2.0
draft,
>> 2.0
>> >> >> > final).
>> >> >> >
>> >> >> > ODE Compiler compile and create OModel instances(such as
>> >> >> > OProcess,OAssign,OInvoke etc) from .bpel, .wsdl and deploy
files.
>> >> >> > Then
>> >> >> > it
>> >> >> > can use Java Object Serializable technology to serialize OModel
>> >> >> > objects
>> >> >> > into
>> >> >> > a .cbp file (Compiled Process Definitions part in the diagram,
>> and we
>> >> >> > can
>> >> >> > see OBase implements interface java.io.Serializable, and all
the
>> >> >> > OModel
>> >> >> > classes are the sub-class of OBase).
>> >> >> >
>> >> >> > ODE runtime module can load a compiled BPEL model(the .cbp
file in
>> >> >> > local
>> >> >> > disk) by de-serialize the .cbp file back into OModel classes.
>> >> >> >
>> >> >> > When a process instance is executed, the navigator operates
on the
>> >> >> > OModel,
>> >> >> > i.e. it is loaded from disk into memory during execution and
is
>> >> >> > unloaded
>> >> >> > afterwards, depending on hydration/dehydration policies to
reduce
>> the
>> >> >> > memory
>> >> >> > footprint. The execution state of the OModel keeps in-memory
>> >> >> > references
>> >> >> > to
>> >> >> > the OModel, but when the state is persisted, it does not store
a
>> copy
>> >> >> > of
>> >> >> > the
>> >> >> > OModel but only the IDs mentioned above. When the state is
loaded,
>> >> >> > the
>> >> >> > IDs
>> >> >> > are looked up and replaced by proper references.
>> >> >> >
>> >> >> > Where Is The Problem
>> >> >> >
>> >> >> > Once you read OModel's serializer and de-serializer methods
in
>> class
>> >> >> > org.apache.ode.bpel.o.Serializer:
>> >> >> >
>> >> >> > public void writeOProcess(OProcess process, OutputStream os)
>> throws
>> >> >> > IOException {
>> >> >> >
>> >> >> >         DataOutputStream out = new DataOutputStream(os);
>> >> >> >
>> >> >> >         out.write(MAGIC_NUMBER);
>> >> >> >
>> >> >> >         out.writeShort(format);
>> >> >> >
>> >> >> >         out.writeLong(compileTime);
>> >> >> >
>> >> >> >         out.writeUTF(process.guid);
>> >> >> >
>> >> >> >         out.writeUTF(process.targetNamespace);
>> >> >> >
>> >> >> >         out.writeUTF(process.processName);
>> >> >> >
>> >> >> >         out.flush();
>> >> >> >
>> >> >> >         ObjectOutputStream oos = new CustomObjectOutputStream(os);
>> >> >> >
>> >> >> >         oos.writeObject(process);
>> >> >> >
>> >> >> >         oos.flush();
>> >> >> >     }
>> >> >> >
>> >> >> > public OProcess readOProcess() throws IOException,
>> >> >> > ClassNotFoundException {
>> >> >> >
>> >> >> >
>> >> >> >         ObjectInputStream ois = new
>> >> >> > CustomObjectInputStream(_inputStream);
>> >> >> >
>> >> >> >         OProcess oprocess;
>> >> >> >         try {
>> >> >> >
>> >> >> >             oprocess = (OProcess) ois.readObject();
>> >> >> >
>> >> >> >         } catch (ClassNotFoundException e) {
>> >> >> >
>> >> >> >             throw new IOException("DataStream Error");
>> >> >> >
>> >> >> >         }
>> >> >> >         return oprocess;
>> >> >> >
>> >> >> >    }
>> >> >> >
>> >> >> > You will find the problem with the current OModel is that
it has a
>> >> >> > static
>> >> >> > structure and is serialized using Java's built-in mechanisms,
>> which
>> >> >> > are
>> >> >> > not
>> >> >> > very tolerant against class changes and could even struggle
with
>> JDK
>> >> >> > upgrades.
>> >> >> >
>> >> >> > For example, there are these attributes in current OProcess:
>> >> >> >
>> >> >> >     public final String version;
>> >> >> >
>> >> >> >     public OConstants constants;
>> >> >> >
>> >> >> >     public String uuid;
>> >> >> >
>> >> >> >     public String targetNamespace;
>> >> >> >
>> >> >> >     public String processName;
>> >> >> >
>> >> >> >     public OScope procesScope;
>> >> >> >
>> >> >> > After ODE Compiler serialize it into .cbp file, the .cbp file
will
>> >> >> > include
>> >> >> > these 6 attributes for OProcess instance. One day, we add
another
>> >> >> > attribute
>> >> >> > for OProcess class, for example:
>> >> >> >
>> >> >> >    pubic String newAttribute;
>> >> >> >
>> >> >> > If we try to de-serialize the old .cbp file into the new OProcess
>> >> >> > class,
>> >> >> > we
>> >> >> > will get a error.
>> >> >> >
>> >> >> > This is a problem because BPEL processes and their instances
are
>> >> >> > potentially
>> >> >> > long-running,they usually run over years. Thus, a serialized
>> OModel
>> >> >> > should
>> >> >> > survive ODE and JDK upgrades. Our problem is that even simple
>> changes
>> >> >> > (that
>> >> >> > are needed to implement new features or even to fix some bugs)
>> break
>> >> >> > backward compatibility in the OModel. We need a good solution
to
>> >> >> > address
>> >> >> > that problem.
>> >> >> >
>> >> >> > What Shall We Do To Solve This Problem
>> >> >> >
>> >> >> > This project want to solve this problem, we will change .cbp
>> file's
>> >> >> > format
>> >> >> > and use a brand new serialize solution to keep compatibility.
>> After
>> >> >> > investigate possible approaches, we would like to use JSON
to
>> >> >> > represent
>> >> >> > OModel(It means new .cbp file will be JSON format). About
the
>> above
>> >> >> > example,
>> >> >> > the old JSON format .cbp file can be de-serialized into the
new
>> >> >> > OProcess
>> >> >> > class with a special migrate condition description string.
>> >> >> >
>> >> >> > Json Format
>> >> >> >
>> >> >> > Jackson is actually a JSON processor (JSON parser + JSON
>> generator)
>> >> >> > written
>> >> >> > in Java. Beyond basic JSON reading/writing (parsing, generating),
>> it
>> >> >> > also
>> >> >> > offers full node-based Tree Model, as well as full OJM
>> (Object/Json
>> >> >> > Mapper)
>> >> >> > data binding functionality. More important, it has good
>> performance,
>> >> >> > even
>> >> >> > faster than Java's built-in means.
>> >> >> >
>> >> >> > So we decided to use Json format file store OModel's serialize
>> >> >> > result,
>> >> >> > and
>> >> >> > Jackson 2.0 is the potential solution to reading/writing OModel
>> >> >> > instance
>> >> >> > from/to Json format .cbp file.
>> >> >> >
>> >> >> > We can use Jackson to store attributes of Current OProcess
>> class to
>> >> >> > Json
>> >> >> > format file like this:
>> >> >> >
>> >> >> >         Map<String,Object> userData1 = new
>> HashMap<String,Object>();
>> >> >> >
>> >> >> >         userData1.put("owner", process.getOwner());
>> >> >> >
>> >> >> >         userData1.put("name", process.getName());
>> >> >> >
>> >> >> >         userData1.put("targetNameSpace", process.targetNamespace);
>> >> >> >
>> >> >> >         userData1.put("processName", process.processName);
>> >> >> >
>> >> >> >         userData1.put("children", process.getChildren());
>> >> >> >
>> >> >> >
>> >> >> >         ObjectMapper mapper = new ObjectMapper();
>> >> >> >
>> >> >> >         StringWriter sw = new StringWriter();
>> >> >> >
>> >> >> >         org.codehaus.jackson.JsonGenerator gen;
>> >> >> >
>> >> >> >         try {
>> >> >> >                 gen = new JsonFactory().createJsonGenerator(sw);
>> >> >> >
>> >> >> >                 mapper.writeValue(gen, userData1);
>> >> >> >
>> >> >> >                 gen.close();
>> >> >> >                 String json = sw.toString();
>> >> >> >
>> >> >> >                 System.out.println(json);
>> >> >> >
>> >> >> >         } catch (IOException e) {
>> >> >> >
>> >> >> >                 e.printStackTrace();
>> >> >> >         }
>> >> >> >
>> >> >> > The target json file will be:
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > {"targetNameSpace":"http://ode/bpel/unit-test
>> ","name":"HelloWorld2","processName":"HelloWorld2"}
>> >> >> >
>> >> >> > But not all the OModel instance can be serialized only using
>> Jacksons
>> >> >> > mixin
>> >> >> > feature, we have to extend Jackson and would probably need
to
>> write
>> >> >> > custom
>> >> >> > (de)serializers for WSDL4J etc.
>> >> >> >
>> >> >> > OModel Migrate Mechanism
>> >> >> >
>> >> >> > Here we use a simple example proposed by my potential mentor
Tammo
>> >> >> > van
>> >> >> > Lessen to describe the migrate mechanism.
>> >> >> >
>> >> >> > Simple Example
>> >> >> >
>> >> >> > OProcess stores currently targetNamespace and processName
as
>> Strings.
>> >> >> > Now
>> >> >> > imagine we change OProcess so that it stores a QName instead.
When
>> >> >> > loading
>> >> >> > an old OModel, the QName field would be null and there would
be
>> the
>> >> >> > two
>> >> >> > String field in the "unknown fields" map.
>> >> >> >
>> >> >> > Migrate Solution
>> >> >> >
>> >> >> > Suppose the old version of OProcess is stored in following
format:
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > {"targetNameSpace":"http://ode/bpel/unit-test
>> ","name":"HelloWorld2","processName":"HelloWorld2"}
>> >> >> >
>> >> >> > and the corresponding new version of OProcess's json .cbp
file
>> will
>> >> >> > be:
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > {"name":"HelloWorld2","namespace":"{
>> http://ode/bpel/unit-test}HelloWorld2"}
>> >> >> >
>> >> >> > The migrate job is transform from one JSON to another JSON
in a
>> >> >> > specific
>> >> >> > rule, We should consider how to define the migrate description
>> >> >> > language
>> >> >> > or
>> >> >> > just re-use some existing JSON migrate DSL.
>> >> >> >
>> >> >> > This migrate DSL may be the most important part of this project.
>> In
>> >> >> > this
>> >> >> > simple demo, maybe looks like this:
>> >> >> >
>> >> >> > {
>> >> >> >    "from":{
>> >> >> >
>> >> >> >       "name":"$value0",
>> >> >> >
>> >> >> >       "targetNameSpace":"$value1",
>> >> >> >
>> >> >> >       "processName":"$value2"
>> >> >> >
>> >> >> >    },
>> >> >> >    "to":{
>> >> >> >
>> >> >> >       "name":"$value0",
>> >> >> >
>> >> >> >       "namespace":"{$value1}$value2"
>> >> >> >
>> >> >> >    }
>> >> >> > }
>> >> >> >
>> >> >> > Use $value to represent migrate attribute values.
>> >> >> >
>> >> >> > Domain-specific language(DSL)
>> >> >> >
>> >> >> > Once we update OModel, and want to migrate the old version
to new
>> >> >> > version.
>> >> >> > If we have both the old version Json format .cbp file and
the
>> >> >> > specificed
>> >> >> > migrate rule, we can get the new version Json format .cbp
file,
>> then
>> >> >> > load as
>> >> >> > new OProcess object, finish the migrate job easily.
>> >> >> >
>> >> >> > We are still survey for exist tools like XSLT, working on
JSON
>> Trees.
>> >> >> > If
>> >> >> > can
>> >> >> > not find such a tool, have both the old OModel and the new
>> OModel, we
>> >> >> > can
>> >> >> > write a tool to generate migrate DSL string.
>> >> >> >
>> >> >> > Of course, we must specific the migrate rule for different
version
>> >> >> > change,
>> >> >> > such as v1.0->v1.1, v1.1->v1.2 and v1.0->v1.2.
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Best Regards From Tiger Gui
>> >> >> > --------------------------------------
>> >> >> > Open source is some kind of life attitude
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Tammo van Lessen - http://www.taval.de
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Best Regards From Tiger Gui
>> >> > --------------------------------------
>> >> > Open source is some kind of life attitude
>> >>
>> >>
>> >>
>> >> --
>> >> Tammo van Lessen - http://www.taval.de
>> >
>> >
>> >
>> >
>> > --
>> > Best Regards From Tiger Gui
>> > --------------------------------------
>> > Open source is some kind of life attitude
>>
>>
>>
>> --
>> Tammo van Lessen - http://www.taval.de
>>
>
>
>
> --
> Best Regards From Tiger Gui
> --------------------------------------
> Open source is some kind of life attitude
>



-- 
Best Regards From Tiger Gui
--------------------------------------
Open source is some kind of life attitude

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message