ode-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tiger Gui <tigergui1...@gmail.com>
Subject Re: Draft version of "Refactor OModel and provide intelligent migration feature for Apache ODE" project proposal
Date Thu, 05 Apr 2012 08:29:46 GMT
Hi Tammo,

This is absolutely what in my mind. I think we should re-contract another
copy of OModel classes to confirm they can be easily serialize and
deserialize by Jackson. I have updated my proposal "schedule" part
http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/tigergui/41002

Now, it may be more clear about this part. Please have a review of it.

Yeah, once you finish pushing the experiment code into Github, please tell
me. I will take a look into it and find useful parts to discuss with you :-)

2012/4/5 Tammo van Lessen <tvanlessen@gmail.com>

> Hi Tiger,
>
> serializing worked for me after defining some mixed-in classes for
> Jackson with some annotations. I have it on a different laptop
> currently, I try to fetch it and share it with you. Perhaps I just
> push my test code to github. So at least it serialized the whole tree,
> but I'm not entirely sure if really all elements and attributes were
> included.
>
> Okay, yes. I think we're on the same page. The important step is: a
> new version of the OModel (e.g. in a different package), that has a
> Map for unknown fields and that has private default constructors so
> that Jackson can easily serialize and deserialize them. This OModel
> needs then to be included into compiler and to be used by the runtime.
> The next step is to provide means to read an old cbp file and migrate
> its contents to the new OModel. And we both believe that Jackson can
> ease this job as well, since we could serialize the old OModel to
> JSON, tweak the output a "little" and the deserialize it to the new
> OModel. I tried this in my experiments as well and simply used
> String.replace to change the package name. I'll write you a second
> mail when I pushed that stuff to github.
>
> HTH,
>  Tammo
>
> On Wed, Apr 4, 2012 at 17:54, Tiger Gui <tigergui1990@gmail.com> wrote:
> > Hi Tammo,
> >
> > I mean, if we use Jackson to serialize OProcess directly, such as
> following
> > code:
> >
> >         JsonGenerator jsonGenerator = null;
> > ObjectMapper objectMapper = null;
> > try {
> > objectMapper = new ObjectMapper();
> >             jsonGenerator =
> > objectMapper.getJsonFactory().createJsonGenerator(System.out,
> > JsonEncoding.UTF8);
> >             jsonGenerator.writeObject(OProcess instance here);
> > } catch (IOException e) {
> >             e.printStackTrace();
> >         }
> >
> > We will get some unsupport exception, right? Did you caught them in your
> > experiment ?
> >
> > So, if we want to use Jackson to serialize the whole OProcess object
> > successfully, we should extend Jackson and
> > create another class (call it OProcessB) which store all the information
> of
> > the original OProcess object by key-value mode.
> >
> > This OProcessB may be similar with OProcess, but they are two different
> > classes. I mean the transfrom job from OProcess -> OProcessB is the
> > "refactor" job.
> >
> > Yeah, i miss deserialization part in the proposal, i will add this part.
> But
> > when i talked about the "Refactor OModel
> > classes Oxxxx to confirm jackson can serialize them", i really mean :
> >
> > We should do some other Jackson extension job (even little OModel class
> > refactor job) to confirm we can serialize and de-serialize these OModel
> > classes to/from Json format file.
> >
> > Your suggestions? Thank you
> >
> >
> > 2012/4/4 Tammo van Lessen <tvanlessen@gmail.com>
> >>
> >> Hi Tiger,
> >>
> >> thanks, I'll have a look. One thing I found is "Refactor OModel
> >> classes Oxxxx to confirm jackson can serialize them.". Serialization
> >> seems to be much easier than deserialization. In my experiments, the
> >> OModel could be serialized without significant changes. Also,
> >> refactoring the original OModel is difficult because we need to keep
> >> the binary compatibility to the cbp file. So we can only refactor the
> >> duplicated, new OModel and make sure that this serializes and
> >> deserializes correctly. I guess this is what you meant, but it could
> >> be more clear in the proposal.
> >>
> >> Thanks,
> >>  Tammo
> >>
> >> On Wed, Apr 4, 2012 at 10:35, Tiger Gui <tigergui1990@gmail.com> wrote:
> >> > Hi Tammo,
> >> >
> >> > I have added project schedule and about me sections for my proposal
> and
> >> > submit it GSoC melange system, you can find the proposal details
> >> > here[1],
> >> > and we can keep improving it before April 6th, only 2 days left. If
> you
> >> > have
> >> > any suggestions about it, please tell me, hope we can get an
> acceptable
> >> > proposal :-)
> >> >
> >> > Thank you
> >> >
> >> >
> >> > [1]
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/tigergui/41002
> >> >
> >> > 2012/4/4 Tammo van Lessen <tvanlessen@gmail.com>
> >> >>
> >> >> Hi Tiger,
> >> >>
> >> >> thanks for your proposal. I'll need to read it more thoroughly, I'll
> >> >> come back to you tomorrow. But what is missing is a rough schedule
> >> >> with defined milestones. This is something other applications provide
> >> >> and are helpful and necessary for the ranking and also for measuring
> >> >> your progress. It would be great if you could add a project plan.
> >> >>
> >> >> Thanks,
> >> >>  Tammo
> >> >>
> >> >> On Fri, Mar 30, 2012 at 10:11, Tiger Gui <tigergui1990@gmail.com>
> >> >> wrote:
> >> >> > Hi all,
> >> >> >
> >> >> > I have drawed up draft version of GSoC 2012 project "Refactor
> OModel
> >> >> > and
> >> >> > provide intelligent migration feature for Apache ODE" project
> >> >> > proposal,
> >> >> > you
> >> >> > guys can find the details here[1].
> >> >> >
> >> >> > This is only initial idea of Tammo and me, if you guys have any
> good
> >> >> > advises
> >> >> > about this project, let's discuss here, and i will update the
> >> >> > proposal
> >> >> > according to the discussion result. As GSoC's student application
> >> >> > period
> >> >> > will close on April 6, we have still a week to improve the
> proposal.
> >> >> > Thank
> >> >> > you all :-)
> >> >> >
> >> >> > [1]
> >> >> >
> >> >> >
> >> >> >
> http://code.google.com/p/tigergui-proposal-temp/wiki/Dynamic_OModel?ts=1333094366&updated=Dynamic_OModel
> >> >> >
> >> >> >
> >> >> >
> >> >> > ===============Refactor OModel and provide intelligent migration
> >> >> > feature
> >> >> > for
> >> >> > Apache ODE====================
> >> >> >
> >> >> > Introduction
> >> >> >
> >> >> > Apache ODE (Orchestration Director Engine) executes business
> >> >> > processes
> >> >> > written following the WS-BPEL standard. It talks to web services,
> >> >> > sending
> >> >> > and receiving messages, handling data manipulation and error
> recovery
> >> >> > as
> >> >> > described by your process definition. It supports both long and
> short
> >> >> > living
> >> >> > process executions to orchestrate all the services that are part
of
> >> >> > your
> >> >> > application.
> >> >> >
> >> >> > Current ODE's OModel has a static structure and is serialized
using
> >> >> > Java's
> >> >> > built-in mechanisms, which are not very tolerant against class
> >> >> > changes
> >> >> > and
> >> >> > could even struggle with JDK upgrades. We want to create a GSoC
> >> >> > project
> >> >> > to
> >> >> > refactor OModel to provide intelligent migration feature for ODE.
> >> >> >
> >> >> > Details Description
> >> >> >
> >> >> > Current ODE OModel Design
> >> >> >
> >> >> > Let's begin with Apache ODE's architectural:
> >> >> >
> >> >> > When a BPEL process is deployed to ODE, it first runs through
the
> >> >> > compiler,
> >> >> > which parses the BPEL file into a BPEL Object Model (BOM), which
> >> >> > basically
> >> >> > aims at abstracting from different BPEL versions (1.1, 2.0 draft,
> 2.0
> >> >> > final).
> >> >> >
> >> >> > ODE Compiler compile and create OModel instances(such as
> >> >> > OProcess,OAssign,OInvoke etc) from .bpel, .wsdl and deploy files.
> >> >> > Then
> >> >> > it
> >> >> > can use Java Object Serializable technology to serialize OModel
> >> >> > objects
> >> >> > into
> >> >> > a .cbp file (Compiled Process Definitions part in the diagram,
and
> we
> >> >> > can
> >> >> > see OBase implements interface java.io.Serializable, and all the
> >> >> > OModel
> >> >> > classes are the sub-class of OBase).
> >> >> >
> >> >> > ODE runtime module can load a compiled BPEL model(the .cbp file
in
> >> >> > local
> >> >> > disk) by de-serialize the .cbp file back into OModel classes.
> >> >> >
> >> >> > When a process instance is executed, the navigator operates on
the
> >> >> > OModel,
> >> >> > i.e. it is loaded from disk into memory during execution and is
> >> >> > unloaded
> >> >> > afterwards, depending on hydration/dehydration policies to reduce
> the
> >> >> > memory
> >> >> > footprint. The execution state of the OModel keeps in-memory
> >> >> > references
> >> >> > to
> >> >> > the OModel, but when the state is persisted, it does not store
a
> copy
> >> >> > of
> >> >> > the
> >> >> > OModel but only the IDs mentioned above. When the state is loaded,
> >> >> > the
> >> >> > IDs
> >> >> > are looked up and replaced by proper references.
> >> >> >
> >> >> > Where Is The Problem
> >> >> >
> >> >> > Once you read OModel's serializer and de-serializer methods in
> class
> >> >> > org.apache.ode.bpel.o.Serializer:
> >> >> >
> >> >> > public void writeOProcess(OProcess process, OutputStream os) throws
> >> >> > IOException {
> >> >> >
> >> >> >         DataOutputStream out = new DataOutputStream(os);
> >> >> >
> >> >> >         out.write(MAGIC_NUMBER);
> >> >> >
> >> >> >         out.writeShort(format);
> >> >> >
> >> >> >         out.writeLong(compileTime);
> >> >> >
> >> >> >         out.writeUTF(process.guid);
> >> >> >
> >> >> >         out.writeUTF(process.targetNamespace);
> >> >> >
> >> >> >         out.writeUTF(process.processName);
> >> >> >
> >> >> >         out.flush();
> >> >> >
> >> >> >         ObjectOutputStream oos = new CustomObjectOutputStream(os);
> >> >> >
> >> >> >         oos.writeObject(process);
> >> >> >
> >> >> >         oos.flush();
> >> >> >     }
> >> >> >
> >> >> > public OProcess readOProcess() throws IOException,
> >> >> > ClassNotFoundException {
> >> >> >
> >> >> >
> >> >> >         ObjectInputStream ois = new
> >> >> > CustomObjectInputStream(_inputStream);
> >> >> >
> >> >> >         OProcess oprocess;
> >> >> >         try {
> >> >> >
> >> >> >             oprocess = (OProcess) ois.readObject();
> >> >> >
> >> >> >         } catch (ClassNotFoundException e) {
> >> >> >
> >> >> >             throw new IOException("DataStream Error");
> >> >> >
> >> >> >         }
> >> >> >         return oprocess;
> >> >> >
> >> >> >    }
> >> >> >
> >> >> > You will find the problem with the current OModel is that it has
a
> >> >> > static
> >> >> > structure and is serialized using Java's built-in mechanisms,
which
> >> >> > are
> >> >> > not
> >> >> > very tolerant against class changes and could even struggle with
> JDK
> >> >> > upgrades.
> >> >> >
> >> >> > For example, there are these attributes in current OProcess:
> >> >> >
> >> >> >     public final String version;
> >> >> >
> >> >> >     public OConstants constants;
> >> >> >
> >> >> >     public String uuid;
> >> >> >
> >> >> >     public String targetNamespace;
> >> >> >
> >> >> >     public String processName;
> >> >> >
> >> >> >     public OScope procesScope;
> >> >> >
> >> >> > After ODE Compiler serialize it into .cbp file, the .cbp file
will
> >> >> > include
> >> >> > these 6 attributes for OProcess instance. One day, we add another
> >> >> > attribute
> >> >> > for OProcess class, for example:
> >> >> >
> >> >> >    pubic String newAttribute;
> >> >> >
> >> >> > If we try to de-serialize the old .cbp file into the new OProcess
> >> >> > class,
> >> >> > we
> >> >> > will get a error.
> >> >> >
> >> >> > This is a problem because BPEL processes and their instances are
> >> >> > potentially
> >> >> > long-running,they usually run over years. Thus, a serialized OModel
> >> >> > should
> >> >> > survive ODE and JDK upgrades. Our problem is that even simple
> changes
> >> >> > (that
> >> >> > are needed to implement new features or even to fix some bugs)
> break
> >> >> > backward compatibility in the OModel. We need a good solution
to
> >> >> > address
> >> >> > that problem.
> >> >> >
> >> >> > What Shall We Do To Solve This Problem
> >> >> >
> >> >> > This project want to solve this problem, we will change .cbp file's
> >> >> > format
> >> >> > and use a brand new serialize solution to keep compatibility.
After
> >> >> > investigate possible approaches, we would like to use JSON to
> >> >> > represent
> >> >> > OModel(It means new .cbp file will be JSON format). About the
above
> >> >> > example,
> >> >> > the old JSON format .cbp file can be de-serialized into the new
> >> >> > OProcess
> >> >> > class with a special migrate condition description string.
> >> >> >
> >> >> > Json Format
> >> >> >
> >> >> > Jackson is actually a JSON processor (JSON parser + JSON generator)
> >> >> > written
> >> >> > in Java. Beyond basic JSON reading/writing (parsing, generating),
> it
> >> >> > also
> >> >> > offers full node-based Tree Model, as well as full OJM (Object/Json
> >> >> > Mapper)
> >> >> > data binding functionality. More important, it has good
> performance,
> >> >> > even
> >> >> > faster than Java's built-in means.
> >> >> >
> >> >> > So we decided to use Json format file store OModel's serialize
> >> >> > result,
> >> >> > and
> >> >> > Jackson 2.0 is the potential solution to reading/writing OModel
> >> >> > instance
> >> >> > from/to Json format .cbp file.
> >> >> >
> >> >> > We can use Jackson to store attributes of Current OProcess class
to
> >> >> > Json
> >> >> > format file like this:
> >> >> >
> >> >> >         Map<String,Object> userData1 = new
> HashMap<String,Object>();
> >> >> >
> >> >> >         userData1.put("owner", process.getOwner());
> >> >> >
> >> >> >         userData1.put("name", process.getName());
> >> >> >
> >> >> >         userData1.put("targetNameSpace", process.targetNamespace);
> >> >> >
> >> >> >         userData1.put("processName", process.processName);
> >> >> >
> >> >> >         userData1.put("children", process.getChildren());
> >> >> >
> >> >> >
> >> >> >         ObjectMapper mapper = new ObjectMapper();
> >> >> >
> >> >> >         StringWriter sw = new StringWriter();
> >> >> >
> >> >> >         org.codehaus.jackson.JsonGenerator gen;
> >> >> >
> >> >> >         try {
> >> >> >                 gen = new JsonFactory().createJsonGenerator(sw);
> >> >> >
> >> >> >                 mapper.writeValue(gen, userData1);
> >> >> >
> >> >> >                 gen.close();
> >> >> >                 String json = sw.toString();
> >> >> >
> >> >> >                 System.out.println(json);
> >> >> >
> >> >> >         } catch (IOException e) {
> >> >> >
> >> >> >                 e.printStackTrace();
> >> >> >         }
> >> >> >
> >> >> > The target json file will be:
> >> >> >
> >> >> >
> >> >> >
> >> >> > {"targetNameSpace":"http://ode/bpel/unit-test
> ","name":"HelloWorld2","processName":"HelloWorld2"}
> >> >> >
> >> >> > But not all the OModel instance can be serialized only using
> Jacksons
> >> >> > mixin
> >> >> > feature, we have to extend Jackson and would probably need to
write
> >> >> > custom
> >> >> > (de)serializers for WSDL4J etc.
> >> >> >
> >> >> > OModel Migrate Mechanism
> >> >> >
> >> >> > Here we use a simple example proposed by my potential mentor Tammo
> >> >> > van
> >> >> > Lessen to describe the migrate mechanism.
> >> >> >
> >> >> > Simple Example
> >> >> >
> >> >> > OProcess stores currently targetNamespace and processName as
> Strings.
> >> >> > Now
> >> >> > imagine we change OProcess so that it stores a QName instead.
When
> >> >> > loading
> >> >> > an old OModel, the QName field would be null and there would be
the
> >> >> > two
> >> >> > String field in the "unknown fields" map.
> >> >> >
> >> >> > Migrate Solution
> >> >> >
> >> >> > Suppose the old version of OProcess is stored in following format:
> >> >> >
> >> >> >
> >> >> >
> >> >> > {"targetNameSpace":"http://ode/bpel/unit-test
> ","name":"HelloWorld2","processName":"HelloWorld2"}
> >> >> >
> >> >> > and the corresponding new version of OProcess's json .cbp file
will
> >> >> > be:
> >> >> >
> >> >> >
> >> >> >
> >> >> > {"name":"HelloWorld2","namespace":"{
> http://ode/bpel/unit-test}HelloWorld2"}
> >> >> >
> >> >> > The migrate job is transform from one JSON to another JSON in
a
> >> >> > specific
> >> >> > rule, We should consider how to define the migrate description
> >> >> > language
> >> >> > or
> >> >> > just re-use some existing JSON migrate DSL.
> >> >> >
> >> >> > This migrate DSL may be the most important part of this project.
In
> >> >> > this
> >> >> > simple demo, maybe looks like this:
> >> >> >
> >> >> > {
> >> >> >    "from":{
> >> >> >
> >> >> >       "name":"$value0",
> >> >> >
> >> >> >       "targetNameSpace":"$value1",
> >> >> >
> >> >> >       "processName":"$value2"
> >> >> >
> >> >> >    },
> >> >> >    "to":{
> >> >> >
> >> >> >       "name":"$value0",
> >> >> >
> >> >> >       "namespace":"{$value1}$value2"
> >> >> >
> >> >> >    }
> >> >> > }
> >> >> >
> >> >> > Use $value to represent migrate attribute values.
> >> >> >
> >> >> > Domain-specific language(DSL)
> >> >> >
> >> >> > Once we update OModel, and want to migrate the old version to
new
> >> >> > version.
> >> >> > If we have both the old version Json format .cbp file and the
> >> >> > specificed
> >> >> > migrate rule, we can get the new version Json format .cbp file,
> then
> >> >> > load as
> >> >> > new OProcess object, finish the migrate job easily.
> >> >> >
> >> >> > We are still survey for exist tools like XSLT, working on JSON
> Trees.
> >> >> > If
> >> >> > can
> >> >> > not find such a tool, have both the old OModel and the new OModel,
> we
> >> >> > can
> >> >> > write a tool to generate migrate DSL string.
> >> >> >
> >> >> > Of course, we must specific the migrate rule for different version
> >> >> > change,
> >> >> > such as v1.0->v1.1, v1.1->v1.2 and v1.0->v1.2.
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Best Regards From Tiger Gui
> >> >> > --------------------------------------
> >> >> > Open source is some kind of life attitude
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Tammo van Lessen - http://www.taval.de
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Best Regards From Tiger Gui
> >> > --------------------------------------
> >> > Open source is some kind of life attitude
> >>
> >>
> >>
> >> --
> >> Tammo van Lessen - http://www.taval.de
> >
> >
> >
> >
> > --
> > Best Regards From Tiger Gui
> > --------------------------------------
> > Open source is some kind of life attitude
>
>
>
> --
> Tammo van Lessen - http://www.taval.de
>



-- 
Best Regards From Tiger Gui
--------------------------------------
Open source is some kind of life attitude

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message