ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Setrakyan <dsetrak...@apache.org>
Subject Re: OSGi migration may require a special marshaller
Date Tue, 03 Nov 2015 00:18:13 GMT
Raul,

Why do we need TCCL for serializing? We already have the Object we want to
serialize, so to get its class loader we can call
myObject.getClass().getClassLoader(), no?

D.

On Mon, Nov 2, 2015 at 3:20 PM, Raul Kripalani <raul@evosent.com> wrote:

> Hi Andrey,
>
> Thanks for the participation in this topic.
>
> I don't like the solution to incorporate the bundle symbolic name in the
> serialised form. Nothing guarantees that the classdef will be located under
> the same bundle in both source and target machines. We also have to take
> into account that OSGi allows for multiple versions of the same
> bundle/packages to co-exist in the same  container. So it becomes more
> complex.
>
> Using the TCCL should work when serialising, but it'll probably be of no
> use when deserialising on the other end.
>
> I need to enhance Ignite to:
>
> 1. Use the TCCL when marshalling on one end.
> 2. Incorporate the package version of the class in the serialised form if
> Ignite is running in an OSGi environment.
> 3. On the receiver end, discover cache entities / DTOs in all bundles
> through a custom OSGi manifest header or the like, as I explained before.
> Package version must be taken into account.
>
> What do you think?
>
> Raúl.
> On 2 Nov 2015 17:25, "Andrey Kornev" <andrewkornev@hotmail.com> wrote:
>
> > Raul,
> >
> > The fundamental hurdle we need to jump over to make Ignite OSGi-enabled
> is
> > the marshalling. More specifically the issue is with deserialization of
> the
> > classes that are provided by the bundles other than the Ignite bundle
> > itself.
> >
> > When the Ignite transport layer receives a message it needs to figure out
> > how to deserialize the bytes and for that it needs to know the bundle
> that
> > provides the class to be deserailized. At this point TCCL is of no use.
> To
> > make things more complex, the class may contain other classes that come
> > from other bundles, and so on recursively. This means that each object in
> > the hierarchy must be serialized with its bundle name (or bundle id), so
> > that the deserializer will then be able to correctly resolve the class
> > while traversing the object hierarchy during deserialization.
> >
> > Unfortunately, Ignite's OptimizedMarshaller is lacking the ability to
> plug
> > in a custom class resolver. Romain's solution was to use Kryo that does
> > provide a way to customize class resolution. It has solved the problem of
> > capturing the bundle info and he was able to successfully run Ignite as a
> > bundle in an OSGi container (after some repackaging and inclusion of the
> > manifest). But Kryo-based marshalling introduced a lot of complexity to
> the
> > code and incorrect use of Kryo's numerous serializers caused some weird
> > hard-to-debug issues in the Ignite core (like duplicate cache entries due
> > to incorrect marshalling of the GridDhtPArtitonFullMap class -- go
> > figure!). Overall the Kryo-based solution is brittle and hard to
> maintain.
> >
> > I feel the correct solution to OSGi problem would be to
> > 1) enhance the OptimizedMarshaller to allow custom class resolution.
> > 2) provide an OSGi-enabled OptimizedMarshaller (in addition to the
> > original one) to be used in OSGi environment.
> >
> > Regards
> > Andrey
> >
> > > From: raulk@apache.org
> > > Date: Mon, 2 Nov 2015 12:41:47 +0000
> > > Subject: Re: OSGi migration may require a special marshaller
> > > To: dev@ignite.apache.org
> > >
> > > Hi Romain,
> > >
> > > I'm working on the OSGi compatibility of Ignite. I appreciate your
> input.
> > >
> > > I'm thinking about the situation you describe and I wonder if you're
> > > exporting Ignite as an OSGi service which is then consumed from other
> > > bundles. Under this situation, it would be quite easy to reproduce the
> > > behaviour you describe if Ignite is not resolving classes via the TCCL.
> > > Need to dig deeper into that.
> > >
> > > Off the top of my head, there are two alternatives to solve it:
> > >
> > > 1. Use the TCCL for marshalling/unmarshalling (if not already used) –
> we
> > > gotta be wary of possible regressions.
> > > 2. Create a special OSGi header Ignite-Export-Package so that bundles
> > > containing DTOs can expose packages to Ignite's marshallers.
> > >
> > > Regards,
> > >
> > > *Raúl Kripalani*
> > > PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data
> and
> > > Messaging Engineer
> > > http://about.me/raulkripalani |
> http://www.linkedin.com/in/raulkripalani
> > > http://blog.raulkr.net | twitter: @raulvk
> > >
> > > On Mon, Nov 2, 2015 at 9:56 AM, Gilles, Romain <
> romain.gilles@misys.com>
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I'm really interested in this issue:
> > > > https://issues.apache.org/jira/browse/IGNITE-1270 . We some stuff to
> > make
> > > > it work in our osgi environment. The main issue for us now is the
> > > > serialization. I think it you will have to rework the
> > OptimizedMarshaller
> > > > or any other marshaller that works with object that come from outside
> > your
> > > > class space.
> > > >
> > > > We have try kryo that works. Kryo provide an extension point in order
> > to
> > > > resolve the classes:
> > > >
> >
> https://github.com/EsotericSoftware/kryo/blob/master/src/com/esotericsoftware/kryo/ClassResolver.java
> > > > . With this extension we are able to solve the problem of external
> > classes.
> > > > The only issue with kryo is that some classes need a certain care in
> > the
> > > > serialization process and therefore a specialized serializer.
> > > >
> > > > So I would like to know from the community what do think of changing
> > the
> > > > way the optimized marshaller works or introducing the support of yet
> > > > another marshaller based on a kryo like technology?
> > > >
> > > >
> > > > Thanks in advance,
> > > >
> > > >
> > > > Best regards,
> > > >
> > > >
> > > > Romain.
> > > >
> > > >
> > > > PS: I'm ready to help in the both cases.
> > > > "Misys" is the trade name of the Misys group of companies. This email
> > and
> > > > any attachments have been scanned for known viruses using multiple
> > > > scanners. This email message is intended for the named recipient
> only.
> > It
> > > > may be privileged and/or confidential. If you are not the named
> > recipient
> > > > of this email please notify us immediately and do not copy it or use
> > it for
> > > > any purpose, nor disclose its contents to any other person. This
> email
> > does
> > > > not constitute the commencement of legal relations between you and
> > Misys.
> > > > Please refer to the executed contract between you and the relevant
> > member
> > > > of the Misys group for the identity of the contracting party with
> > which you
> > > > are dealing.
> > > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message