ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Kuznetsov <akuznet...@apache.org>
Subject Re: Ignite as distributed file storage
Date Fri, 03 Aug 2018 11:39:05 GMT
Pavel,

Will this internal storage mounted on all nodes hosts?
Otherwise we will have a huge traffic on rebalance of blobs.

On Fri, Aug 3, 2018 at 5:21 PM Pavel Kovalenko <jokserfn@gmail.com> wrote:

> Dmitriy,
>
> I think we don't need a separate implementation of cache like BLOB cache.
> Instead of it, a user can mark value class or value class field with the
> special annotation "@BLOB".
> During cache put, marshaller will place a special placeholder on such
> fields, write byte[] array payload of a field to special internal blob
> storage and place the only reference to actual DataEntry in the page
> memory.
> During cache get, marshaller will place a special proxy instead of an
> actual class that can be downloaded and unmarshalled by demand from
> internal storage on the user side.
> Using such approach we will also solve eager/lazy problem, this will also
> give user possibility to adjust his own marshallers (like Jackson, Jaxb,
> etc.) to marshal/unmarshal his blob classes from/to byte[] arrays.
> No major changes in public API are required, it can be pluggable component.
>
>
> 2018-08-03 0:25 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org>:
>
> > On Thu, Aug 2, 2018 at 1:08 AM, Pavel Kovalenko <jokserfn@gmail.com>
> > wrote:
> >
> > > Dmitriy,
> > >
> > > I still don't understand why do you think that it will be file system?
> > > In all my previous messages I emphasized that this storage shouldn't be
> > > considered as a file system. It's just a large data storage, whose
> > entities
> > > can be easily accessed using key/link (internally, or externally using
> > > web/binary protocol interfaces).
> > >
> > > > Instead, if we must focus on large blobs, I would solve the problem
> of
> > > supporting large blobs in regular Ignite caches, as I suggested before.
> > >
> > > This is impossible. Our page memory can't handle efficiently it by
> > design.
> > >
> >
> > But our API does. What is stopping us from creating a cache as a BLOB
> cache
> > and using whatever storage we need?
> >
>
> --
> Alexey Kuznetsov
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message