ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pavel Tupitsyn <ptupit...@apache.org>
Subject Re: IEP-68: Thin Client Data Streamer
Date Fri, 05 Mar 2021 12:54:22 GMT
> How do you suggests to serialize this object?

Like a normal binary object. This scenario already exists for ScanQuery and
ContinuousQuery filters.
- It works for Java and .NET; potentially for other platforms
- Corresponding types should exist on server nodes

E.g. if a Java class `MyFilter { String containsText }` is loaded on server
nodes,
a Python thin client can easily write a corresponding BinaryObject with one
field to achieve server-side filtering.


> I also suppose, that closing should be done wia OP_RESOURCE_CLOSE

Thanks, forgot to mention this - updated the IEP.


On Fri, Mar 5, 2021 at 3:42 PM Ivan Daschinsky <ivandasch@gmail.com> wrote:

> I also suppose, that closing should be done wia OP_RESOURCE_CLOSE. It'is
> more consistent and resembles with existing cursor api.
>
> пт, 5 мар. 2021 г. в 15:37, Ivan Daschinsky <ivandasch@gmail.com>:
>
> > >> Both proposed requests have a Flush flag - please see Details sections
> > in the IEP.
> > Ok, sorry, I missed this.
> > >> StreamReceiver is public API [1]
> > I know it. But this "object" should contains custom logic for putting
> data
> > in cache. How do you suggests to serialize this object?
> > Implement custom classloader for java? What about other platforms?
> >
> > I suppose that we should not add this field, because it is confusing.
> > Everything will work for local unit tests and only for java.
> > But in real environment this will not work at all.
> >
> >
> > пт, 5 мар. 2021 г. в 15:23, Pavel Tupitsyn <ptupitsyn@apache.org>:
> >
> >> Ivan,
> >>
> >> > Receiver is mostly internal stuff
> >> StreamReceiver is public API [1]
> >>
> >> > STREAMER_FLUSH_REQUEST should be added too
> >> Both proposed requests have a Flush flag - please see Details sections
> in
> >> the IEP.
> >> When user code calls client-side Flush method, we send the current
> >> client-side batch with that flag enabled,
> >> and flush server-side batches too.
> >>
> >> A separate request for that does not seem to be necessary, or am I
> missing
> >> some different use case?
> >>
> >> [1]
> >>
> >>
> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/stream/package-summary.html
> >>
> >>
> >> On Fri, Mar 5, 2021 at 2:50 PM Ivan Daschinsky <ivandasch@gmail.com>
> >> wrote:
> >>
> >> > IMHO, also STREAMER_FLUSH_REQUEST should be added too. Client can
> flush
> >> > large batches instead of closing resource.
> >> > IMHO this is preferable than creating streamer per batch.
> >> >
> >> > пт, 5 мар. 2021 г. в 14:48, Ivan Daschinsky <ivandasch@gmail.com>:
> >> >
> >> > > Pavel, thank you for your effort.
> >> > >
> >> > > BTW, are you sure that receiver should be a param of
> >> > > STREAMER_START_REQUEST?
> >> > > Receiver is mostly internal stuff and I can hardly imagine why
> >> > > someone would decide to change defaults.
> >> > >
> >> > > пт, 5 мар. 2021 г. в 13:01, Pavel Tupitsyn <ptupitsyn@apache.org>:
> >> > >
> >> > >> Igor,
> >> > >>
> >> > >> As per our private conversation, I'll try to provide some perf
> >> > >> measurements
> >> > >> for those 4 variants, and more detailed descriptions too.
> >> > >>
> >> > >> Thanks!
> >> > >>
> >> > >> On Fri, Mar 5, 2021 at 12:55 PM Igor Sapego <isapego@apache.org>
> >> wrote:
> >> > >>
> >> > >> > Pavel,
> >> > >> >
> >> > >> > I've checked the IEP and I like it. The only thing that seems
a
> bit
> >> > >> > confusing to me
> >> > >> > is that there are 4 different variants for clients but there
are
> >> cons
> >> > >> and
> >> > >> > pros for
> >> > >> > different variants. Maybe at least few sentences should be
> written
> >> > here
> >> > >> to
> >> > >> > give developers who are not familiar with DataStreamer some
ideas
> >> and
> >> > >> make
> >> > >> > it easier for them to choose.
> >> > >> >
> >> > >> > Best Regards,
> >> > >> > Igor
> >> > >> >
> >> > >> >
> >> > >> > On Thu, Mar 4, 2021 at 3:14 PM Pavel Tupitsyn <
> >> ptupitsyn@apache.org>
> >> > >> > wrote:
> >> > >> >
> >> > >> > > Igniters,
> >> > >> > >
> >> > >> > > Please review the IEP [1] and let me know what you think.
> >> > >> > >
> >> > >> > > [1]
> >> > >> > >
> >> > >> > >
> >> > >> >
> >> > >>
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-68%3A+Thin+Client+Data+Streamer
> >> > >> > >
> >> > >> >
> >> > >>
> >> > >
> >> > >
> >> > > --
> >> > > Sincerely yours, Ivan Daschinskiy
> >> > >
> >> >
> >> >
> >> > --
> >> > Sincerely yours, Ivan Daschinskiy
> >> >
> >>
> >
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
> >
>
>
> --
> Sincerely yours, Ivan Daschinskiy
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message