ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nikita Amelchev <nsamelc...@gmail.com>
Subject Re: Transport compression (not store compression)
Date Thu, 01 Feb 2018 15:03:37 GMT
Thanks for your comments,

1. At first, I would define final design and then I will make it pluggable.

2. We cannot prioritize messages at this network communication level. If
compression utilizes processor poorly then we can increase count of
selectors (make more threads when compression enabled).

These benchmarks show how much bytes and time will be saved on transfer
messages with different length and regularity. Tests provide data which
have a simple physical meaning. The use of "real" operations will give the
unstable result and will change as you change the implementation of these
"real" operations. Test that generated a lot of puts will also be
synthetic, just like any other.
However, I can try to come up with yardstick test with operations like a
real case. What will we test? Do we test putAll + getAll with text data?

2018-01-25 13:23 GMT+03:00 gvvinblade <gvvinblade@gmail.com>:

> Nikita,
>
> I took a look at the changes and see two issues.
>
> The first one is additional dependencies in core module. That means you
> should whether make the filter plugable and provide some additional module
> for it or put all the necessary classes to ignite-core.
>
> The second is possible bottleneck in NIO threads. Since each worker serves
> several connections we may face troubles on big clusters (for example one
> or
> two ignite server nodes and a few dozens client nodes) because
> compression/decompression is an expensive operation and happens (in your
> implementation) in NIO threads where message processing becames sequential.
>
> The provided benchmarks are quite synthetic (they are local, they involve
> only two nodes, they don't use "real" operations like put, get, putAll etc
> and don't show a real picture)
>
> We have to be careful with such changes and check all possible scenarios.
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>



-- 
Best wishes,
Amelchev Nikita

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message