ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Kuznetsov <akuznet...@apache.org>
Subject Re: Transport compression (not store compression)
Date Fri, 02 Feb 2018 15:52:38 GMT
I think Igor is right.

Ususally servers connected via fast local network.
But clients could be in external and slow network.
In this scenario compression will be very useful.

Once I had such scenario - client connected to cluster via 300 kb/s network
and tries to transfer ~10Mb of uncumpressed data.
So it takse ~30 seconds.
After I implemented compression it becamed 1M and transfered for ~3 seconds.

I think we should take care of all mentioned problems with NIO threads in
order to not slow down whole cluster.


On Fri, Feb 2, 2018 at 10:05 PM, gvvinblade <gvvinblade@gmail.com> wrote:

> Nikita,
>
> Yes, you're right. Maybe I wasn't clear enough.
>
> Usually server nodes are placed in the same fast network segment (one
> datacenter); in any case we need an ability to setup compression per
> connection using some filter like useCompression(ClusterNode, ClusterNode)
> to compress traffic only between servers and client nodes.
>
> But issue is still there, since the same NIO worker serves both client and
> server connections, enabled compression may impact whole cluster
> performance
> because NIO threads will compress client messages instead of processing
> servers' compute requests. That was my concern.
>
> Compression for clients is really cool feature and usefull in some cases.
> Probably it makes sense to have two NIO servers with and without
> compression
> to process server and client requests separately or pin somehow worker
> threads to client or server sessions...
>
> Also we have to think about client connections (JDBC, ODBC, .Net thin
> client, etc) and setup compression for them separately.
>
> Anyway I would compare put, get, putAll, getAll and SQL SELECT operations
> for strings and POJOs, one server, several clients with and without
> compression, setting up the server to utilize all cores by NIO workers,
> just
> to get know possible impact.
>
> Possible configuration for servers with 16 cores:
>
> Selectors cnt = 16
> Connections per node = 4
>
> Where client nodes perform operations in 16 threads
>
> Regards,
> Igor
>
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>



-- 
Alexey Kuznetsov

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message