mina-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frédéric Brégier <fred.breg...@free.fr>
Subject Re: New project on starting block: OpenLSD using MINA
Date Mon, 14 May 2007 05:44:08 GMT
Hi Trustin,

The integration process should end during summer time.
As for the "official" part, it is not published or whatever
relative thing, but I am working on it to made it one of
the choice for the public administration.
This project should replace a software which is very
expensive (I talk of millions of euro) and not totally
what we want/need to do as document archiving.
So for now it is a project that must make its proof
in the real world (after integration so sometime
from september 2007). Of course, like any project
and those in public administration, "official" publications
will take place... once it is in production and bug free. ;-)

But I think you can still reference it at least as an on going
project as well as the performance test result.
Of course, whenever I get some news I will try to inform
MINA list asap ;-)

For instance, the next step will be, during integration,
new benchmark but on a 32 Power5 server.

Thank again for Mina people, without Mina I was in trouble
to make my "network" protocol efficient and evolutive.

PS: I use demuxing Io Handler, just to point it since it is not
very usual as I saw from the mailing list.

----- Original Message ----- 
From: "Trustin Lee"
Sent: Monday, May 14, 2007 4:05 AM
Subject: Re: New project on starting block: OpenLSD using MINA

Hi Frederic,

Thank you for sharing your test results, and I am very proud that MINA
is being used for such an important mission! :D

When will be the integration process end and when can we officially
list OpenLSD and Ministery of Finance in France as a reference and
post the performance test result?


On 5/13/07, Frederic Bregier wrote:
> Hi all,
> I am using MINA for one year now and I finally arrived at a point
> where my project is reaching a level that start to satisfy me a little.
> I post this not to make some publicity on my project but in case
> Trustin wants to add it as one of the project that use MINA or some
> benchmarks (at the end of this mail) review.
> I have not finishing everything and of course I am sure that some
> parts could be better, but I think it was time to open it to the 
> community.
> Of course I am open to any comment ! I do that on my spare time
> so obviously it is not perfect... ;-)
> This project, named OpenLSD, is about a framework for
> legal document archiving and specifically for huge amount of files,
> which stands for Open Legacy Storage Document.
> You can find some infor on it on sourceforge and/or on
> my web site : http://openlsd.free.fr/en/OpenLSDandCo.html
> My employer (Ministery of Finance in France) has, like a lot
> of administrations, some problems with electronic archiving:
> - How to archive in electronic form about hundred of TeraBytes
>   or even thousand of Terabytes
> - How to be able to access them in a secure and in a performant way
>   (well, Mina came here just at the right time)
> - How to enable to adapt the code as easy as possible to new
>    problem (my next focus will be OpenLSM for Open Legacy
>    Storage Mail for email legal archiving)
> - How to ensure security on storage (crypto on files for instance)
> So I decided to develop something that could answear to the
> various problems that we have to face.
> It is currently starting its integration in the Ministery IT.
> Of course, it should be also possible to use it for smaller storage
> like departemental archive.
> The project is not finished (will it be one day ?).
> Today there is an example of a simple application (simple business)
> based on this framework, using Oracle as database, MINA as network
> framework and NIO also on filesystem access.
> The next steps will be to correct MySQL support (my compagny has to use
> Oracle so the priority on this one but I merge regularly the 
> capabilities),
> to enable cache support (read and write), to enable multiple OpenLSD
> server (mirroring by OpenLSD for security and performance aspect),
> to enable PostGreSQL also (I use JDBC but some optimization can be done
> with each database).
> Last but not least, my benchmark can be resume as follow :
> - Import of 10 KB documents at a rate of 1420 files/second
>      (so 1420 messages by second using MINA as framework between
>     1 server and 8 MINA clients)
>     The main time is in the file handling not in the MINA's part.
>     The server was a 8 CPU Power5 AIX 5.3 IBM server using
>     the IBM JVM 1.5.
> - Retrieve 1000 documents by second through network (3 Tomcat servers
>     using servlet with MINA client connected to OpenLSD MINA server)
>     with 88 Mbits of bandwith (fully using MINA support) with a mean time
>     of 0,012 seconds by document
>     The Tomcat servers are 2-Xeon CPU 64bits using Linux (Suse 10),
>     IBM JDK 1.5 and Tomcat 5.5. The LSD Server (storage and MINA server)
>     was on a 2 CPU Power5 AIX 5.3 IBM server using the IBM JVM 1.5.
> - Another simple test (without Database persistence) shows that I was
> able to
>     get 2500 documents of 10KB by second using 1 Tomcat server with
>     each servlet connecting to OpenLSD server using MINA, so
>     400 Mbits of bandwith.
>     The Tomcat server is a 2-Xeon CPU 64bits using Linux (Suse 10),
>     IBM JDK 1.5 and Tomcat 5.5. The LSD Server (storage and MINA server)
>     was on a 2 CPU Power5 AIX 5.3 IBM server using the IBM JVM 1.5.
> Other benchmarks were done but are not relative to MINA, so I just put
> it here to be compelete:
> - Import in the same condition but in crypted mode (files are save using
> a key)
>     gives 1290 files / second. (same condition as normal import)
> - Check of consistency database/filesystem is done at 2400 files/second
>     when MD5 computation is done (50 MB/s or 400 Mbits on a 2Gb SAN)
> - Check of consistency database/filesystem is done at 30 000 files/second
>     without MD5 computation is done (80 MB/s or 640 Mbits on a 2Gb SAN)
> Just to be complete, the version of MINA is the trunk one.
> I hope other projects will come on and on using MINA, a great piece of
> software!!!
> Thank you all, since MINA is powerful thanks to all its users.
> Frederic

what we call human nature is actually human habit
PGP Key ID: 0x0255ECA6

View raw message