commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen, Haifeng" <haifeng.c...@intel.com>
Subject RE: Chimera as new component in Apache Commons
Date Thu, 18 Feb 2016 02:06:12 GMT
Thanks Benedikt for your support!

>> I'm no crypto expert but I can help with the Apache Commons related tasks, like moving
the code over to Apache Commons, setting up the maven build, publishing the project website
etc.
This is really great help.

>> 1. There are no Apache <Component> sub communities. There is only the Apache
Commons community. This means, there won't be a separat mailing list for the new component.
It is important to understand that we are a community maintaining a number of components.
Not a group of sub communities.
Makes sense. Sub modules share the same Commons community. This doesn't conflict with the
fact that sub modules has its own release cycle and versioning.

>> 2. The Apache Commons versioning guide lines are very restrictive [1]. We put great
effort into binary compatibility. This is, because we expect our components to be reused by
a lot of other projects and we try our best to avoid jar hell. Often this means, that greater
refactorings simply can not be implemented since they would break BC. This is usually not
a problem for the major components. But it my be a problem for a young component.
Yes, keeping a stable API is very important for shared libraries. 

3. Apache Commons components usually have a (boring) descriptive name, rather then a fancy
one. This is the reason why we renamed Apache Commons Sanselan zu Apache Commons Imaging.
People should be able to tell just by looking at the name of a component what that component
is about. IMHO Chimera falls into the fancy name category, so maybe we will discuss that name.
There is no problem that Chimera be renamed to something like "crypto".

Thanks,
Haifeng


-----Original Message-----
From: Benedikt Ritter [mailto:britter@apache.org] 
Sent: Wednesday, February 17, 2016 3:57 PM
To: Commons Developers List <dev@commons.apache.org>
Cc: common-dev@hadoop.apache.org
Subject: Re: Chimera as new component in Apache Commons

Hello Uma,

welcome to the Apache Commons dev list. It's great to see that two projects get together to
share code via Apache Commons.

2016-02-16 22:36 GMT+01:00 Gangumalla, Uma <uma.gangumalla@intel.com>:

> Hi Devs,
>
>
>
> Recently we worked with Spark community to implement the shuffle 
> encryption. While implementing that, we realized some/most of the code 
> in Apache Hadoop encryption code and this implementation code have to 
> be duplicated. This leads to an idea to create separate reusable 
> library, named it as Chimera 
> (https://github.com/intel-hadoop/chimera). It is an optimized 
> cryptographic library. It provides Java API for both cipher level and 
> Java stream level to help developers implement high performance AES encryption/decryption
with the minimum code and effort.
>
>
>
> We know that Java has Cipher implementations, but why we need this 
> optimized cryptographic library:
>
> 1. Performance is very critical for encryption and decryption. The JDK 
> Cipher implementation of AES are not yet optimized with the modern 
> hardware. For example, the optimized implementation is 17x+ faster 
> than
> JDK6 implementations for some modes such as CBC decryption, CTR and GCM.
> Even some optimizations has included JDK7 or JDK8, there are still 5x 
> to 6x gap with the most optimized implementations.
>

That sounds pretty useful! :-)


>
> 2. Java Stream based API on cryptographic data stream. Cipher API is 
> powerful but a lot of code needs to be written for layered stream 
> processing applications. The design pattern is very common in modern 
> applications such as Hadoop or Spark.
>
>
>
> Chimera was originally based Hadoop crypto code but was improved and 
> generalized a lot for supporting wider scope of data encryption needs 
> for more components in the community. The encryption related code in 
> Hadoop was developed a year and so far it is running well. So we feel 
> that code part of stable enough already.
>
>
>
> So, we propose to contribute this Chimera (optimized encryption 
> library) code to Apache Commons and we wanted to have independent 
> release cycles for this module like any other modules in Apache 
> Commons. This module is basically provides Java based interfaces for 
> encryption based IO and It will have native based AES-NI encryption integration code.
>
>
>
> We already discussed about this proposal in Apache Hadoop dev lists 
> and the discussion conclusion was positive to contribute this module 
> to Apache Commons.
>
>
>
> We need your help and support in adopting this code to make as Apache 
> Commons sub module and establish for making it to have its own 
> development community (of course we can discuss more about this 
> factors in this thread). And Hadoop and Spark will be the two visible projects to use
it.
> We do expect there will be more projects using it.
>
>
I'm no crypto expert but I can help with the Apache Commons related tasks, like moving the
code over to Apache Commons, setting up the maven build, publishing the project website etc.


>
>
> Once Apache Commons PMC agreed to place this module under Commons, I 
> will work on getting the interested developers etc for establishing 
> Chimera development community as part of next steps. Please help on the process.
>

I'd love see you moving Chimera here. however, there are a few things I'd like to make you
aware of:

1. There are no Apache <Component> sub communities. There is only the Apache Commons
community. This means, there won't be a separat mailing list for the new component. It is
important to understand that we are a community maintaining a number of components. Not a
group of sub communities.
2. The Apache Commons versioning guide lines are very restrictive [1]. We put great effort
into binary compatibility. This is, because we expect our components to be reused by a lot
of other projects and we try our best to avoid jar hell. Often this means, that greater refactorings
simply can not be implemented since they would break BC. This is usually not a problem for
the major components. But it my be a problem for a young component.
3. Apache Commons components usually have a (boring) descriptive name, rather then a fancy
one. This is the reason why we renamed Apache Commons Sanselan zu Apache Commons Imaging.
People should be able to tell just by looking at the name of a component what that component
is about. IMHO Chimera falls into the fancy name category, so maybe we will discuss that name.

I hope these are no blockers for you.

Thank you for your interest and your effort in bringing new code to Apache Commons!

Best regards,
Benedikt

[1] https://commons.apache.org/releases/versioning.html


>
>
>
> Regards,
>
> Uma (An Apache Hadoop PMC member)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>


--
http://home.apache.org/~britter/
http://twitter.com/BenediktRitter
http://github.com/britter
Mime
View raw message