commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gangumalla, Uma" <uma.ganguma...@intel.com>
Subject Re: [crypto][chimera] Next steps
Date Thu, 17 Mar 2016 06:30:06 GMT

Dear All,

Please find the proposal document text for Apache Commons Crypto project
below and the same has been committed at Chimera project home folder [1] .

Please provide your feedbacks and let me know if anything else need to
cover in this document.

Thanks a lot for the help and support so far.

[1]  https://github.com/intel-hadoop/chimera/blob/master/PROPOSAL.html
<https://github.com/intel-hadoop/chimera/blob/master/PROPOSAL.html>https://
github.com/intel-hadoop/chimera/blob/master/PROPOSAL.html>


<PROPOSAL TEXT STARTS>


Proposal for Apache Commons Crypto Package

(0) Rationale

Providing Java based optimized and high performance cryptographic IO
streams for the applications who wants to implement the data encryption.
It also provides cipher level API to use. It does provide the openssl API
integration and provide the fallback mechanism to use JCE when openssl
library unavailable.
(Note: Please note that Commons Crypto doesn’t implement the cryptographic
algorithm such as AES directly. It wraps to Openssl or JCE which implement
algorithms.)

(1) Scope of the Package

This proposal is to create a package of cryptographic IO classes with the
integration of Openssl library.
It focuses on AES-NI optimizations mainly and it can be extended to other
algorithms based on demand from the users later.

(1.5) Interaction With Other Packages

Commons Crypto relies on standard JDK 7 (or later) APIs for production
deployment and on OpenSSL 1.0.1c devl libraries.
It utilizes the JUnit unit testing framework, but this is of interest only
to developers of the component.
The functionality provided by Commons Crypto is currently in use by Apache
Hadoop and Apache Spark, and both of those communities
have expressed interest in changing their dependency to be on the central
Commons Crypto package once it exists.

(2) Initial Source of the Package

The initial classes came from the Apache Hadoop.
The proposed package name for the new component is
org.apache.commons.crypto.

(3) Required Apache Commons Resources

* Git Repository - New repository commons-crypto
* Mailing List - Discussions will take place on the general
dev@commons.apache.org mailing list.
  To help list subscribers identify messages of interest, it is suggested
that the message subject of messages about this component be prefixed with
[Crypto].
* JIRA - New component "Crypto" under the "Commons" project.
* Confluence FAQ - New category "commons-crypto" (when available).

(4) Initial Committers (Names in alphabetical order)

* Aaron T. Myers (atm@apache.org, Apache Hadoop PMC, one of the original
Crypto dev team in Apache Hadoop)
* Andrew Wang (wang@apache.org, Apache Hadoop PMC, one of the original
Crypto dev team in Apache Hadoop)
* Chris Nauroth (cnauroth@apache.org, Apache Hadoop PMC and active
reviewer)
* Colin P. McCabe (cmccabe@apache.org, Apache Hadoop PMC, one of the
original Crypto dev team in Apache Hadoop)
* Dapeng Sun (sdp@apache.org, Apache Sentry Committer, Chimera contributor)
* Dian Fu (dianfu@apache.org, Apache Sqoop Committer, Chimera contributor)
* Dong Chen (dongc@apache.org, Apache Hive Committer,interested on Chimera)
* Ferdinand Xu (xuf@apache.org, Apache Hive Committer, Chimera contributor)
* Haifeng Chen (haifengchen@apache.org, Chimera lead and code contributor)
* Marcelo Vanzin (Apache Spark Committer, Chimera contributor)
* Uma Maheswara Rao G (umamahesh@apache.org, Apache Hadoop PMC, One of the
original Crypto dev/review team in Apache Hadoop)
* Yi Liu (yliu@apache.org, Apache Hadoop PMC, One of the original Crypto
dev/review team in Apache Hadoop)


<PROPOSAL TEXT ENDS>


Regards,
Uma

On 3/7/16, 5:20 PM, "Gangumalla, Uma" <uma.gangumalla@intel.com> wrote:

>Dear all, Sorry for the delay. Working out on the proposal document, we
>will get it posted here soon.
>
>Regards,
>Uma
>
>
>On 2/26/16, 1:26 PM, "Gary Gregory" <garydgregory@gmail.com> wrote:
>
>>I take it the Crypto squared is a typo and that you wanted to close the
>>quote.
>>
>>G
>>
>>On Fri, Feb 26, 2016 at 12:47 PM, Gangumalla, Uma
>><uma.gangumalla@intel.com>
>>wrote:
>>
>>>
>>>Ok. Thanks Benedikt. We would get the proposal document ready soon.
>>>Also one question, shall we rename the package to org.apache.* in
>>>Chimera
>>>git project first before pushing the code to Apache Commons? Or we can
>>>work here once moved the code?
>>>What do you suggest? For making package rename, component name should be
>>>finalized I think.
>>>
>>>Does every one ok with "Apache Commons Crypto² ?
>>>
>>>Regards,
>>>Uma
>>>
>>>On 2/26/16, 5:11 AM, "Benedikt Ritter" <britter@apache.org> wrote:
>>>
>>>>Okay, so I think the only thing missing before we start the
>>>integration of
>>>>the component is a PROPOSAL document, describing the scope of the
>>>>component
>>>>and a list of initial commiters/contributors.
>>>>
>>>>2016-02-26 3:24 GMT+01:00 Chen, Haifeng <haifeng.chen@intel.com>:
>>>>
>>>>> Come back to clear out the codebase and IP concerns.
>>>>>
>>>>> [Benedikt] I still have concerns about the IP, since this seems to
>>>be an
>>>>> Intel codebase.
>>>>> I have checked this. Chimera was developed as Apache 2 License from
>>>>>ground
>>>>> up. Agree with Jochen that the license matters.
>>>>> Internally, this was approved as part of larger open source efforts
>>>on
>>>>> Apache Hadoop and related projects in Intel. We have IP plan
>>>considered
>>>>>as
>>>>> part the open source process.
>>>>>
>>>>> As to the codebase, such as the package name is com.intel prefixed,
>>>it
>>>>>was
>>>>> technical decision when using com.intel.chimera as the package
>>>prefix.
>>>>>We
>>>>> original planned to use org.apache.chimera prefix. But we found that
>>>we
>>>>> couldnot publish org.apache. grouped artifacts to maven central
>>>>>repository,
>>>>> which needs to somewhat ownership for org.apache domain.
>>>>>
>>>>> To resolve the codebase problem, once all things are ready from
>>>Commons,
>>>>> we rename in a branch. And the branched code can be copied into
>>>Commons
>>>>> github as final.
>>>>>
>>>>> Thanks,
>>>>> Haifeng
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: Chen, Haifeng [mailto:haifeng.chen@intel.com]
>>>>> Sent: Wednesday, February 24, 2016 12:40 PM
>>>>> To: Commons Developers List <dev@commons.apache.org>
>>>>> Cc: common-dev@hadoop.apache.org
>>>>> Subject: RE: [crypto][chimera] Next steps
>>>>>
>>>>> >> The same should be there with Chimera/Apache Crypto.
>>>>> Yes, current implementation will fallback to JCE Cipher if native is
>>>not
>>>>> available.
>>>>>
>>>>> [Uma] we would fix up IP issues if any sooner. If you see all the
>>>code
>>>>> file license header is with Apache License files.
>>>>> The current repo and package structure there with name Intel. I will
>>>>>check
>>>>> with Haifeng on resolution of this.
>>>>> I will work with this ASAP for clear this out.
>>>>>
>>>>> Thanks,
>>>>> Haifeng
>>>>>
>>>>> -----Original Message-----
>>>>> From: Gangumalla, Uma [mailto:uma.gangumalla@intel.com]
>>>>> Sent: Wednesday, February 24, 2016 5:13 AM
>>>>> To: Commons Developers List <dev@commons.apache.org>
>>>>> Cc: common-dev@hadoop.apache.org
>>>>> Subject: Re: [crypto][chimera] Next steps
>>>>>
>>>>> Thanks all for the valuable feedbacks and discussions.
>>>>> Here are my replies for some of the questions..
>>>>> [Mark wrote]
>>>>> It depends. I care less about the quality of the code than I do about
>>>>>the
>>>>> community that comes with it / forms around it. A strong community
>>>can
>>>>>fix
>>>>> code issues. Great code can't save a weak community.
>>>>> [uma] Nice point. Fully agreed to it.
>>>>>
>>>>>
>>>>> [Jochen wrote]
>>>>> Therefore, I suggest that you provide at least fallback
>>>implementations
>>>>>in
>>>>> pure Java, which are being used, if the JNI based stuff is not
>>>available
>>>>> (for whatever reason).
>>>>> [Uma] Thank you for the suggestion Jochen, If I understand your point
>>>>> right,  Yes its there in Hadoop when we develop.
>>>>> Here is the JIRA HADOOP-10735 : Fall back AesCtrCryptoCodec
>>>>>implementation
>>>>> from OpenSSL to JCE if non native support.
>>>>>
>>>>> The same should be there with Chimera/Apache Crypto.
>>>>>
>>>>>
>>>>> [Benedikt]
>>>>> I still have concerns about the IP, since this seems to be an Intel
>>>>> codebase. I do not have the necessary experience to say what would be
>>>>>the
>>>>> right way here. My gut feeling tells me, that we should go through
>>>the
>>>>> incubator. WDYT?
>>>>> And [Jochen wrote]
>>>>> "An Intel codebase" is not a problem as such. Question is: "Available
>>>>> under what license?"
>>>>>
>>>>> [Uma] we would fix up IP issues if any sooner. If you see all the
>>>code
>>>>> file license header is with Apache License files.
>>>>> The current repo and package structure there with name Intel. I will
>>>>>check
>>>>> with Haifeng on resolution of this.
>>>>>
>>>>>
>>>>> [Jochen wrote]
>>>>> So, have the Chimera project attempt to resolve them quickly. If
>>>>> possible: Fine. If not: We still have the Incubator as a possibility.
>>>>> [Uma] Agree. We would resolve on this points in sooner.
>>>>>
>>>>>
>>>>> Regards,
>>>>> Uma
>>>>>
>>>>>
>>>>> On 2/23/16, 1:18 AM, "Mark Thomas" <markt@apache.org> wrote:
>>>>>
>>>>> >On 23/02/2016 09:12, sebb wrote:
>>>>> >> On 23 February 2016 at 07:34, Benedikt Ritter <britter@apache.org>
>>>>> >>wrote:
>>>>> >>> I'm confused. None of the other PMC members has expressed
>>>whether he
>>>>> >>>or she  want's the see Chimera/crypto joining Apache Commons,
yet
>>>>> >>>we're already  discussing how JNI bindings should be handled.
>>>>> >>>
>>>>> >>> I'd like to see:
>>>>> >>> 1) a clear statement whether Chimera/crypto should become
part of
>>>>> >>>Apache  Commons. Do we need a vote for that?
>>>>> >>
>>>>> >> Yes, of course.
>>>>> >>
>>>>> >> However that decision clearly depends on at least some of the
>>>design
>>>>> >> aspects of the code.
>>>>> >> If it were written entirely in C or Fortran, it would not be
a
>>>>> >> suitable candidate.
>>>>> >>
>>>>> >>> 2) Discuss a plan on how to do that (I've described a possible
>>>plan
>>>>> >>>[1])
>>>>> >>> 3) After that is clear: discuss design details regarding
the
>>>>>component.
>>>>> >>
>>>>> >> Some design details impact on the decision.
>>>>> >>
>>>>> >> Indeed even for pure Java code the code quality has a bearing
on
>>>>> >> whether Commons would/could want to take it.
>>>>> >> Would we want a large code base with no unit-tests, no build
>>>>> >> mechanism, and no comments?
>>>>> >
>>>>> >It depends. I care less about the quality of the code than I do
>>>about
>>>>> >the community that comes with it / forms around it. A strong
>>>community
>>>>> >can fix code issues. Great code can't save a weak community.
>>>>> >
>>>>> >How about creating a new sandbox component, let folks start work
and
>>>>> >see how the community develops?
>>>>> >
>>>>> >Mark
>>>>> >
>>>>> >
>>>>> >>
>>>>> >>> Thanks! :-)
>>>>> >>> Benedikt
>>>>> >>>
>>>>> >>> [1] http://markmail.org/message/74j4el6bpfpt4evs
>>>>> >>>
>>>>> >>> 2016-02-23 3:03 GMT+01:00 Xu, Cheng A <cheng.a.xu@intel.com>:
>>>>> >>>
>>>>> >>>> At this point, it has just Java interfaces only.
>>>>> >>>>
>>>>> >>>> -----Original Message-----
>>>>> >>>> From: Colin P. McCabe [mailto:cmccabe@apache.org]
>>>>> >>>> Sent: Tuesday, February 23, 2016 1:29 AM
>>>>> >>>> To: Hadoop Common
>>>>> >>>> Cc: Commons Developers List
>>>>> >>>> Subject: Re: [crypto][chimera] Next steps
>>>>> >>>>
>>>>> >>>> I would highly recommend shading this library when it
is used in
>>>>> >>>> Hadoop and/or Spark, to prevent version skew problems
between
>>>>> >>>> Hadoop and Spark like we have had in the past.
>>>>> >>>>
>>>>> >>>> What is the strategy for handling JNI components?  I
think at a
>>>>> >>>> minimum, we should include the version number in the
native
>>>library
>>>>> >>>> name to avoid problems when deploying multiple versions
of
>>>Chimera.
>>>>> >>>> This is something that has been problematic in Hadoop
with
>>>>> >>>> libhadoop.so.
>>>>> >>>>
>>>>> >>>> Is this library going to have Scala interfaces as well
as Java
>>>>> >>>> ones, or just Java?
>>>>> >>>>
>>>>> >>>> cheers,
>>>>> >>>> Colin
>>>>> >>>>
>>>>> >>>> On Sat, Feb 20, 2016 at 3:15 AM, Benedikt Ritter
>>>>> >>>> <britter@apache.org>
>>>>> >>>> wrote:
>>>>> >>>>> Hi,
>>>>> >>>>>
>>>>> >>>>> I'd like to discuss the next steps for moving the
Chimera
>>>>> >>>>>component to  Apache Commons. So far, none of the
other PMC
>>>members
>>>>> >>>>>has expressed his
>>>>> >>>> or
>>>>> >>>>> her thoughts about this. If nobody brings up objections
about
>>>>> >>>>>moving the  component to Apache Commons, I'm assuming
lazy
>>>>> >>>>>consensus about this.
>>>>> >>>>>
>>>>> >>>>> So the next steps would be:
>>>>> >>>>> - decide on a name for the new component (my proposal
was
>>>Apache
>>>>> >>>>>Commons
>>>>> >>>>> Crypto)
>>>>> >>>>> - move code to an Apache repo (probably git?!)
>>>>> >>>>> - request a Jira project
>>>>> >>>>> - setup maven build
>>>>> >>>>> - setup project website
>>>>> >>>>> - work on an initial release under Apache Commons
coordinates
>>>>> >>>>>
>>>>> >>>>> Anything missing?
>>>>> >>>>>
>>>>> >>>>> Regards,
>>>>> >>>>> Benedikt
>>>>> >>>>>
>>>>> >>>>> --
>>>>> >>>>> http://home.apache.org/~britter/
>>>>> >>>>> http://twitter.com/BenediktRitter
>>>>> >>>>> http://github.com/britter
>>>>> >>>>
>>>>> >>>> 
>>>-------------------------------------------------------------------
>>>>> >>>> -- To unsubscribe, e-mail:
>>>dev-unsubscribe@commons.apache.org
>>><mailto:dev-unsubscribe@commons.apache.org>
>>>>> >>>> For additional commands, e-mail:
>>>dev-help@commons.apache.org <mailto:dev-help@commons.apache.org>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>
>>>>> >>>
>>>>> >>> --
>>>>> >>> http://home.apache.org/~britter/
>>>>> >>> http://twitter.com/BenediktRitter
>>>>> >>> http://github.com/britter
>>>>> >>
>>>>> >> 
>>>---------------------------------------------------------------------
>>>>> >> To unsubscribe, e-mail:
>>>dev-unsubscribe@commons.apache.org
>>><mailto:dev-unsubscribe@commons.apache.org>
>>>>> >> For additional commands, e-mail:
>>>dev-help@commons.apache.org <mailto:dev-help@commons.apache.org>
>>>>> >>
>>>>> >
>>>>> >
>>>>> 
>>>>---------------------------------------------------------------------
>>>>> >To unsubscribe, e-mail:
>>>dev-unsubscribe@commons.apache.org
>>><mailto:dev-unsubscribe@commons.apache.org>
>>>>> >For additional commands, e-mail:
>>>dev-help@commons.apache.org <mailto:dev-help@commons.apache.org>
>>>>> >
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail:
>>>dev-unsubscribe@commons.apache.org
>>><mailto:dev-unsubscribe@commons.apache.org>
>>>>> For additional commands, e-mail:
>>>dev-help@commons.apache.org <mailto:dev-help@commons.apache.org>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail:
>>>dev-unsubscribe@commons.apache.org
>>><mailto:dev-unsubscribe@commons.apache.org>
>>>>> For additional commands, e-mail:
>>>dev-help@commons.apache.org <mailto:dev-help@commons.apache.org>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail:
>>>dev-unsubscribe@commons.apache.org
>>><mailto:dev-unsubscribe@commons.apache.org>
>>>>> For additional commands, e-mail:
>>>dev-help@commons.apache.org <mailto:dev-help@commons.apache.org>
>>>>>
>>>>>
>>>>
>>>>
>>>>--
>>>>http://home.apache.org/~britter/
>>>>http://twitter.com/BenediktRitter
>>>>http://github.com/britter
>>>
>>>
>>>---------------------------------------------------------------------
>>>To unsubscribe, e-mail:
>>>dev-unsubscribe@commons.apache.org
>>><mailto:dev-unsubscribe@commons.apache.org>
>>>For additional commands, e-mail:
>>>dev-help@commons.apache.org <mailto:dev-help@commons.apache.org>
>>>
>>>
>>
>>
>>-- 
>>E-Mail: garydgregory@gmail.com |
>>ggregory@apache.org
>>Java Persistence with Hibernate, Second Edition
>><http://www.manning.com/bauer3/>
>>JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
>>Spring Batch in Action <http://www.manning.com/templier/>
>>Blog: http://garygregory.wordpress.com
>>Home: http://garygregory.com/
>>Tweet! http://twitter.com/GaryGregory
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail:
>dev-unsubscribe@commons.apache.org
><mailto:dev-unsubscribe@commons.apache.org>
>For additional commands, e-mail:
>dev-help@commons.apache.org <mailto:dev-help@commons.apache.org>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Mime
View raw message