spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matei Zaharia <matei.zaha...@gmail.com>
Subject Re: Licensing for PySpark's CloudPickle Module
Date Thu, 08 Aug 2013 01:48:51 GMT
Yeah, I think we're fine with that. We don't have to switch its license.

Matei

On Aug 7, 2013, at 6:30 PM, Henry Saputra <henry.saputra@gmail.com> wrote:

> So I guess we could close this discussion as not as issue?
> 
> This assuming that we could keep BSD license for CloudPickle module in
> Spark as ASF project.
> 
> - Henry
> 
> 
> On Sun, Jul 28, 2013 at 10:21 PM, Mattmann, Chris A (398J) <
> chris.a.mattmann@jpl.nasa.gov> wrote:
> 
>> Hi Josh,
>> 
>> BSD is a Category-A approved license at the ASF:
>> 
>> http://www.apache.org/legal/3party.html#category-a
>> 
>> Meaning it can be incorporated in Apache projects.
>> 
>> HTH!
>> 
>> 
>> Cheers,
>> Chris
>> 
>> 
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW:  http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> 
>> 
>> 
>> 
>> 
>> 
>> -----Original Message-----
>> From: Josh Rosen <rosenville@gmail.com>
>> Reply-To: "dev@spark.incubator.apache.org" <dev@spark.incubator.apache.org
>>> 
>> Date: Sunday, July 28, 2013 9:46 PM
>> To: "dev@spark.incubator.apache.org" <dev@spark.incubator.apache.org>
>> Subject: Licensing for PySpark's CloudPickle Module
>> 
>>> PySpark's CloudPickle library was originally developed by PiCloud (
>>> http://www.picloud.com/) and distributed under a non-BSD license.  I
>>> contacted them last year and they agreed to let us bundle the CloudPickle
>>> module under a BSD license.  Now that Spark is moving to an Apache
>>> license,
>>> how does this impact this module?  What license will apply to future
>>> changes to this module?  Do we need to obtain additional licensing from
>>> the
>>> PiCloud folks?  I've attached my original correspondence with PiCloud, in
>>> case that helps.
>>> 
>>> I ask because I'm interested in making some fixes to the cloudpickle code
>>> and I'd like to collaborate with the PiCloud folks, if possible, since
>>> they're more familiar with that code and may be interested in some of the
>>> bugs that I've found.
>>> 
>>> Thanks,
>>> Josh Rosen
>>> 
>>> ---------- Forwarded message ----------
>>> From: Josh Rosen <joshrosen@berkeley.edu>
>>> Date: Wed, Aug 15, 2012 at 11:47 PM
>>> Subject: Re: Request to release the CloudPickler module as its own Python
>>> package
>>> To: Aaron Staley <aaron@picloud.com>
>>> Cc: contact@picloud.com, Matei Zaharia <matei@eecs.berkeley.edu>
>>> 
>>> 
>>> Hi Aaron,
>>> 
>>> I'm just interested in cloudpickle.py, debugpickle.py, and their small
>>> dependencies.  We'll develop our own module transfer / dependency
>>> deployment system or build on existing systems in Spark or Mesos, so I
>>> don't need to use other code from PiCloud.
>>> 
>>> The CloudPickle module has been very useful and I appreciate your help
>>> with
>>> the licensing.  I'll bundle cloudpickle.py and its dependencies with
>>> PySpark and add the proper attribution in the docstring.
>>> 
>>> Thanks for your help,
>>> Josh Rosen
>>> 
>>> 
>>> On Aug 11, 2012, at 12:23 AM, Aaron Staley wrote:
>>> 
>>> Hi Josh,
>>> 
>>> How much of the functionality do you need to utilize?
>>> 
>>> If we are just talking cloudpickle.py and debugpickle.py (and their small
>>> dependencies; 2 functions the cloudpickler uses from util and the
>>> xmlhandlers library used by pickledebug), we are fine with you moving that
>>> into your own code and re-releasing it under the BSD license. Just modify
>>> the license the license in the source code; all we ask is that you
>>> attribute the original work to PiCloud, Inc. and provide a link to our
>>> website in the top level comments of the modules.
>>> 
>>> If you are looking for all of the functionality relating to getting code
>>> running on X machine to Y machine (module transfer, some of the import
>>> hacks in adapter, etc.), that's a whole different matter. It's difficult
>>> to
>>> pull it out of PiCloud itself as a separate package, due to it being
>>> spread
>>> across so many modules.  Are just the picklers enough?
>>> 
>>> Thanks,
>>> Aaron Staley
>>> PiCloud, Inc.
>>> 
>>> 
>>> On Thu, Aug 9, 2012 at 10:58 PM, Josh Rosen <joshrosen@berkeley.edu>
>>> wrote:
>>> 
>>>> Hello,
>>>> 
>>>> My name is Josh Rosen.  I'm a grad student at UC Berkeley and I'm
>>>> working
>>>> on implementing a Python API for the Spark cluster computing system (
>>>> http://www.spark-project.org/).
>>>> 
>>>> Like PiCloud, my application needs to serialize Python functions in
>>>> order
>>>> to execute them across multiple machines.
>>>> 
>>>> I'm currently using PiCloud's CloudPickle serializer code in my
>>>> prototype (
>>>> http://pydoc.net/cloud/2.3.9/cloud.serialization.cloudpickle).
>>>> Serializing arbitrary Python functions is non-trivial, but PiCloud's
>>>> serializer is very robust and easy to use; I haven't written a function
>>>> that it can't serialize.
>>>> 
>>>> I'm interested in extending the CloudPickler module to work with PyPy (
>>>> http://pypy.org/).  I am concerned that the inclusion of a modified
>>>> CloudPickler with Spark would cause Spark to become a ³work based on the
>>>> Library² and require Spark to become LGPL-licensed, in place of its
>>>> current
>>>> BSD license.
>>>> 
>>>> Would you be interested in releasing the CloudPickler module and its
>>>> dependencies as a BSD-licensed Python package (an LGPL-license would
>>>> work
>>>> too)?
>>>> 
>>>> CloudPickler has much more functionality than other Python pickling /
>>>> serialization libraries (
>>>> http://pypi.python.org/pypi?%3Aaction=search&term=pickle&submit=search)
>>>> and I hope to be able to use it in Spark.
>>>> 
>>>> I would be very grateful if you are able to accommodate this request.
>>>> 
>>>> Sincerely,
>>>> Josh Rosen
>>> 
>>> 
>>> 
>>> 
>>> --
>>> Aaron Staley
>>> *PiCloud, Inc.*
>> 
>> 


Mime
View raw message