spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henry Saputra <henry.sapu...@gmail.com>
Subject Re: Licensing for PySpark's CloudPickle Module
Date Thu, 08 Aug 2013 01:30:20 GMT
So I guess we could close this discussion as not as issue?

This assuming that we could keep BSD license for CloudPickle module in
Spark as ASF project.

- Henry


On Sun, Jul 28, 2013 at 10:21 PM, Mattmann, Chris A (398J) <
chris.a.mattmann@jpl.nasa.gov> wrote:

> Hi Josh,
>
> BSD is a Category-A approved license at the ASF:
>
> http://www.apache.org/legal/3party.html#category-a
>
> Meaning it can be incorporated in Apache projects.
>
> HTH!
>
>
> Cheers,
> Chris
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW:  http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>
>
> -----Original Message-----
> From: Josh Rosen <rosenville@gmail.com>
> Reply-To: "dev@spark.incubator.apache.org" <dev@spark.incubator.apache.org
> >
> Date: Sunday, July 28, 2013 9:46 PM
> To: "dev@spark.incubator.apache.org" <dev@spark.incubator.apache.org>
> Subject: Licensing for PySpark's CloudPickle Module
>
> >PySpark's CloudPickle library was originally developed by PiCloud (
> >http://www.picloud.com/) and distributed under a non-BSD license.  I
> >contacted them last year and they agreed to let us bundle the CloudPickle
> >module under a BSD license.  Now that Spark is moving to an Apache
> >license,
> >how does this impact this module?  What license will apply to future
> >changes to this module?  Do we need to obtain additional licensing from
> >the
> >PiCloud folks?  I've attached my original correspondence with PiCloud, in
> >case that helps.
> >
> >I ask because I'm interested in making some fixes to the cloudpickle code
> >and I'd like to collaborate with the PiCloud folks, if possible, since
> >they're more familiar with that code and may be interested in some of the
> >bugs that I've found.
> >
> >Thanks,
> >Josh Rosen
> >
> >---------- Forwarded message ----------
> >From: Josh Rosen <joshrosen@berkeley.edu>
> >Date: Wed, Aug 15, 2012 at 11:47 PM
> >Subject: Re: Request to release the CloudPickler module as its own Python
> >package
> >To: Aaron Staley <aaron@picloud.com>
> >Cc: contact@picloud.com, Matei Zaharia <matei@eecs.berkeley.edu>
> >
> >
> >Hi Aaron,
> >
> >I'm just interested in cloudpickle.py, debugpickle.py, and their small
> >dependencies.  We'll develop our own module transfer / dependency
> >deployment system or build on existing systems in Spark or Mesos, so I
> >don't need to use other code from PiCloud.
> >
> >The CloudPickle module has been very useful and I appreciate your help
> >with
> >the licensing.  I'll bundle cloudpickle.py and its dependencies with
> >PySpark and add the proper attribution in the docstring.
> >
> >Thanks for your help,
> >Josh Rosen
> >
> >
> >On Aug 11, 2012, at 12:23 AM, Aaron Staley wrote:
> >
> >Hi Josh,
> >
> >How much of the functionality do you need to utilize?
> >
> >If we are just talking cloudpickle.py and debugpickle.py (and their small
> >dependencies; 2 functions the cloudpickler uses from util and the
> >xmlhandlers library used by pickledebug), we are fine with you moving that
> >into your own code and re-releasing it under the BSD license. Just modify
> >the license the license in the source code; all we ask is that you
> >attribute the original work to PiCloud, Inc. and provide a link to our
> >website in the top level comments of the modules.
> >
> >If you are looking for all of the functionality relating to getting code
> >running on X machine to Y machine (module transfer, some of the import
> >hacks in adapter, etc.), that's a whole different matter. It's difficult
> >to
> >pull it out of PiCloud itself as a separate package, due to it being
> >spread
> >across so many modules.  Are just the picklers enough?
> >
> >Thanks,
> >Aaron Staley
> >PiCloud, Inc.
> >
> >
> >On Thu, Aug 9, 2012 at 10:58 PM, Josh Rosen <joshrosen@berkeley.edu>
> >wrote:
> >
> >> Hello,
> >>
> >> My name is Josh Rosen.  I'm a grad student at UC Berkeley and I'm
> >>working
> >> on implementing a Python API for the Spark cluster computing system (
> >> http://www.spark-project.org/).
> >>
> >> Like PiCloud, my application needs to serialize Python functions in
> >>order
> >> to execute them across multiple machines.
> >>
> >> I'm currently using PiCloud's CloudPickle serializer code in my
> >>prototype (
> >> http://pydoc.net/cloud/2.3.9/cloud.serialization.cloudpickle).
> >>  Serializing arbitrary Python functions is non-trivial, but PiCloud's
> >> serializer is very robust and easy to use; I haven't written a function
> >> that it can't serialize.
> >>
> >> I'm interested in extending the CloudPickler module to work with PyPy (
> >> http://pypy.org/).  I am concerned that the inclusion of a modified
> >> CloudPickler with Spark would cause Spark to become a ³work based on the
> >> Library² and require Spark to become LGPL-licensed, in place of its
> >>current
> >> BSD license.
> >>
> >> Would you be interested in releasing the CloudPickler module and its
> >> dependencies as a BSD-licensed Python package (an LGPL-license would
> >>work
> >> too)?
> >>
> >> CloudPickler has much more functionality than other Python pickling /
> >> serialization libraries (
> >> http://pypi.python.org/pypi?%3Aaction=search&term=pickle&submit=search)
> >> and I hope to be able to use it in Spark.
> >>
> >> I would be very grateful if you are able to accommodate this request.
> >>
> >> Sincerely,
> >> Josh Rosen
> >
> >
> >
> >
> >--
> >Aaron Staley
> >*PiCloud, Inc.*
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message