spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil Das <ak...@sigmoidanalytics.com>
Subject Re: Re: Pig 0.13, Spark, Spork
Date Wed, 09 Jul 2014 07:28:08 GMT
Hi Bertrand,

We've updated the document
http://docs.sigmoidanalytics.com/index.php/Setting_up_spork_with_spark_0.9.0

This is our working Github repo
https://github.com/sigmoidanalytics/spork/tree/spork-0.9

Feel free to open issues over here
https://github.com/sigmoidanalytics/spork/issues

Thanks
Best Regards


On Tue, Jul 8, 2014 at 2:33 PM, Bertrand Dechoux <dechouxb@gmail.com> wrote:

> @Mayur : I won't fight with the semantic of a fork but at the moment, no
> Spork does take the standard Pig as dependency. On that, we should agree.
>
> As for my use of Pig, I have no limitation. I am however interested to see
> the rise of a 'no-sql high level non programming language' for Spark.
>
> @Zhang : Could you elaborate your reference about Twitter?
>
>
> Bertrand Dechoux
>
>
> On Tue, Jul 8, 2014 at 4:04 AM, 张包峰 <pelickzhang@qq.com> wrote:
>
>> Hi guys, previously I checked out the old "spork" and updated it to
>> Hadoop 2.0, Scala 2.10.3 and Spark 0.9.1, see github project of mine
>> https://github.com/pelick/flare-spork‍
>>
>> It it also highly experimental, and just directly mapping pig physical
>> operations to spark RDD transformations/actions. It works for simple
>> requests. :)
>>
>> I am also interested on the progress of spork, is it undergoing in
>> Twitter in an un open-source way?
>>
>> ------------------
>> Thanks
>> Zhang Baofeng
>> Blog <http://blog.csdn.net/pelick> | Github <https://github.com/pelick>
>> | Weibo <http://weibo.com/pelickzhang> | LinkedIn
>> <http://www.linkedin.com/pub/zhang-baofeng/70/609/84>
>>
>>
>>
>>
>> ------------------ 原始邮件 ------------------
>> *发件人:* "Mayur Rustagi";<mayur.rustagi@gmail.com>;
>> *发送时间:* 2014年7月7日(星期一) 晚上11:55
>> *收件人:* "user@spark.apache.org"<user@spark.apache.org>;
>> *主题:* Re: Pig 0.13, Spark, Spork
>>
>> That version is old :).
>> We are not forking pig but cleanly separating out pig execution engine.
>> Let me know if you are willing to give it a go.
>>
>> Also would love to know what features of pig you are using ?
>>
>> Regards
>> Mayur
>>
>> Mayur Rustagi
>> Ph: +1 (760) 203 3257
>> http://www.sigmoidanalytics.com
>> @mayur_rustagi <https://twitter.com/mayur_rustagi>
>>
>>
>>
>> On Mon, Jul 7, 2014 at 8:46 PM, Bertrand Dechoux <dechouxb@gmail.com>
>> wrote:
>>
>>> I saw a wiki page from your company but with an old version of Spark.
>>>
>>> http://docs.sigmoidanalytics.com/index.php/Setting_up_spork_with_spark_0.8.1
>>>
>>> I have no reason to use it yet but I am interested in the state of the
>>> initiative.
>>> What's your point of view (personal and/or professional) about the Pig
>>> 0.13 release?
>>> Is the pluggable execution engine flexible enough in order to avoid
>>> having Spork as a fork of Pig? Pig + Spark + Fork = Spork :D
>>>
>>> As a (for now) external observer, I am glad to see competition in that
>>> space. It can only be good for the community in the end.
>>>
>>> Bertrand Dechoux
>>>
>>>
>>> On Mon, Jul 7, 2014 at 5:00 PM, Mayur Rustagi <mayur.rustagi@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>> We have fixed many major issues around Spork & deploying it with some
>>>> customers. Would be happy to provide a working version to you to try out.
>>>> We are looking for more folks to try it out & submit bugs.
>>>>
>>>> Regards
>>>> Mayur
>>>>
>>>> Mayur Rustagi
>>>> Ph: +1 (760) 203 3257
>>>> http://www.sigmoidanalytics.com
>>>> @mayur_rustagi <https://twitter.com/mayur_rustagi>
>>>>
>>>>
>>>>
>>>> On Mon, Jul 7, 2014 at 8:21 PM, Bertrand Dechoux <dechouxb@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I was wondering what was the state of the Pig+Spark initiative now
>>>>> that the execution engine of Pig is pluggable? Granted, it was done in
>>>>> order to use Tez but could it be used by Spark? I know about a
>>>>> 'theoretical' project called Spork but I don't know any stable and
>>>>> maintained version of it.
>>>>>
>>>>> Regards
>>>>>
>>>>> Bertrand Dechoux
>>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message