spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Compton <compton.r...@gmail.com>
Subject Re: Bagel?
Date Wed, 14 Aug 2013 21:29:45 GMT
spark-0.7.3/bagel/target or spark-0.7.3/bagel/bagel/target ?

On Wed, Aug 7, 2013 at 9:17 PM, Matei Zaharia <matei.zaharia@gmail.com> wrote:
> Hi Ryan,
>
> To use Bagel, you can just add the spark-bagel.jar built within the bagel/target directory,
in addition to the spark-core-assembly. It doesn't have other dependencies beyond Spark Core.
>
> GraphX is an upcoming project to do richer types of graph processing, but it's fine to
use Bagel for now. We may eventually update the Bagel API to run on top of GraphX.
>
> Matei
>
> On Aug 7, 2013, at 8:54 PM, Ryan Compton <compton.ryan@gmail.com> wrote:
>
>> I'm interested in processing graphs with Spark. I'm able to do sbt/sbt
>> assembly and the process exits with success. I can find:
>>
>> -bash-3.2$ find . | grep jar$ | grep -assembly-
>> ./core/target/spark-core-assembly-0.7.3cdh3u3.jar
>> ./streaming/target/spark-streaming-assembly-0.7.3cdh3u3.jar
>>
>> I add these to my local Nexus repo, include them in my pom.xml, and
>> build/run distributed jobs just fine.
>>
>> Now, how do I code up iterative graph algorithms? "sbt/sbt assembly"
>> didn't make a Bagel assembly.
>>
>> Also, I can't find example code. The code here:
>> https://github.com/mesos/spark/wiki/Bagel-Programming-Guide has lots
>> of @serializable depreciation warnings and I can't find the ./run
>> script they are taking about at the bottom of this page:
>> http://spark-project.org/docs/latest/bagel-programming-guide.html
>>
>> I get the impression that Bagel may be discontinued. I also noticed
>> some ads for something called "GraphX" on the BDAS website. Should I
>> be using GraphX instead?
>>
>> On Wed, Aug 7, 2013 at 7:05 PM, Ryan Compton <compton.ryan@gmail.com> wrote:
>>> I'm interested in processing graphs with Spark. I'm able to do
>>> sbt/sbt assembly and the process exits with success. I can find:
>>>
>>> -bash-3.2$ find . | grep jar$ | grep -assembly-
>>> ./core/target/spark-core-assembly-0.7.3cdh3u3.jar
>>> ./streaming/target/spark-streaming-assembly-0.7.3cdh3u3.jar
>>>
>>> I add these to my local Nexus repo, include them in my pom.xml, and
>>> build/run distributed jobs just fine.
>>>
>>> Now, how do I code up iterative graph algorithms? "sbt/sbt assembly"
>>> didn't make a Bagel assembly.
>>>
>>> Also, I can't find example code. The code here:
>>> https://github.com/mesos/spark/wiki/Bagel-Programming-Guide has lots
>>> of @serializable depreciation warnings and I can't find the ./run
>>> script they are taking about at the bottom of this page:
>>> http://spark-project.org/docs/latest/bagel-programming-guide.html
>>>
>>> I get the impression that Bagel may be discontinued. Is there
>>> something else I can use for graphs?
>>>
>>> On Wed, Aug 7, 2013 at 7:02 PM, Ryan Compton <compton.ryan@gmail.com> wrote:
>>>> I'm interesting in processing graphs with Spark. I'm able to do
>>>> sbt/sbt assembly and the process exits with success. I can find:
>>>>
>>>> -bash-3.2$ find . | grep jar$ | grep -assembly-
>>>> ./core/target/spark-core-assembly-0.7.3cdh3u3.jar
>>>> ./streaming/target/spark-streaming-assembly-0.7.3cdh3u3.jar
>>>>
>>>> I add these to my local Nexus repo, include them in my pom.xml, and
>>>> build/run distributed jobs just fine.
>>>>
>>>> Now, how do I code up iterative graph algorithms? "sbt/sbt assembly"
>>>> didn't make a Bagel assembly.
>>>>
>>>> Also, I can't find example code. The code here:
>>>> https://github.com/mesos/spark/wiki/Bagel-Programming-Guide has lots
>>>> of @serializable depreciation warnings and I can't find the ./run
>>>> script they are taking about at the bottom of this page:
>>>> http://spark-project.org/docs/latest/bagel-programming-guide.html
>>>>
>>>> I get the impression that Bagel may be discontinued. Is there
>>>> something else I can use for graphs?
>

Mime
View raw message