spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Compton <compton.r...@gmail.com>
Subject Re: Bagel?
Date Thu, 08 Aug 2013 03:54:40 GMT
I'm interested in processing graphs with Spark. I'm able to do sbt/sbt
assembly and the process exits with success. I can find:

-bash-3.2$ find . | grep jar$ | grep -assembly-
./core/target/spark-core-assembly-0.7.3cdh3u3.jar
./streaming/target/spark-streaming-assembly-0.7.3cdh3u3.jar

I add these to my local Nexus repo, include them in my pom.xml, and
build/run distributed jobs just fine.

Now, how do I code up iterative graph algorithms? "sbt/sbt assembly"
didn't make a Bagel assembly.

Also, I can't find example code. The code here:
https://github.com/mesos/spark/wiki/Bagel-Programming-Guide has lots
of @serializable depreciation warnings and I can't find the ./run
script they are taking about at the bottom of this page:
http://spark-project.org/docs/latest/bagel-programming-guide.html

I get the impression that Bagel may be discontinued. I also noticed
some ads for something called "GraphX" on the BDAS website. Should I
be using GraphX instead?

On Wed, Aug 7, 2013 at 7:05 PM, Ryan Compton <compton.ryan@gmail.com> wrote:
> I'm interested in processing graphs with Spark. I'm able to do
> sbt/sbt assembly and the process exits with success. I can find:
>
> -bash-3.2$ find . | grep jar$ | grep -assembly-
> ./core/target/spark-core-assembly-0.7.3cdh3u3.jar
> ./streaming/target/spark-streaming-assembly-0.7.3cdh3u3.jar
>
> I add these to my local Nexus repo, include them in my pom.xml, and
> build/run distributed jobs just fine.
>
> Now, how do I code up iterative graph algorithms? "sbt/sbt assembly"
> didn't make a Bagel assembly.
>
> Also, I can't find example code. The code here:
> https://github.com/mesos/spark/wiki/Bagel-Programming-Guide has lots
> of @serializable depreciation warnings and I can't find the ./run
> script they are taking about at the bottom of this page:
> http://spark-project.org/docs/latest/bagel-programming-guide.html
>
> I get the impression that Bagel may be discontinued. Is there
> something else I can use for graphs?
>
> On Wed, Aug 7, 2013 at 7:02 PM, Ryan Compton <compton.ryan@gmail.com> wrote:
>> I'm interesting in processing graphs with Spark. I'm able to do
>> sbt/sbt assembly and the process exits with success. I can find:
>>
>> -bash-3.2$ find . | grep jar$ | grep -assembly-
>> ./core/target/spark-core-assembly-0.7.3cdh3u3.jar
>> ./streaming/target/spark-streaming-assembly-0.7.3cdh3u3.jar
>>
>> I add these to my local Nexus repo, include them in my pom.xml, and
>> build/run distributed jobs just fine.
>>
>> Now, how do I code up iterative graph algorithms? "sbt/sbt assembly"
>> didn't make a Bagel assembly.
>>
>> Also, I can't find example code. The code here:
>> https://github.com/mesos/spark/wiki/Bagel-Programming-Guide has lots
>> of @serializable depreciation warnings and I can't find the ./run
>> script they are taking about at the bottom of this page:
>> http://spark-project.org/docs/latest/bagel-programming-guide.html
>>
>> I get the impression that Bagel may be discontinued. Is there
>> something else I can use for graphs?

Mime
View raw message