spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mayur Rustagi <mayur.rust...@gmail.com>
Subject Re: sbt assembly fails
Date Tue, 18 Mar 2014 06:40:43 GMT
you need to assemble the code to get spark working (unless you are using
hadoop 1.0.4).

to run the code you can follow any of the standalone guides here:
https://spark.apache.org/docs/0.9.0/quick-start.html#a-standalone-app-in-scalayou
would still need sbt though.



Mayur Rustagi
Ph: +1 (760) 203 3257
http://www.sigmoidanalytics.com
@mayur_rustagi <https://twitter.com/mayur_rustagi>



On Tue, Mar 18, 2014 at 2:32 AM, Chengi Liu <chengi.liu.86@gmail.com> wrote:

> Yeah.. The http_proxy is set up.. and so is https_proxy..
> Basically, my maven projects, git pulls etc everything is working fine..
> except this.
>
> Here is another question which might help me to bypass this issue....
> If I create a jar using eclipse... how do i run that jar in code. Like in
> hadoop, I create a jar and then that jar like hadoop jar jar_name
> Earlier what i was trying to do was basically, write a code inside
> spark_examples directory.. do a new sbt build and create new jars.. and use
> run_examples script to run my code.. But since sbt assembly is having
> connection issue maybe someone can help me with how to build jars and
> deploy code on spark (not using spark shell)
> Thanks
>
>
> On Mon, Mar 17, 2014 at 11:27 PM, Mayur Rustagi <mayur.rustagi@gmail.com>wrote:
>
>> is it translating to sbt?
>> are you also setting command line proxy HTTP_PROXY
>> easiest is to build a small code & just test it out by building in
>> command line..
>>
>> Mayur Rustagi
>> Ph: +1 (760) 203 3257
>> http://www.sigmoidanalytics.com
>> @mayur_rustagi <https://twitter.com/mayur_rustagi>
>>
>>
>>
>> On Tue, Mar 18, 2014 at 2:15 AM, Chengi Liu <chengi.liu.86@gmail.com>wrote:
>>
>>> Hi Sean,
>>>   Yeah.. I am seeing erros across all repos and yepp.. this error is
>>> mainly because of connectivity issue...
>>> How do I set up proxy.. I did set up proxy as suggested by Mayur:
>>>
>>> export JAVA_OPTS="$JAVA_OPTS -Dhttp.proxyHost=yourserver
>>> -Dhttp.proxyPort=8080 -Dhttp.proxyUser=username
>>> -Dhttp.proxyPassword=password"
>>>
>>>
>>> How do I rectify this error. :(
>>>
>>>
>>>
>>> On Mon, Mar 17, 2014 at 6:07 PM, Sean Owen <sowen@cloudera.com> wrote:
>>>
>>>> It's in the main Maven repo:
>>>> http://central.maven.org/maven2/io/netty/netty-all/
>>>>
>>>> I assume you're seeing errors accessing all repos? the last few you
>>>> quote are not where they are intended to be, you're just seeing it
>>>> fail through all of them. I think it remains a connectivity problem
>>>> from your env to the repos, possibly because of a proxy?
>>>> --
>>>> Sean Owen | Director, Data Science | London
>>>>
>>>>
>>>> On Mon, Mar 17, 2014 at 8:39 PM, Chengi Liu <chengi.liu.86@gmail.com>
>>>> wrote:
>>>> > I have set it up.. still it fails.. Question:
>>>> >
>>>> https://oss.sonatype.org/content/repositories/snapshots/io/netty/netty-all/
>>>> >
>>>> > 4.0.13 is not there? Instead 4.0.18 is there?? Is this a bug?
>>>> >
>>>> >
>>>> > On Mon, Mar 17, 2014 at 11:01 AM, Mayur Rustagi <
>>>> mayur.rustagi@gmail.com>
>>>> > wrote:
>>>> >>
>>>> >>
>>>> >>
>>>> http://mail-archives.apache.org/mod_mbox/spark-user/201403.mbox/%3CCAAqHKj48jAPuZqc476Es67c+RRfime87uPRAMbdooFhCL0kdrA@mail.gmail.com%3E
>>>> >>
>>>> >> You also have to specify git proxy as code may be copied off git
>>>> also.
>>>> >>
>>>> >> Mayur Rustagi
>>>> >> Ph: +1 (760) 203 3257
>>>> >> http://www.sigmoidanalytics.com
>>>> >> @mayur_rustagi
>>>> >>
>>>> >>
>>>> >>
>>>> >> On Mon, Mar 17, 2014 at 1:25 PM, Chengi Liu <chengi.liu.86@gmail.com
>>>> >
>>>> >> wrote:
>>>> >>>
>>>> >>> Hi,
>>>> >>>   I am trying to compile the spark project using sbt/sbt assembly..
>>>> >>> And i see this error:
>>>> >>> [info] Resolving io.netty#netty-all;4.0.13.Final ...
>>>> >>> [error] Server access Error: Connection timed out
>>>> >>> url=
>>>> https://oss.sonatype.org/content/repositories/snapshots/io/netty/netty-all/4.0.13.Final/netty-all-4.0.13.Final.pom
>>>> >>> [error] Server access Error: Connection timed out
>>>> >>> url=
>>>> https://oss.sonatype.org/service/local/staging/deploy/maven2/io/netty/netty-all/4.0.13.Final/netty-all-4.0.13.Final.pom
>>>> >>> [error] Server access Error: Connection timed out
>>>> >>> url=
>>>> https://repository.cloudera.com/artifactory/cloudera-repos/io/netty/netty-all/4.0.13.Final/netty-all-4.0.13.Final.pom
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>> I followed the url and seemslike 4.0.13 is not present..Am i
missing
>>>> >>> something.. Also, I am behind proxy.. can that be an issue?
>>>> >>> How do i resolve this.
>>>> >>> Thanks
>>>> >>>
>>>> >>
>>>> >
>>>>
>>>
>>>
>>
>

Mime
View raw message