spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Hamstra <m...@clearstorydata.com>
Subject Re: Build Spark with maven
Date Mon, 09 Dec 2013 02:52:45 GMT
Yeah, don't do that.  The akka packages you need are not in Maven Central,
so you won't be able to find them by looking only in a mirror.  If you
temporarily rename your settings.xml file, build Spark 0.8.1, then put your
settings back, you should be okay as long as you don't remove the akka
packages from your ~/.m2/repository



On Sun, Dec 8, 2013 at 6:45 PM, Azuryy Yu <azuryyyu@gmail.com> wrote:

> @Mark
>
> I configured under apache-maven/conf/settings.xml:
>
>   <mirrors>
>     <!-- mirror
>      | Specifies a repository mirror site to use instead of a given
> repository. The repository that
>      | this mirror serves has an ID that matches the mirrorOf element of
> this mirror. IDs are used
>      | for inheritance and direct lookup purposes, and must be unique
> across the set of mirrors.
>      |
>     -->
>     <mirror>
>       <id>maven-nexus</id>
>       <mirrorOf>external:*</mirrorOf>
>       <name>Official Maven Repo</name>
>       <url>http://repo1.maven.org/maven2/</url>
>     </mirror>
>   </mirrors>
>
>
>
> On Mon, Dec 9, 2013 at 10:41 AM, Azuryy Yu <azuryyyu@gmail.com> wrote:
>
>> @Mater, what's your maven mirror used in your setting.xml, can you share
>> with me? Thanks.
>>
>>
>>
>> On Mon, Dec 9, 2013 at 10:14 AM, Azuryy Yu <azuryyyu@gmail.com> wrote:
>>
>>> Hi Mark,
>>>
>>> I build the current releast candidate,
>>> It complained during build:
>>> Downloading:
>>> http://repo1.maven.org/maven2/com/typesafe/akka/akka-actor/2.0.5/akka-actor-2.0.5.pom
>>> [WARNING] The POM for com.typesafe.akka:akka-actor:jar:2.0.5 is missing,
>>> no dependency information available
>>> Downloading:
>>> http://repo1.maven.org/maven2/com/typesafe/akka/akka-remote/2.0.5/akka-remote-2.0.5.pom
>>> [WARNING] The POM for com.typesafe.akka:akka-remote:jar:2.0.5 is
>>> missing, no dependency information available
>>> Downloading:
>>> http://repo1.maven.org/maven2/com/typesafe/akka/akka-slf4j/2.0.5/akka-slf4j-2.0.5.pom
>>> [WARNING] The POM for com.typesafe.akka:akka-slf4j:jar:2.0.5 is missing,
>>> no dependency information available
>>> Downloading:
>>> http://repo1.maven.org/maven2/com/typesafe/akka/akka-actor/2.0.5/akka-actor-2.0.5.jar
>>> Downloading:
>>> http://repo1.maven.org/maven2/com/typesafe/akka/akka-remote/2.0.5/akka-remote-2.0.5.jar
>>> Downloading:
>>> http://repo1.maven.org/maven2/com/typesafe/akka/akka-slf4j/2.0.5/akka-slf4j-2.0.5.jar
>>>
>>>
>>>
>>> then, throw Error:
>>> [ERROR] Failed to execute goal on project spark-core_2.9.3: Could not
>>> resolve dependencies for project
>>> org.apache.spark:spark-core_2.9.3:jar:0.8.1-incubating: The following
>>> artifacts could not be resolved: com.typesafe.akka:akka-actor:jar:2.0.5,
>>> com.typesafe.akka:akka-remote:jar:2.0.5,
>>> com.typesafe.akka:akka-slf4j:jar:2.0.5: Could not find artifact
>>> com.typesafe.akka:akka-actor:jar:2.0.5 in maven-nexus (
>>> http://repo1.maven.org/maven2/) -> [Help 1]
>>>
>>>
>>>
>>> I checked on the http://repo1.maven.org/maven2, there is no akka-*
>>>  2.0.5, and there is no URL such as '
>>> http://repo1.maven.org/maven2/com/typesafe/akka/akka-actor/2.0.5', it's
>>> http://repo1.maven.org/maven2/com/typesafe/akka/akka-actor_'VERSION'/<http://repo1.maven.org/maven2/com/typesafe/akka/akka-actor_2.10/>
>>>
>>>
>>>
>>> On Mon, Dec 9, 2013 at 9:54 AM, Azuryy Yu <azuryyyu@gmail.com> wrote:
>>>
>>>> Thanks Mark, I will.
>>>>
>>>>
>>>> On Mon, Dec 9, 2013 at 9:53 AM, Mark Hamstra <mark@clearstorydata.com>wrote:
>>>>
>>>>> You probably want to try the current release candidate:
>>>>> https://github.com/apache/incubator-spark/archive/v0.8.1-incubating.tar.gz
>>>>>
>>>>>
>>>>> On Sun, Dec 8, 2013 at 5:51 PM, Azuryy Yu <azuryyyu@gmail.com>
wrote:
>>>>>
>>>>>> Thanks Matei, I'll try.
>>>>>>
>>>>>> @Mark, I download source package from
>>>>>> http://spark-project.org/download/spark-0.8.0-incubating.tgz, Sorry,
>>>>>> I build 0.8.0. not 0.8.1.
>>>>>>
>>>>>>
>>>>>> On Mon, Dec 9, 2013 at 9:31 AM, Mark Hamstra <mark@clearstorydata.com
>>>>>> > wrote:
>>>>>>
>>>>>>> There is no released source package of Spark 0.8.1.  It's just
gone
>>>>>>> into release candidate in the past day.  Is that what you are
trying to
>>>>>>> build?  It should be exactly the same as what I just checked
out from the
>>>>>>> v0.8.1-incubating tag.
>>>>>>>
>>>>>>>
>>>>>>> On Sun, Dec 8, 2013 at 5:28 PM, Azuryy Yu <azuryyyu@gmail.com>wrote:
>>>>>>>
>>>>>>>> I am not check out from repository, I download source package
and
>>>>>>>> build.
>>>>>>>>  On 2013-12-09 9:22 AM, "Mark Hamstra" <mark@clearstorydata.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I don't believe that is true of the Spark 0.8.1 code.
 I just got
>>>>>>>>> done building Spark from the v0.8.1-incubating tag after
first removing
>>>>>>>>> anything to do with akka from my ~/.m2/repository.  After
a successful
>>>>>>>>> build without incident, my local repo now only contains
akka 2.0.5 packages
>>>>>>>>> within the com/typesafe/akka subtree.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sun, Dec 8, 2013 at 5:01 PM, Azuryy Yu <azuryyyu@gmail.com>wrote:
>>>>>>>>>
>>>>>>>>>> I build 0.8.1, maven try to download akka-actor-2.0.1,
which is
>>>>>>>>>> used by scala-core-io.
>>>>>>>>>>  On 2013-12-09 8:40 AM, "Matei Zaharia" <matei.zaharia@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Which version of Spark are you building? AFAIK
it should be
>>>>>>>>>>> using Akka 2.0.5, not 2.0.1.
>>>>>>>>>>>
>>>>>>>>>>> Matei
>>>>>>>>>>>
>>>>>>>>>>> On Dec 8, 2013, at 3:35 PM, Azuryy Yu <azuryyyu@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> any thoughs here? I still cannot compile spark
using maven,
>>>>>>>>>>> thanks for any inputs.
>>>>>>>>>>>  On 2013-12-07 2:31 PM, "Azuryy Yu" <azuryyyu@gmail.com>
wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hey dears,
>>>>>>>>>>>>
>>>>>>>>>>>> Can you give me a maven repo, so I can compile
Spark with Maven.
>>>>>>>>>>>>
>>>>>>>>>>>> I'm using http://repo1.maven.org/maven2/
currently
>>>>>>>>>>>>
>>>>>>>>>>>> but It complains cannot find akka-actor-2.0.1,
 I searched on
>>>>>>>>>>>> the repo1.maven, and I am also cannot find
akka-actor-2.0.1, which is too
>>>>>>>>>>>> old.
>>>>>>>>>>>>
>>>>>>>>>>>> another strange output I can see:
>>>>>>>>>>>> <scala.version>2.9.3</scala.version>
 in the pom,
>>>>>>>>>>>> but Maven download scala-2.9.2 during compile,
why is that?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message