spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tony Reix <tony.r...@bull.net>
Subject RE:Results of tests
Date Fri, 09 Jan 2015 09:15:45 GMT
Hi Ted

Thanks for the info.
However, I'm still unable to understand how the page:
   https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/testReport/
has been built.
This page contains details I do not find in the page you indicated to me:
   https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/consoleFull

As an example, I'm still unable to find these details:
org.apache.spark<https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/testReport/org.apache.spark/>
      12 mn   0
        1
        247
        248

org.apache.spark.api.python<https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/testReport/org.apache.spark.api.python/>
20 ms   0
        0
        2
        2

org.apache.spark.bagel<https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/testReport/org.apache.spark.bagel/>
  7.7 s   0
        0
        4
        4

org.apache.spark.broadcast<https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/testReport/org.apache.spark.broadcast/>
  43 s    0
        0
        17
        17

org.apache.spark.deploy<https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/testReport/org.apache.spark.deploy/>
16 s    0
        0
        29
        29

org.apache.spark.deploy.worker<https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/testReport/org.apache.spark.deploy.worker/>
  0.55 s  0
        0
        12
        12

........


Moreover, in my Ubuntu/x86_64 environment, I do not find 3745 tests and 0 failures, but 3485
tests and 4 failures (when using Oracle JVM 1.7 ). When using IBM JVM, there are only 2566
tests and 5 failures (in same component: Streaming).

On my PPC64BE (BE = Big-Endian)environment, the tests block after 2 hundreds of tests.
Is Spark independent of Little/Big-Endian stuff ?

On my PPC64LE (LE = Little-Endian) environment, I have 3485 tests only (like on Ubuntu/x86_64
with IBM JVM), with 6 or 285 failures...

So, I need to learn more about how your Jenkins environment extracts details about the results.
Moreover, which JVM is used ?

Do you plan to use IBM JVM in order to check that Spark and IBM JVM are compatible ? (they
already do not look to be compatible 100% ...).

Thanks

Tony

IBM Coop Architect & Technical Leader
Office : +33 (0) 4 76 29 72 67
1 rue de Provence - 38432 Échirolles - France
www.atos.net<http://www.atos.net/>
________________________________
De : Ted Yu [yuzhihong@gmail.com]
Envoyé : jeudi 8 janvier 2015 17:43
À : Tony Reix
Cc : dev@spark.apache.org
Objet : Re: Results of tests

Here it is:

[centos] $ /home/jenkins/tools/hudson.tasks.Maven_MavenInstallation/Maven_3.0.5/bin/mvn -DHADOOP_PROFILE=hadoop-2.4
-Dlabel=centos -DskipTests -Phadoop-2.4 -Pyarn -Phive clean package


You can find the above in https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/consoleFull


Cheers

On Thu, Jan 8, 2015 at 8:05 AM, Tony Reix <tony.reix@bull.net<mailto:tony.reix@bull.net>>
wrote:
Thanks !

I've been able to see that there are 3745 tests for version 1.2.0 with profile Hadoop 2.4
 .
However, on my side, the maximum tests I've seen are 3485... About 300 tests are missing on
my side.
Which Maven option has been used for producing the report file used for building the page:
     https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/testReport/
  ? (I'm not authorized to look at the "configuration" part)

Thx !

Tony

________________________________
De : Ted Yu [yuzhihong@gmail.com<mailto:yuzhihong@gmail.com>]
Envoyé : jeudi 8 janvier 2015 16:11
À : Tony Reix
Cc : dev@spark.apache.org<mailto:dev@spark.apache.org>
Objet : Re: Results of tests

Please take a look at https://amplab.cs.berkeley.edu/jenkins/view/Spark/

On Thu, Jan 8, 2015 at 5:40 AM, Tony Reix <tony.reix@bull.net<mailto:tony.reix@bull.net>>
wrote:
Hi,
I'm checking that Spark works fine on a new environment (PPC64 hardware).
I've found some issues, with versions 1.1.0, 1.1.1, and 1.2.0, even when running on Ubuntu
on x86_64 with Oracle JVM. I'd like to know where I can find the results of the tests of Spark,
for each version and for the different versions, in order to have a reference to compare my
results with. I cannot find them on Spark web-site.
Thx
Tony




Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message