spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "shane knapp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-4021) Issues observed after upgrading Jenkins to JDK7u71
Date Tue, 21 Oct 2014 03:13:35 GMT

    [ https://issues.apache.org/jira/browse/SPARK-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177872#comment-14177872
] 

shane knapp commented on SPARK-4021:
------------------------------------

thing thing that seemed to trigger this failure was when i added a second JDK, via the jenkins
management interface (7u71).  after that, the JAVA_HOME env variable (in a jenkins job) disappeared
and the spark jobs started failing.  there were no job config changes, or anything related
to this on the systems themselves.  as well, i am absolutely, 100% certain that i did not
click/check/configure/whatever anything related to java 5.

these failures may be isolated to spark, as SparkR passed a build after i added that second
JDK and triggered it manually:
https://amplab.cs.berkeley.edu/jenkins/job/SparkR/74/

regarding the disappearing JAVA_HOME environment variables, compare:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21929/injectedEnvVars/
(passed, just before change) 

to:
https://amplab.cs.berkeley.edu/jenkins/job/SparkR/74/injectedEnvVars/ (passed, right after
change)
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21930/injectedEnvVars/
(failed, right after change)
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21941/injectedEnvVars/
(passed, after i removed 7u71)

i'll look in to this tomorrow and see if i can recreate.  it seems that somehow the JAVA_HOME
env got clobbered, and jenkins used some random java on the build agent.  to test this, i'll
reinstall 7u71, and create a PR that runs 'java -version' from run-tests-jenkins.  once the
build kicks off, i'll remove that JDK so that we don't mess anything else up (this is also
a good impetus to start scrounging some resources up for a staging instance so i don't have
to test this on prod).

if we're looking at an infrastructure problem, rather than code, i'll close this out.

> Issues observed after upgrading Jenkins to JDK7u71
> --------------------------------------------------
>
>                 Key: SPARK-4021
>                 URL: https://issues.apache.org/jira/browse/SPARK-4021
>             Project: Spark
>          Issue Type: Bug
>          Components: Project Infra
>         Environment: JDK 7u71
>            Reporter: Patrick Wendell
>
> The following compile failure was observed after adding JDK7u71 to Jenkins. However,
this is likely a misconfiguration from Jenkins rather than an issue with Spark (these errors
are specific to JDK5, in fact).
> {code}
> [error] ----------
> [error] 1. WARNING in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java
(at line 83)
> [error] 	private static final Logger logger = Logger.getLogger(JavaKinesisWordCountASL.class);
> [error] 	                            ^^^^^^
> [error] The field JavaKinesisWordCountASL.logger is never read locally
> [error] ----------
> [error] 2. WARNING in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java
(at line 151)
> [error] 	JavaDStream<String> words = unionStreams.flatMap(new FlatMapFunction<byte[],
String>() {
> [error] 	                                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> [error] The serializable class  does not declare a static final serialVersionUID field
of type long
> [error] ----------
> [error] 3. ERROR in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java
(at line 153)
> [error] 	public Iterable<String> call(byte[] line) {
> [error] 	                        ^^^^^^^^^^^^^^^^^
> [error] The method call(byte[]) of type new FlatMapFunction<byte[],String>(){}
must override a superclass method
> [error] ----------
> [error] 4. WARNING in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java
(at line 160)
> [error] 	new PairFunction<String, String, Integer>() {
> [error] 	    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> [error] The serializable class  does not declare a static final serialVersionUID field
of type long
> [error] ----------
> [error] 5. ERROR in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java
(at line 162)
> [error] 	public Tuple2<String, Integer> call(String s) {
> [error] 	                               ^^^^^^^^^^^^^^
> [error] The method call(String) of type new PairFunction<String,String,Integer>(){}
must override a superclass method
> [error] ----------
> [error] 6. WARNING in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java
(at line 165)
> [error] 	}).reduceByKey(new Function2<Integer, Integer, Integer>() {
> [error] 	                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> [error] The serializable class  does not declare a static final serialVersionUID field
of type long
> [error] ----------
> [error] 7. ERROR in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java
(at line 167)
> [error] 	public Integer call(Integer i1, Integer i2) {
> [error] 	               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> [error] The method call(Integer, Integer) of type new Function2<Integer,Integer,Integer>(){}
must override a superclass method
> [error] ----------
> [error] 7 problems (3 errors, 4 warnings)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message