[ https://issues.apache.org/jira/browse/SPARK-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177872#comment-14177872
]
shane knapp commented on SPARK-4021:
------------------------------------
thing thing that seemed to trigger this failure was when i added a second JDK, via the jenkins
management interface (7u71). after that, the JAVA_HOME env variable (in a jenkins job) disappeared
and the spark jobs started failing. there were no job config changes, or anything related
to this on the systems themselves. as well, i am absolutely, 100% certain that i did not
click/check/configure/whatever anything related to java 5.
these failures may be isolated to spark, as SparkR passed a build after i added that second
JDK and triggered it manually:
https://amplab.cs.berkeley.edu/jenkins/job/SparkR/74/
regarding the disappearing JAVA_HOME environment variables, compare:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21929/injectedEnvVars/
(passed, just before change)
to:
https://amplab.cs.berkeley.edu/jenkins/job/SparkR/74/injectedEnvVars/ (passed, right after
change)
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21930/injectedEnvVars/
(failed, right after change)
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21941/injectedEnvVars/
(passed, after i removed 7u71)
i'll look in to this tomorrow and see if i can recreate. it seems that somehow the JAVA_HOME
env got clobbered, and jenkins used some random java on the build agent. to test this, i'll
reinstall 7u71, and create a PR that runs 'java -version' from run-tests-jenkins. once the
build kicks off, i'll remove that JDK so that we don't mess anything else up (this is also
a good impetus to start scrounging some resources up for a staging instance so i don't have
to test this on prod).
if we're looking at an infrastructure problem, rather than code, i'll close this out.
> Issues observed after upgrading Jenkins to JDK7u71
> --------------------------------------------------
>
> Key: SPARK-4021
> URL: https://issues.apache.org/jira/browse/SPARK-4021
> Project: Spark
> Issue Type: Bug
> Components: Project Infra
> Environment: JDK 7u71
> Reporter: Patrick Wendell
>
> The following compile failure was observed after adding JDK7u71 to Jenkins. However,
this is likely a misconfiguration from Jenkins rather than an issue with Spark (these errors
are specific to JDK5, in fact).
> {code}
> [error] ----------
> [error] 1. WARNING in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java
(at line 83)
> [error] private static final Logger logger = Logger.getLogger(JavaKinesisWordCountASL.class);
> [error] ^^^^^^
> [error] The field JavaKinesisWordCountASL.logger is never read locally
> [error] ----------
> [error] 2. WARNING in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java
(at line 151)
> [error] JavaDStream<String> words = unionStreams.flatMap(new FlatMapFunction<byte[],
String>() {
> [error] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> [error] The serializable class does not declare a static final serialVersionUID field
of type long
> [error] ----------
> [error] 3. ERROR in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java
(at line 153)
> [error] public Iterable<String> call(byte[] line) {
> [error] ^^^^^^^^^^^^^^^^^
> [error] The method call(byte[]) of type new FlatMapFunction<byte[],String>(){}
must override a superclass method
> [error] ----------
> [error] 4. WARNING in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java
(at line 160)
> [error] new PairFunction<String, String, Integer>() {
> [error] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> [error] The serializable class does not declare a static final serialVersionUID field
of type long
> [error] ----------
> [error] 5. ERROR in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java
(at line 162)
> [error] public Tuple2<String, Integer> call(String s) {
> [error] ^^^^^^^^^^^^^^
> [error] The method call(String) of type new PairFunction<String,String,Integer>(){}
must override a superclass method
> [error] ----------
> [error] 6. WARNING in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java
(at line 165)
> [error] }).reduceByKey(new Function2<Integer, Integer, Integer>() {
> [error] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> [error] The serializable class does not declare a static final serialVersionUID field
of type long
> [error] ----------
> [error] 7. ERROR in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java
(at line 167)
> [error] public Integer call(Integer i1, Integer i2) {
> [error] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> [error] The method call(Integer, Integer) of type new Function2<Integer,Integer,Integer>(){}
must override a superclass method
> [error] ----------
> [error] 7 problems (3 errors, 4 warnings)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org
|