flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sunjincheng (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FLINK-11971) Fix `Command: start_kubernetes_if_not_ruunning failed` error
Date Wed, 20 Mar 2019 06:20:00 GMT

     [ https://issues.apache.org/jira/browse/FLINK-11971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

sunjincheng updated FLINK-11971:
--------------------------------
    Description: 
When I did the end-to-end test under Mac OS, I found the following two problems:
 1. The verification returned for different `minikube status` is not enough for the robustness.
The strings returned by different versions of different platforms are different. the following
misjudgment is caused:
 When the `Command: start_kubernetes_if_not_ruunning failed` error occurs, the `minikube`
has actually started successfully. The core reason is that there is a bug in the `test_kubernetes_embedded_job.sh`
script.  The error message as follows:

!image-2019-03-20-14-02-29-636.png!

  !image-2019-03-20-14-04-17-933.png!

 
{code:java}
Current check logic: echo ${status} | grep -q "minikube: Running cluster: Running kubectl:
Correctly Configured"

==== My local messae====
jinchengsunjcs-iMac:flink-1.8.0 jincheng$ minikube status

host: Running

kubelet: Running

apiserver: Running

kubectl: Correctly Configured: pointing to minikube-vm at 192.168.99.101{code}
So, I think we should improve the check logic of `minikube status`, What do you think?

  was:
Since the difference between 1.8.0 and 1.7.x is that 1.8.x does not put the `hadoop-shaded`
JAR integrated into the dist.  It will cause an error when the end-to-end test cannot be
found with `Hadoop` Related classes,  such as: `java.lang.NoClassDefFoundError: Lorg/apache/hadoop/fs/FileSystem`.
So we need to improve the end-to-end test script, or explicitly stated in the README, i.e.
end-to-end test need to add `flink-shaded-hadoop2-uber-XXXX.jar` to the classpath. So, we
will get the exception something like:
{code:java}
[INFO] 3 instance(s) of taskexecutor are already running on jinchengsunjcs-iMac.local.

Starting taskexecutor daemon on host jinchengsunjcs-iMac.local.

java.lang.NoClassDefFoundError: Lorg/apache/hadoop/fs/FileSystem;

at java.lang.Class.getDeclaredFields0(Native Method)

at java.lang.Class.privateGetDeclaredFields(Class.java:2583)

at java.lang.Class.getDeclaredFields(Class.java:1916)

at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:72)

at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.clean(StreamExecutionEnvironment.java:1558)

at org.apache.flink.streaming.api.datastream.DataStream.clean(DataStream.java:185)

at org.apache.flink.streaming.api.datastream.DataStream.addSink(DataStream.java:1227)

at org.apache.flink.streaming.tests.BucketingSinkTestProgram.main(BucketingSinkTestProgram.java:80)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)

at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)

at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:423)

at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:813)

at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:287)

at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)

at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)

at org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)

at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)

at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)

Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.FileSystem

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

... 22 more

Job () is running.{code}
So, I think we can import the test script or import the README.

What do you think?


> Fix `Command: start_kubernetes_if_not_ruunning failed` error
> ------------------------------------------------------------
>
>                 Key: FLINK-11971
>                 URL: https://issues.apache.org/jira/browse/FLINK-11971
>             Project: Flink
>          Issue Type: Bug
>          Components: Tests
>    Affects Versions: 1.8.0, 1.9.0
>            Reporter: sunjincheng
>            Priority: Major
>
> When I did the end-to-end test under Mac OS, I found the following two problems:
>  1. The verification returned for different `minikube status` is not enough for the robustness.
The strings returned by different versions of different platforms are different. the following
misjudgment is caused:
>  When the `Command: start_kubernetes_if_not_ruunning failed` error occurs, the `minikube`
has actually started successfully. The core reason is that there is a bug in the `test_kubernetes_embedded_job.sh`
script.  The error message as follows:
> !image-2019-03-20-14-02-29-636.png!
>   !image-2019-03-20-14-04-17-933.png!
>  
> {code:java}
> Current check logic: echo ${status} | grep -q "minikube: Running cluster: Running
kubectl: Correctly Configured"
> ==== My local messae====
> jinchengsunjcs-iMac:flink-1.8.0 jincheng$ minikube status
> host: Running
> kubelet: Running
> apiserver: Running
> kubectl: Correctly Configured: pointing to minikube-vm at 192.168.99.101{code}
> So, I think we should improve the check logic of `minikube status`, What do you think?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message