flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [flink] XComp opened a new pull request #15853: [FLINK-22566][test] Adds NodeManager log extraction to YARN-related e2e tests
Date Fri, 07 May 2021 15:07:29 GMT

XComp opened a new pull request #15853:
URL: https://github.com/apache/flink/pull/15853


   ## What is the purpose of the change
   
   In [FLINK-22566](https://issues.apache.org/jira/browse/FLINK-22566) we ran an issue where
the TaskManager didn't come up immediately after spawning the YARN containers. There was no
way to investigate the gap between the container transitioning into `RUNNING` state and the
`TaskManager` starting to generate output.
   
   The suspicion is that there's some infrastructure issue that is hidden. We're missing the
NodeManager logs to investigate that. They are located on the worker nodes. This PR adds the
extraction of logs located on the worker nodes.
   
   ## Brief change log
   
   * Moves log extraction and printing into its own functions
   * calls these functions for the `master` and the `worker*` nodes
   
   ## Verifying this change
   
   * ✅ Functions were tested locally
   * ❌ we have to wait for some test runs to fail (like the [loop AzureCI build](https://dev.azure.com/mapohl/flink/_build/results?buildId=414&view=results))
to see whether the extraction works as expected.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): no
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
     - The serializers: no
     - The runtime per-record code paths (performance sensitive): no
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing,
Kubernetes/Yarn/Mesos, ZooKeeper: no
     - The S3 file system connector: no
   
   ## Documentation
   
     - Does this pull request introduce a new feature? no
     - If yes, how is the feature documented? not applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message