flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-9703) Mesos does not expose TM Prometheus port
Date Thu, 12 Jul 2018 14:47:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-9703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541753#comment-16541753
] 

ASF GitHub Bot commented on FLINK-9703:
---------------------------------------

Github user zentol commented on a diff in the pull request:

    https://github.com/apache/flink/pull/6288#discussion_r202062001
  
    --- Diff: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/LaunchableMesosWorker.java
---
    @@ -332,6 +334,22 @@ public String toString() {
     		return taskInfo.build();
     	}
     
    +	/**
    +	 * Get port keys representing the TM's configured endpoints. This includes mandatory
TM endpoints such as
    +	 * data and rpc as well as optionally configured endpoints for services such as prometheus
reporter
    +	 *
    +	 * @return A deterministicly ordered Set of port keys to expose from the TM container
    +	 */
    +	private Set<String> getPortKeys() {
    +		LinkedHashSet<String> tmPortKeys = new LinkedHashSet<>(Arrays.asList(TM_PORT_KEYS));
    +		containerSpec.getDynamicConfiguration().keySet().stream()
    +			.filter(key -> key.endsWith(".port") || key.endsWith(".ports"))  // This matches
property naming convention
    --- End diff --
    
    You have to check the configured value as well, I know of at least one case where negative
ports are used to disable features (and we wouldn't want enable these again).
    Are there any use-cases where one might _not_ want mesos to provide the port?


> Mesos does not expose TM Prometheus port
> ----------------------------------------
>
>                 Key: FLINK-9703
>                 URL: https://issues.apache.org/jira/browse/FLINK-9703
>             Project: Flink
>          Issue Type: Bug
>          Components: Mesos
>            Reporter: Rune Skou Larsen
>            Priority: Major
>              Labels: pull-request-available
>
> LaunchableMesosWorker makes Mesos expose these ports for a Task Manager:
> {{private static final String[] TM_PORT_KEYS = {}}
> {{ "taskmanager.rpc.port",}}
> {{ "taskmanager.data.port"};}}
> But when running Prometheus Exporter on a TM, another port needs to be exposed to make Flink's Prometheos
endpoint externally scrapable by the Prometheus server. By default this is port 9249, but it
is configurable according to:
> [https://ci.apache.org/projects/flink/flink-docs-release-1.6/monitoring/metrics.html#prometheus-orgapacheflinkmetricsprometheusprometheusreporter]
>  
> My plan is to make a PR, that just adds another config option for mesos, to enable custom
ports to be exposed in the provisioned TMs.
> I considered carrying parts of the Metrics config into the Mesos code to automatically map metrics
ports in mesos. But making such a "shortcut" between Flink's metrics and mesos modules would probably
need some sort of integration testing, so I prefer the simple solution of just adding another
Mesos config option. But comments are welcome.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message