flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhu.qing (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FLINK-8526) When use some parallelism, the program will stuck in some setting.
Date Tue, 30 Jan 2018 11:45:00 GMT

     [ https://issues.apache.org/jira/browse/FLINK-8526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

zhu.qing updated FLINK-8526:
----------------------------
    Description: The next program attached will stuck at some special parallelism in some
situation. When parallelism is 80. The program will always stuck. And when parallelism is
100. everything goes well.  According to my research I found when the parallelism equals
to number of taskslots. The program is not fastest and probably caused network buffer not
enough. How networker buffer related to parallelism and  how parallelism relate to running
task (In other words we have 160 taskslots but running task can be far more than taskslots).
 (was: The next program will stuck at some special parallelism in some situation. In the code
attached. And I want to know the relation between parallelism and running task.When parallelism
is 80. The program will always stuck. And when parallelism is 100. everything goes well. And
I found when the parallelism equals to number of taskslots. The program is not fasted. And
probably caused network buffer not enough. How networker buffer related to parallelism)

> When use some parallelism, the program will stuck in some setting. 
> -------------------------------------------------------------------
>
>                 Key: FLINK-8526
>                 URL: https://issues.apache.org/jira/browse/FLINK-8526
>             Project: Flink
>          Issue Type: Bug
>          Components: Cluster Management, Java API, Local Runtime
>    Affects Versions: 1.4.0
>         Environment: 8 machines(96GB and 24 cores)  and 20 taskslot per taskmanager.
twitter-2010 dataset. And parallelism setting to 80. I run my code in standalone mode. 
>            Reporter: zhu.qing
>            Priority: Major
>         Attachments: T2AdjActiveV.java, T2AdjMessage.java
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> The next program attached will stuck at some special parallelism in some situation. When
parallelism is 80. The program will always stuck. And when parallelism is 100. everything
goes well.  According to my research I found when the parallelism equals to number of taskslots.
The program is not fastest and probably caused network buffer not enough. How networker buffer
related to parallelism and  how parallelism relate to running task (In other words we have
160 taskslots but running task can be far more than taskslots).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message