flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Yao <g...@data-artisans.com>
Subject Re: Testing on Flink 1.5
Date Thu, 19 Apr 2018 15:47:39 GMT
Hi Amit,

Thank you for the follow up. What you describe sounds like a bug but I am
not
able to reproduce it. Can you open an issue in Jira with an outline of your
code
and how you submit the job?

> Could you also recommend us the best practice in FLIP6, should we use
YARN session or submit jobs in non-detached mode?

This should be independent of FLIP-6. We are aiming at backwards
compatibility
but apparently there are still things that require polishing.

Best,
Gary

On Thu, Apr 19, 2018 at 12:49 PM, Amit Jain <aj2011it@gmail.com> wrote:
>
> Hi Gary,
>
> We found the underlying issue with the following problem.
> Few of our jobs are stuck with logs [1], these jobs are only able to
allocate JM and couldn't get any TM, however, there are ample resource on
our cluster.
>
> We are running ETL merge job here. In this job, we first find new deltas
and if there is no delta detected then we make exit without actually
executing the job. I think this is the reason we see no TM allocation is
happening.
>
> I believe in above case (non-detached mode) we should mark the submitted
application as complete compare to running. Please share your thoughts on
this.
> Should I log this improvement in JIRA?
>
> Could you also recommend us the best practice in FLIP6, should we use
YARN session or submit jobs in non-detached mode?
>
> --
> Thanks,
> Amit
>

Mime
View raw message