flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Cannot see optimizer plan in web client
Date Mon, 03 Nov 2014 21:02:57 GMT
Hey!

here is what I see:

 - I do not get the first preview, which is expected (it it obtained
without the program arguments, so the program exits early)

 - I do not get the error stack trace that you describe when I submit the
program and supply two arguments to the program (as required), but I do not
get the plan preview either. Using the web developer tools in chrome, I see
that the produces JSON is malformed.

I will look into that tomorrow...

Stephan

On Mon, Nov 3, 2014 at 12:46 PM, Vasiliki Kalavri <vasilikikalavri@gmail.com
> wrote:

> I don't get any of the 2 previews :/
>
> On 3 November 2014 12:40, Stephan Ewen <sewen@apache.org> wrote:
>
>> Hey!
>>
>> Just to be sure I understand the problem correctly:
>>
>> Do you not get the initial preview (on the page where you upload the
>> jars), or do you also not get the execution plan preview (after you clicked
>> run), even if you supply arguments?
>>
>> Stephan
>>
>>
>>
>>
>> On Mon, Nov 3, 2014 at 12:29 PM, Vasiliki Kalavri <
>> vasilikikalavri@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> yes it's reproducible.
>>> I get the same behavior with the following dummy program:
>>>
>>> public class TestOptimizerPlan implements ProgramDescription {
>>>  @SuppressWarnings("serial")
>>> public static void main(String[] args) throws Exception {
>>> if (args.length < 2) {
>>> System.err.println("Usage: TestOptimizerPlan <input-file-path>
>>> <output-file-path>");
>>> return;
>>> }
>>>  ExecutionEnvironment env =
>>> ExecutionEnvironment.getExecutionEnvironment();
>>>  DataSet<Tuple2<Long, Long>> input = env.readCsvFile(args[0])
>>> .fieldDelimiter('\t').types(Long.class, Long.class);
>>>  DataSet<Tuple2<Long, Long>> result = input.map(
>>> new MapFunction<Tuple2<Long,Long>, Tuple2<Long,Long>>() {
>>> public Tuple2<Long, Long> map(Tuple2<Long, Long> value){
>>> return new Tuple2<Long, Long>(value.f0, value.f1+1);
>>> }
>>> });
>>> result.writeAsCsv(args[1], "\n", "\t");
>>> env.execute();
>>> }
>>> @Override
>>> public String getDescription() {
>>> return "TestOptimizerPlan <input-file-path> <output-file-path>";
>>> }
>>> }
>>>
>>> Cheers,
>>> Vasia.
>>>
>>> On 3 November 2014 09:23, Stephan Ewen <sewen@apache.org> wrote:
>>>
>>>> Hi!
>>>>
>>>> The web Frontend needs to "intercept" the execution plan when the call
>>>> to "execute()" comes, which is done with a special exception. The error
>>>> message says it failed, which is typically because that exception was
>>>> caught.
>>>>
>>>> However, in your code I see nothing that should prevent that plan
>>>> interception. Is the problem reproducible?
>>>>
>>>> Greetings,
>>>> Stephan
>>>>  Am 02.11.2014 16:36 schrieb "Vasiliki Kalavri" <
>>>> vasilikikalavri@gmail.com>:
>>>>
>>>> Hi all,
>>>>>
>>>>> when using the web client to upload and run a jar, the optimizer plan
>>>>> for my program does not appear.
>>>>> I have checked the "Show optimizer plan" box and provided args in the
>>>>> Arguments field.
>>>>> When I continue to submit the job, it runs correctly. There is no
>>>>> problem when using the plan visualizer tool.
>>>>>
>>>>> Checking the webclient logs, I see this exception in the .log file:
>>>>>
>>>>> org.apache.flink.client.program.ProgramInvocationException: The
>>>>> program plan could not be fetched. The program silently swallowed the
>>>>> control flow exceptions.
>>>>>         at
>>>>> org.apache.flink.client.program.PackagedProgram.getPreviewPlan(PackagedProgram.java:243)
>>>>>         at
>>>>> org.apache.flink.client.web.PactJobJSONServlet.doGet(PactJobJSONServlet.java:90)
>>>>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:734)
>>>>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
>>>>>         at
>>>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:532)
>>>>>         at
>>>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
>>>>>         at
>>>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
>>>>>         at
>>>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:965)
>>>>>         at
>>>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:388)
>>>>>         at
>>>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:187)
>>>>>         at
>>>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:901)
>>>>>         at
>>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>>>>>         at
>>>>> org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:47)
>>>>>         at
>>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:113)
>>>>>         at org.eclipse.jetty.server.Server.handle(Server.java:352)
>>>>>         at
>>>>> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:596)
>>>>>         at
>>>>> org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048)
>>>>>         at
>>>>> org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:549)
>>>>>         at
>>>>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:211)
>>>>>         at
>>>>> org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:425)
>>>>>         at
>>>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:489)
>>>>>         at
>>>>> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:436)
>>>>>         at java.lang.Thread.run(Thread.java:745)
>>>>>
>>>>> and the usage message in the .out file, as if I did not provide the
>>>>> correct number of arguments (which I did).
>>>>>
>>>>> If it helps, here's the program I was trying to visualize:
>>>>> https://github.com/vasia/incubator-flink/blob/semimetric-preprocessing/flink-examples/flink-java-examples/src/main/java/org/apache/flink/examples/java/preprocessing/Jaccard.java
>>>>>
>>>>> Anything I'm missing here?
>>>>>
>>>>> Thanks,
>>>>> V.
>>>>>
>>>>
>>>
>>
>

Mime
View raw message