On 29. Jul 2017, at 00:00, Chris Hebert <firstname.lastname@example.org> wrote:I should also note that the above steps did get the Flink JobManager and TaskManagers to save their tmp web dashboard files to /my/tmp/ and to show in the Dashboard that the taskmanager.tmp.dirs property had been properly set to /my/tmp/, but the tmp files I wrote in my jobs stubbornly wrote to /tmp/ anyway.On Fri, Jul 28, 2017 at 4:55 PM, Chris Hebert <email@example.com> wrote:Hi,My jobs create tmp files like so:java.nio.file.Path tmpFilePath = java.nio.file.Files.
createTempFile("tmpFile", "txt");They currently appear in /tmp/, but I want them somewhere else, say /my/tmp/.The Flink on YARN docs say:Flink on YARN will overwrite the following configuration parameters jobmanager.rpc.address (because the JobManager is always allocated at different machines), taskmanager.tmp.dirs (we are using the tmp directories given by YARN) and parallelism.default if the number of slots has been specified.How would I specify a different tmp directory for a job without modifying my YARN tmp directories?I tried the taskmanager.tmp.dirs property inconf/flink-conf.yaml anyway, that failed.I appended -Djava.io.tmpdir=/my/ tmp/to JVM_ARGS and all three variations of DEFAULT_ENV_JAVA_OPTS in bin/config.sh, that failed.I passed -Djava.io.tmpdir=/my/ tmp/and variations as arguments to ./bin/yarn-session.sh and ./bin/flink run et cetera, that failed.Odd observation:The hadoop.tmp.dir property is set in my core-site.xml to /some/other/tmp/, yet Flink writes to /tmp/. My yarn-site.xml specifies no tmp.Side note:My Flink job is a Beam pipeline. I doubt that's relevant, but let me know if it is.Thanks,Chris