I'd like to specify the total sum of cores / memory as command line arguments with spark-submit. That is, I'd like to set yarn.nodemanager.resource.memory-mb and the yarn.nodemanager.resource.cpu-vcores parameters as described in this blog post.
when submitting through the command line, what is the correct way to do it? Is it:
or something else? I tried these, and I tried looking in the ResourceManager UI to see if they were set, but couldn't find them.