Need advice on executing multiple streaming jobs.
Problem:- We have 100's of streaming job. Every streaming job uses new port. Also, Spark automatically checks port from 4040 to 4056, post that it fails. One of the workaround, is to provide port explicitly.
Is there a way to tackle this situation? or Am I missing any thing?
Thanking you in advance.