nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nava <chrisnav...@gmail.com>
Subject Setting thread counts based on worker node size
Date Mon, 25 Jan 2021 17:05:27 GMT
Good morning from Denver!

I'm having a hard time trying to find the answer within the documentation
online. I am fairly new to NiFi and was trying to research the possibility
of adjusting the maximum thread count of a process based on the
environment and the node size.  We have the following code and I FEEL like
doing some kind of request to see how many nodes are on the given process
and then replacing the thread_count with some equation based on the return
is the way to go. Am I on the right track or way off-base here?:

def update_thread():
    """ Updates the overall thread count for NiFi """
    if (os.getenv('NIFI_THREAD_OVERRIDE') == 'true'):
        thread_count = int(os.getenv('NIFI_THREAD_COUNT'))
    else:
        thread_count = int(round(multiprocessing.cpu_count()*0.8*4))
    api = nipyapi.nifi.apis.controller_api.ControllerApi()
    controller = api.get_controller_config()
    controller.component =
nipyapi.nifi.models.controller_configuration_dto.ControllerConfigurationDTO(thread_count)
    api.update_controller_config(controller)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message