mxnet-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anirudh Subramanian (JIRA)" <>
Subject [jira] [Created] (MXNET-1096) Improve customer experience with CUDNN Auto Tuning
Date Mon, 15 Oct 2018 19:44:00 GMT
Anirudh Subramanian created MXNET-1096:

             Summary: Improve customer experience with CUDNN Auto Tuning
                 Key: MXNET-1096
             Project: Apache MXNet
          Issue Type: Improvement
          Components: Apache MXNet Backend
            Reporter: Anirudh Subramanian

Look into improvement of Customer Experience for CUDNN_AUTOTUNE_DEFAULT and how it can be
improved for use cases where it may become a bottleneck. 

Suggestion from Marco: "One possible mitigation for cases like this where the parameter distribution
is non-uniform(varied input shape, output shape, weight shape combination), we could have
a strategy that doesn't start CuDNN auto-tuning on the first time but only if that combination
has been received multiple times. The optimization might then happen in the background. If
the background task finishes, the chosen implementation is then atomically switched. This
would give us the benefit of low latency for every single request as well as even further
reduced latency in the long run as soon as the background optimization finished. We might
just have to check back with Nvidia whether there's a CuDNN autotuning strategy that allows
to run in the background with reduced resource ultilization (to avoid running out of memory
or bottlenecking the main thread). "

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message