1

Hello,

I am using cudnnActivationMode_t enum for sigmoid, relu, tanh activation functions as my C++ class member m_activationMode to set member m_activationDescriptor (which is used for subsequent cudnnActivationForward call for inferencing):

cudnnSetActivationDescriptor (m_activationDescriptor, m_activationMode, CUDNN_NOT_PROPAGATE_NAN, alpha));

Unfortunately, I see no CUDNN_ACTIVATION_SOFTPLUS kind under 9.5/include/cudnn_graph_v9.h for enum cudnnActivationMode_t.

I find softplus in above header, but it is in cudnnPointwiseMode_t and cudnnBackendAttributeName_t enums. The former enum is not used anywhere else in the library. There is hope since latter enum is used in cudnnBackendSetAttribute but not sure how to correctly use that function.

Could you please point to or provide an example for softplus activation?

Thank you

2

Hi @westwood ,
Cheking on this and will get back,.

Thanks

1 Like
3

From the Documentation:

cudnnActivationMode_t

This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.

The new Graph-API works with Pointwise Descriptors, that use PointwiseMode:

cudnn_graph Library — NVIDIA cuDNN

4

Hello,
I see no examples in your response. Could you please either point to an example or provide an example?
Thanks,

5

Here is an example with CuDnn-Frontend for Pointwise: cudnn-frontend/test/cpp/pointwise_tests.cpp at main · NVIDIA/cudnn-frontend · GitHub

You can extrapolate softplus from this.

Edit: Dokumentation for Pointwise: cudnn_graph Library — NVIDIA cuDNN