[tmva][sofie] Add HardSigmoid operator for ONNX inference

[AdityaDRathore]

This Pull Request

This PR implements the HardSigmoid operator for the SOFIE (System for Optimized Fast Inference code Emit) engine. This is a standalone operator implementation compliant with the ONNX specification (Opset 6+).
The implementation is critical for supporting MobileNetV3 architectures (also for the HardSwish activation).

Changes or fixes:

• New Operator: Implemented TMVA::Experimental::SOFIE::ROperator_HardSigmoid.

  • Optimization: It uses std::fmax and std::fmin with specific hexfloat constants (0x0p+0f, 0x1p+0f) to ensure the compiler emits pure AVX MAXSS/MINSS instructions without intermediate double-precision conversions.
    It Enforces f-suffix on alpha and beta coefficients to prevent implicit promotion to double during inference.
    Updated RModelParser_ONNX to correctly parse alpha and beta attributes (defaults are set to 0.2, 0.5).

• Validation: Added a new unit test suite TestSofieHardSigmoid.

  • Verified numerical correctness against SciPy reference values (Golden Data).
  • Verified code generation structure to ensure AVX-friendly constants are emitted.

  Checklist:

• tested changes locally (TestSofieHardSigmoid passing)
• updated the docs (if necessary)

This PR fixes # (No specific issue linked, strictly feature addition)

cc: @guitargeek @moneta @devajithvs @bellenot

[sanjibansg]

Hi @AdityaDRathore,
Thanks for your contribution and developing the operator implementation for Hard-sigmoid. The implementation looks good to me. But, we are in the process of consolidating unary operations (like activation functions) into the ROperator_BasicUnary.hxx file. Can you make the corresponding changes?

Thanks.

[github-actions]

Test Results

    22 files      22 suites   3d 14h 55m 11s ⏱️
 3 814 tests  3 813 ✅ 0 💤 1 ❌
76 755 runs  76 754 ✅ 0 💤 1 ❌

For more details on these failures, see this check.

Results for commit 7963b08.

[AdityaDRathore]

Hi @AdityaDRathore, Thanks for your contribution and developing the operator implementation for Hard-sigmoid. The implementation looks good to me. But, we are in the process of consolidating unary operations (like activation functions) into the ROperator_BasicUnary.hxx file. Can you make the corresponding changes?

Thanks.

Hi @sanjibansg, thanks for the review!

I started refactoring to move this into ROperator_BasicUnary, but I noticed a potential architectural mismatch ROperator_BasicUnary appears to be designed specifically for stateless operators via the UnaryOpTraits pattern.

Since HardSigmoid is stateful (it must store alpha and beta attributes parsed from the ONNX node), it doesn't fit naturally into the current BasicUnary design without forcing all other simple operators (like Abs, Cos) to carry unused attribute overhead.

I also checked how other stateful unary operators are handled and saw that LeakyRelu (which also carries an alpha attribute) is implemented as a standalone operator.

Given this precedent, do you think it is safer to keep HardSigmoid separate to preserve the clean, stateless nature of BasicUnary? Or do you have a specific pattern in mind for handling attributes within the traits system?

[sanjibansg]

Hi @AdityaDRathore, Thanks for your contribution and developing the operator implementation for Hard-sigmoid. The implementation looks good to me. But, we are in the process of consolidating unary operations (like activation functions) into the ROperator_BasicUnary.hxx file. Can you make the corresponding changes?
Thanks.

Hi @sanjibansg, thanks for the review!

I started refactoring to move this into ROperator_BasicUnary, but I noticed a potential architectural mismatch ROperator_BasicUnary appears to be designed specifically for stateless operators via the UnaryOpTraits pattern.

Since HardSigmoid is stateful (it must store alpha and beta attributes parsed from the ONNX node), it doesn't fit naturally into the current BasicUnary design without forcing all other simple operators (like Abs, Cos) to carry unused attribute overhead.

I also checked how other stateful unary operators are handled and saw that LeakyRelu (which also carries an alpha attribute) is implemented as a standalone operator.

Given this precedent, do you think it is safer to keep HardSigmoid separate to preserve the clean, stateless nature of BasicUnary? Or do you have a specific pattern in mind for handling attributes within the traits system?

Ah yes! That is correct. Since this is a stateful operator, maybe better to keep this separately. Can you fix the formatting issues flagged in the CI? In the meantime, I will wait for the review of @lmoneta before we can merge this,

Thanks!

[AdityaDRathore]

Hi @sanjibansg
I have pushed the formatting fixes. The implementation is now clean and ready for the review.