add support for Operators for Generic target needed in MAGIA (again)

CHANGELOG.md

@@ -21,6 +21,7 @@ This file contains the changelog for the Deeploy project. The changelog is divid
 - Fix for python error when using python 3.12.11 [#189]( https://github.com/pulp-platform/Deeploy/pull/189)
 - Add support for Operators for Generic target needed in MAGIA [#193]( https://github.com/pulp-platform/Deeploy/pull/193)
 - Fix GAP9 L3 Board Tests: readfs Flash Ordering and Duplicate Input Data [#196](https://github.com/pulp-platform/Deeploy/pull/196)
+- Add support for Operators for Generic target needed in MAGIA (again) [#195]( https://github.com/pulp-platform/Deeploy/pull/195)
 
 ### Added
 - tests for Regular and DW Conv2D with 3x3 kernel
@@ -40,6 +41,7 @@ This file contains the changelog for the Deeploy project. The changelog is divid
 - Added GAP9 Platform Support: Deployer, Bindings, Templates, Tiler, DMA (L3Dma/MchanDma), target library, CI workflows
 - Per-layer microbenchmarking on PULPOpen via `--profileMicrobenchmark`: new `PULPMicrobenchmark` code-transformation pass + `perf_utils.h` helpers report cycles, instructions, stalls and cache misses per layer in `RunNetwork`
 - Add support for the Generic target for the following operators [Ceil](https://onnx.ai/onnx/operators/onnx__Ceil.html), [Floor](https://onnx.ai/onnx/operators/onnx__Floor.html), [Clip](https://onnx.ai/onnx/operators/onnx__Clip.html), [Sub](https://onnx.ai/onnx/operators/onnx__Sub.html), [Exp](https://onnx.ai/onnx/operators/onnx__Exp.html), [Sigmoid](https://onnx.ai/onnx/operators/onnx__Sigmoid.html), [Swish](https://onnx.ai/onnx/operators/onnx__Swish.html), [HardSigmoid](https://onnx.ai/onnx/operators/onnx__HardSigmoid.html), [HardSwish](https://onnx.ai/onnx/operators/onnx__HardSwish.html), [InstanceNormalization](https://onnx.ai/onnx/operators/onnx__InstanceNormalization.html), [GroupNormalization](https://onnx.ai/onnx/operators/onnx__GroupNormalization.html), [AveragePool](https://onnx.ai/onnx/operators/onnx__AveragePool.html), [GlobalAveragePool](https://onnx.ai/onnx/operators/onnx__GlobalAveragePool.html), [GlobalMaxPool](https://onnx.ai/onnx/operators/onnx__GlobalMaxPool.html).
+- Add support for the Generic target for the following operators: [Elu](https://onnx.ai/onnx/operators/onnx__Elu.html), [LeakyRelu](https://onnx.ai/onnx/operators/onnx__LeakyRelu.html), [Selu](https://onnx.ai/onnx/operators/onnx__Selu.html), [Scatter](https://onnx.ai/onnx/operators/onnx__Scatter.html), [ScatterElements](https://onnx.ai/onnx/operators/onnx__ScatterElements.html), [Col2Im](https://onnx.ai/onnx/operators/onnx__Col2Im.html), [Resize](https://onnx.ai/onnx/operators/onnx__Resize.html)
 
 ### Changed
 - Refactor the topology optimization pass `NeurekaReshapePointwiseConvolutionPass` and Neureka's Tile constraints
@@ -59,6 +61,7 @@ This file contains the changelog for the Deeploy project. The changelog is divid
 - Aligned CLI commands across the project
 - Added @runwangdl as a code owner
 - Skip emitting duplicate `testInputVector` data for inputs placed in L3 (loaded at runtime from the readfs hex instead), reducing test binary size
+- Allowing ONNX Operators with empty inputs.
 
 ### Fixed
 - Fix Neureka's output-channels subtile size (in ConvTemplate) and Dense/DW/PW tile constraints
@@ -77,6 +80,7 @@ This file contains the changelog for the Deeploy project. The changelog is divid
 - Reduce RunNetwork stack usage by scoping per-layer variables with braces and moving tileIdxPtr allocation into per-layer execution blocks
 - Fix invalid escape sequence python error in DeeployTypes.py: appearing when using pytest to launch regressions
 - Fix GAP9 board tests with `--defaultMemLevel L3` reading garbage inputs: place all gapy `--flash-property` options before the positional subcommand and use `image flash run` so the readfs partition (input hex files) is flashed to the device
+- Fix `ConvTranspose` layer: output buffer shape computation.
 
 ### Removed
 - removed experimental `enable3x3` flag, from Neureka Engine. Now, 3x3 mode is enabled by default.

Deeploy/CommonExtensions/TypeCheckers/SignPropTypeChecker.py

@@ -25,6 +25,9 @@ def typeInferGlobalCtxt(self, ctxt: NetworkContext, node: gs.Node) -> NetworkCon
         ctxt = super().typeInferGlobalCtxt(ctxt, node)
 
         for inputNode, _type in zip(node.inputs, self.input_types):
+            if not inputNode.name:
+                continue
+
             if isinstance(ctxt.lookup(inputNode.name), ConstantBuffer):
                 reference = ctxt.lookup(inputNode.name)
                 if not _type.referencedType.checkPromotion(reference.values):
@@ -39,8 +42,8 @@ def typeInferOutput(self, ctxt: NetworkContext, node: gs.Node,
                         operatorRepresentation: OperatorRepresentation) -> NetworkContext:
         ctxt = super().typeInferOutput(ctxt, node, operatorRepresentation)
 
-        inputs = [ctxt.lookup(inputNode.name) for inputNode in node.inputs]
-        outputs = [ctxt.lookup(outputNode.name) for outputNode in node.outputs]
+        inputs = [ctxt.lookup(inputNode.name) for inputNode in node.inputs if inputNode.name]
+        outputs = [ctxt.lookup(outputNode.name) for outputNode in node.outputs if outputNode.name]
 
         signProp = all([hasattr(_input, "_signed") and hasattr(_input, "nLevels") for _input in inputs])
 

Deeploy/DeeployTypes.py

@@ -1110,6 +1110,10 @@ def parseInputs(cls, ctxt: NetworkContext, node: gs.Node) -> NetworkContext:
         for inputNode in node.inputs:
             data_in = inputNode.name
 
+            # Skip absent optional inputs (ONNX represents them as empty-name Variables)
+            if not data_in:
+                continue
+
             # Hoist constant inputs
             if type(inputNode) == gs.ir.tensor.Constant and not ctxt.is_global(data_in):
                 ctxt.hoistConstant(inputNode)
@@ -1277,7 +1281,7 @@ def typeInferOutput(self, ctxt: NetworkContext, node: gs.Node,
         """
         newCtxt = ctxt.copy()
 
-        inputs = [ctxt.lookup(inputNode.name) for inputNode in node.inputs]
+        inputs = [ctxt.lookup(inputNode.name) for inputNode in node.inputs if inputNode.name]
         outputNames = [node.name for node in node.outputs]
 
         outputTypes = self.output_types
@@ -1306,6 +1310,9 @@ def typeCheckNodeInputs(self, ctxt: NetworkContext, node: gs.Node) -> bool:
         retCheck = True
 
         for inputNode, _type in zip(node.inputs, self.input_types):
+            if not inputNode.name:
+                continue
+
             reference = ctxt.lookup(inputNode.name)
 
             if not isinstance(reference, VariableBuffer):
@@ -1326,6 +1333,9 @@ def typeCheckNodeInputs(self, ctxt: NetworkContext, node: gs.Node) -> bool:
 
     def typeInferGlobalCtxt(self, ctxt: NetworkContext, node: gs.Node) -> NetworkContext:
         for inputNode, _type in zip(node.inputs, self.input_types):
+            if not inputNode.name:
+                continue
+
             if isinstance(ctxt.lookup(inputNode.name), ConstantBuffer):
                 reference = ctxt.lookup(inputNode.name)
                 if not _type.referencedType.checkPromotion(reference.values):
@@ -1348,7 +1358,7 @@ def annotateDict(self, ctxt: NetworkContext, node: gs.Node, operatorRepresentati
             The NodeParser's operatorRepresentation
 
         """
-        env = [node.name for node in node.inputs + node.outputs]
+        env = [node.name for node in node.inputs + node.outputs if node.name]
         for key, value in operatorRepresentation.items():
             # check if the referenced buffer is in the environment
             if isinstance(value, str) and value in env:
@@ -1903,7 +1913,9 @@ def broadcast(self, ctxt: NetworkContext, default_channels_first: bool = True) -
             broadcast to the target shape
 
         """
-        inputShapes = [ctxt.lookup(node.name).shape for node in self.node.inputs]
+        # Absent optional inputs are represented in ONNX as empty-name Variables; skip them.
+        validInputNodes = [node for node in self.node.inputs if node.name]
+        inputShapes = [ctxt.lookup(node.name).shape for node in validInputNodes]
         outputShapes = [ctxt.lookup(node.name).shape for node in self.node.outputs]
 
         if not "channels_first" in self.mapper.parser.operatorRepresentation:
@@ -1914,7 +1926,7 @@ def broadcast(self, ctxt: NetworkContext, default_channels_first: bool = True) -
         newInputShapes, newOutputShapes = self.computeShapes(inputShapes, outputShapes,
                                                              self.mapper.parser.operatorRepresentation, channels_first)
 
-        for node, newShape in zip(self.node.inputs + self.node.outputs, newInputShapes + newOutputShapes):
+        for node, newShape in zip(validInputNodes + self.node.outputs, newInputShapes + newOutputShapes, strict = True):
             if ctxt.is_local(node.name):
                 ctxt.localObjects[node.name].shape = newShape
                 # Update shape of tensors in onnx graph
@@ -2103,7 +2115,7 @@ def bind(self, ctxt: NetworkContext) -> Tuple[NetworkContext, bool]:
                     npType = self._broadcastToNpType(ctxt.localObjects[node.name]._type)
                     if npType is not None:
                         node.dtype = npType
-                elif ctxt.is_global(node.name):
+                elif ctxt.is_global(node.name) and hasattr(ctxt.globalObjects[node.name], '_type'):
                     npType = self._broadcastToNpType(ctxt.globalObjects[node.name]._type)
                     if isinstance(ctxt.globalObjects[node.name], ConstantBuffer):
                         if isinstance(node, gs.Constant):
@@ -2961,6 +2973,8 @@ def generateBufferInitializationCode(self) -> str:
         callStack = ''
         for node in ctxt.globalObjects.values():
             if isinstance(node, VariableBuffer) and not isinstance(node, StructBuffer):
+                if not hasattr(node, '_type'):
+                    continue
                 assert issubclass(node._type, Pointer), f"Global VariableBuffer {node.name} is not a Pointer!"
                 if node._deploy:
                     name = node.name
@@ -3006,6 +3020,8 @@ def generateBufferAllocationCode(self) -> str:
 
         for node in ctxt.globalObjects.values():
             if isinstance(node, VariableBuffer) and not isinstance(node, StructBuffer):
+                if not hasattr(node, '_type'):
+                    continue
                 assert issubclass(node._type, Pointer), f"Global VariableBuffer {node.name} is not a Pointer!"
                 if node._deploy:
                     name = node.name
@@ -3542,6 +3558,8 @@ def _printMemorySummary(self):
                 # We do not count structs for now, since they are not properly modeled
                 if isinstance(_buffer, ConstantBuffer) or (isinstance(_buffer, VariableBuffer) and _buffer._deploy):
                     # SCHEREMO: We only
+                    if not hasattr(_buffer, '_type'):
+                        continue
                     if (hasattr(_buffer, "_memoryLevel") and _buffer._memoryLevel == level) or level == "None":
                         staticSize += int((np.prod(_buffer.shape) * _buffer._type.referencedType.typeWidth // 8))
                     else:

Deeploy/Targets/Generic/Bindings.py

@@ -11,23 +11,24 @@
     int8_t, int32_t, uint8_t
 from Deeploy.DeeployTypes import CodeTransformation, NodeBinding
 from Deeploy.FutureExtension.CodeTransformationPasses.FutureCodeTransformation import FutureGeneration
-from Deeploy.Targets.Generic.Templates import AddTemplate, BatchNormalizationTemplate, ConcatTemplate, ConvTemplate, \
-    ConvTransposeTemplate, DebugPrintTemplate, DequantTemplate, DummyTemplate, DWConvTemplate, FloatAddTemplate, \
-    FloatAveragePoolTemplate, FloatCeilTemplate, FloatClipTemplate, FloatConvTemplate, FloatDivTemplate, \
-    FloatDWConvTemplate, FloatExpTemplate, FloatFloorTemplate, FloatGELUTemplate, FloatGemmTemplate, \
-    FloatGlobalAveragePoolTemplate, FloatGlobalMaxPoolTemplate, FloatGroupNormTemplate, FloatHardSigmoidTemplate, \
-    FloatHardSwishTemplate, FloatInstanceNormTemplate, FloatLayernormTemplate, FloatMatMulTemplate, \
-    FloatMaxPoolTemplate, FloatMulTemplate, FloatPadTemplate, FloatPowTemplate, FloatReduceMeanTemplate, \
-    FloatReluTemplate, FloatSigmoidTemplate, FloatSoftmaxTemplate, FloatSqrtTemplate, FloatSubTemplate, \
-    FloatSwishTemplate, GatherTemplate, GemmTemplate, IntegerDivTemplate, ITAMaxTemplate, ITAPartialMaxTemplate, \
-    MatMulTemplate, MaxPoolTemplate, MulTemplate, PadTemplate, QuantTemplate, ReduceMeanTemplate, ReduceSumTemplate, \
-    RequantShiftTemplate, ReshapeTemplate, RQIntegerDivTemplate, RQSiGELUTemplate, SliceTemplate, SubTemplate, \
+from Deeploy.Targets.Generic.Templates import AddTemplate, BatchNormalizationTemplate, Col2ImTemplate, ConcatTemplate, \
+    ConvTemplate, ConvTransposeTemplate, DebugPrintTemplate, DequantTemplate, DummyTemplate, DWConvTemplate, \
+    FloatAddTemplate, FloatAveragePoolTemplate, FloatCeilTemplate, FloatClipTemplate, FloatConvTemplate, \
+    FloatDivTemplate, FloatDWConvTemplate, FloatEluTemplate, FloatExpTemplate, FloatFloorTemplate, FloatGELUTemplate, \
+    FloatGemmTemplate, FloatGlobalAveragePoolTemplate, FloatGlobalMaxPoolTemplate, FloatGroupNormTemplate, \
+    FloatHardSigmoidTemplate, FloatHardSwishTemplate, FloatInstanceNormTemplate, FloatLayernormTemplate, \
+    FloatLeakyReluTemplate, FloatMatMulTemplate, FloatMaxPoolTemplate, FloatMulTemplate, FloatPadTemplate, \
+    FloatPowTemplate, FloatReduceMeanTemplate, FloatReluTemplate, FloatSeluTemplate, FloatSigmoidTemplate, \
+    FloatSoftmaxTemplate, FloatSqrtTemplate, FloatSubTemplate, FloatSwishTemplate, GatherTemplate, GemmTemplate, \
+    IntegerDivTemplate, ITAMaxTemplate, ITAPartialMaxTemplate, MatMulTemplate, MaxPoolTemplate, MulTemplate, \
+    PadTemplate, QuantTemplate, ReduceMeanTemplate, ReduceSumTemplate, RequantShiftTemplate, ReshapeTemplate, \
+    ResizeTemplate, RQIntegerDivTemplate, RQSiGELUTemplate, ScatterTemplate, SliceTemplate, SubTemplate, \
     TransposeTemplate, iGELUTemplate, iLayernormTemplate, iRMSNormTemplate, iSoftmaxTemplate
 from Deeploy.Targets.Generic.TypeCheckers import AddChecker, BatchNormChecker, ConcatChecker, ConvChecker, \
     DebugPrintChecker, DequantChecker, DivChecker, DummyChecker, GatherChecker, GELUChecker, GEMMChecker, \
-    LayerNormChecker, MatMulChecker, MaxPoolChecker, MulChecker, PadChecker, QuantChecker, ReduceMeanChecker, \
-    ReduceSumChecker, ReluChecker, RequantShiftChecker, ReshapeChecker, RQIntegerDivChecker, SliceChecker, \
-    SoftmaxChecker, TransposeChecker
+    LayerNormChecker, MatMulChecker, MaxPoolChecker, MulChecker, PadChecker, PassThroughTypeChecker, QuantChecker, \
+    ReduceMeanChecker, ReduceSumChecker, ReluChecker, RequantShiftChecker, ReshapeChecker, RQIntegerDivChecker, \
+    SliceChecker, SoftmaxChecker, TransposeChecker
 
 BasicTransformer = CodeTransformation([ArgumentStructGeneration(), MemoryManagementGeneration(), FutureGeneration()])
 
@@ -326,19 +327,35 @@
     for type in FloatDataTypes
 ]
 
-BasicConvTransposeBindings = [
+BasicConvTranspose1DBindings = [
+    NodeBinding(
+        ConvChecker(
+            [PointerClass(dtype), PointerClass(dtype), PointerClass(dtype)],  # input, weight, bias
+            [PointerClass(dtype)]),
+        ConvTransposeTemplate.referenceTemplate1D,
+        BasicTransformer) for dtype in FloatDataTypes
+] + [
+    NodeBinding(
+        ConvChecker(
+            [PointerClass(dtype), PointerClass(dtype)],  # input, weight
+            [PointerClass(dtype)]),
+        ConvTransposeTemplate.referenceTemplate1D,
+        BasicTransformer) for dtype in FloatDataTypes
+]
+
+BasicConvTranspose2DBindings = [
     NodeBinding(
         ConvChecker(
             [PointerClass(type), PointerClass(type), PointerClass(type)],  # input, weight, bias
             [PointerClass(type)]),
-        ConvTransposeTemplate.referenceTemplate,
+        ConvTransposeTemplate.referenceTemplate2D,
         BasicTransformer) for type in FloatDataTypes
 ] + [
     NodeBinding(
         ConvChecker(
             [PointerClass(type), PointerClass(type)],  # input, weight
             [PointerClass(type)]),
-        ConvTransposeTemplate.referenceTemplate,
+        ConvTransposeTemplate.referenceTemplate2D,
         BasicTransformer) for type in FloatDataTypes
 ]
 
@@ -385,6 +402,21 @@
                 FloatHardSwishTemplate.referenceTemplate, BasicTransformer),
 ]
 
+BasicEluBindings = [
+    NodeBinding(DummyChecker([PointerClass(float32_t)], [PointerClass(float32_t)]), FloatEluTemplate.referenceTemplate,
+                BasicTransformer),
+]
+
+BasicSeluBindings = [
+    NodeBinding(DummyChecker([PointerClass(float32_t)], [PointerClass(float32_t)]), FloatSeluTemplate.referenceTemplate,
+                BasicTransformer),
+]
+
+BasicLeakyReluBindings = [
+    NodeBinding(DummyChecker([PointerClass(float32_t)], [PointerClass(float32_t)]),
+                FloatLeakyReluTemplate.referenceTemplate, BasicTransformer),
+]
+
 BasicInstanceNormBindings = [
     NodeBinding(
         DummyChecker(
@@ -420,3 +452,22 @@
     NodeBinding(DummyChecker([PointerClass(float32_t)], [PointerClass(float32_t)]),
                 FloatGlobalMaxPoolTemplate.referenceTemplate, BasicTransformer)
 ]
+
+BasicCol2ImBindings = [
+    NodeBinding(
+        PassThroughTypeChecker([PointerClass(type), PointerClass(int32_t),
+                                PointerClass(int32_t)], [PointerClass(type)]), Col2ImTemplate.referenceTemplate,
+        BasicTransformer) for type in (int8_t, uint8_t, float32_t)
+]
+
+BasicScatterBindings = [
+    NodeBinding(
+        PassThroughTypeChecker(
+            [PointerClass(type), PointerClass(int32_t), PointerClass(type)], [PointerClass(type)]),
+        ScatterTemplate.referenceTemplate, BasicTransformer) for type in (int8_t, uint8_t, float32_t)
+]
+
+BasicResizeBindings = [
+    NodeBinding(PassThroughTypeChecker([PointerClass(type)], [PointerClass(type)]), ResizeTemplate.referenceTemplate,
+                BasicTransformer) for type in (int8_t, uint8_t, float32_t)
+]