Fix Neureka

CHANGELOG.md

@@ -5,6 +5,7 @@ This file contains the changelog for the Deeploy project. The changelog is divid
 
 
 ### List of Pull Requests
+- Fix Neureka [#188](https://github.com/pulp-platform/Deeploy/pull/188)
 - XDNA2 Platform Support [#179](https://github.com/pulp-platform/Deeploy/pull/179)
 - Add Microbenchmarking Infrastructure and CI Using GVSoC CSR [#162](https://github.com/pulp-platform/Deeploy/pull/162)
 - Fix CI Cache Generation [#176](https://github.com/pulp-platform/Deeploy/pull/176)
@@ -21,6 +22,8 @@ This file contains the changelog for the Deeploy project. The changelog is divid
 - Fix GAP9 L3 Board Tests: readfs Flash Ordering and Duplicate Input Data [#196](https://github.com/pulp-platform/Deeploy/pull/196)
 
 ### Added
+- tests for Regular and DW Conv2D with 3x3 kernel
+- Neureka's engine-aware DW lowering pass `NeurekaNCHWtoNHWCDwConvPass`
 - XDNA2 (AIE2p) platform beta: first MLIR backend for Deeploy, targeting AMD/Xilinx NPU2 with a single BF16 Add kernel
 - `MLIRNodeTemplate` and `MLIRCodeTransformation` base classes for MLIR-emitting backends
 - Auto-tiling with L1 memory constraints for XDNA2
@@ -38,6 +41,7 @@ This file contains the changelog for the Deeploy project. The changelog is divid
 - Add support for the Generic target for the following operators [Ceil](https://onnx.ai/onnx/operators/onnx__Ceil.html), [Floor](https://onnx.ai/onnx/operators/onnx__Floor.html), [Clip](https://onnx.ai/onnx/operators/onnx__Clip.html), [Sub](https://onnx.ai/onnx/operators/onnx__Sub.html), [Exp](https://onnx.ai/onnx/operators/onnx__Exp.html), [Sigmoid](https://onnx.ai/onnx/operators/onnx__Sigmoid.html), [Swish](https://onnx.ai/onnx/operators/onnx__Swish.html), [HardSigmoid](https://onnx.ai/onnx/operators/onnx__HardSigmoid.html), [HardSwish](https://onnx.ai/onnx/operators/onnx__HardSwish.html), [InstanceNormalization](https://onnx.ai/onnx/operators/onnx__InstanceNormalization.html), [GroupNormalization](https://onnx.ai/onnx/operators/onnx__GroupNormalization.html), [AveragePool](https://onnx.ai/onnx/operators/onnx__AveragePool.html), [GlobalAveragePool](https://onnx.ai/onnx/operators/onnx__GlobalAveragePool.html), [GlobalMaxPool](https://onnx.ai/onnx/operators/onnx__GlobalMaxPool.html).
 
 ### Changed
+- Refactor the topology optimization pass `NeurekaReshapePointwiseConvolutionPass` and Neureka's Tile constraints
 - `aie.dialects` API: move `link_with` from `aie_d.core()` to `aie_d.external_func()` (mlir-aie v1.3.2)
 - Decouple XDNA requirements (`requirements-xdna.txt`) from base dev requirements
 - Make `aie` import optional to not enforce mlir-aie package installation for non-XDNA users
@@ -56,6 +60,9 @@ This file contains the changelog for the Deeploy project. The changelog is divid
 - Skip emitting duplicate `testInputVector` data for inputs placed in L3 (loaded at runtime from the readfs hex instead), reducing test binary size
 
 ### Fixed
+- Fix Neureka's output-channels subtile size (in ConvTemplate) and Dense/DW/PW tile constraints
+- in `NetworkContainer._createIOBindings`, set `_live = True` on network input and output buffers so that any buffer aliasing a network I/O tensor is no longer deallocated while the I/O tensor is still in use.
+- Fix latent bug in `VariableBuffer.has_live_aliases` where `visited` variable was storing buffer names as a set of characters instead of strings.
 - Add missing `shell: bash` directive to CI cache generation steps to ensure correct shell execution
 - Fix wrong test case in GAP9 ccache workflow (`test_gap9_tiled_kernels_l2_singlebuffer` using `MatMul/Regular` instead of `Add/Large`)
 - Fix Docker flow to fetch `*.so` git lfs files
@@ -69,6 +76,7 @@ This file contains the changelog for the Deeploy project. The changelog is divid
 - Fix GAP9 board tests with `--defaultMemLevel L3` reading garbage inputs: place all gapy `--flash-property` options before the positional subcommand and use `image flash run` so the readfs partition (input hex files) is flashed to the device
 
 ### Removed
+- removed experimental `enable3x3` flag, from Neureka Engine. Now, 3x3 mode is enabled by default.
 - `testDMA.py` was an old test; we now have `test_dmas.py` instead.
 
 ## Release v0.2.1 (2026-02-05) [#158](https://github.com/pulp-platform/Deeploy/pull/158)

Deeploy/CommonExtensions/OptimizationPasses/TopologyOptimizationPasses/LoweringOptimizationPasses.py

@@ -106,7 +106,7 @@ def _prependSqueezeDims(tensor: gs.Tensor, name: str, axis: Union[int, Sequence[
 
 # Permute (0,1,2,3,...,N-2,N-1) -> (0,1,2,3,...,N-1,N-2)
 def _swapLastTwoDimsPermutation(N: int) -> List[int]:
-    assert N >= 2, "N needs to be larger then 2"
+    assert N >= 2, "N needs to be larger than 2"
     return [*range(N - 2), N - 1, N - 2]
 
 
@@ -393,12 +393,18 @@ def _requantized_gemm_to_pw_fun(graph: gs.Graph, match: Match, name: str):
     if not isinstance(matrixB, gs.Constant):
         return graph
 
-    assert len(matrixA.shape) in [
-        2, 3
-    ], f"Unsupported number of dimensions for input matrix A of GEMM operation: {len(matrixA.shape)}; shape: {matrixA.shape}"
-    assert len(matrixY.shape) in [
-        2, 3
-    ], f"Unsupported number of dimensions for output matrix of GEMM operation: {len(matrixY.shape)}; shape: {matrixY.shape}"
+    assert len(matrixA.shape) in (2, 3), f"Unsupported GEMM's input matrix A with shape {matrixA.shape}"
+    assert len(matrixY.shape) in (2, 3), f"Unsupported GEMM's output matrix with shape {matrixY.shape}"
+
+    # The pointwise conv this node is about to be lowered into only supports
+    # per-tensor or per-output-channel requantization via its RQS unit. If
+    # mul/add don't fit that (e.g. per-row requantization, where M becomes a
+    # spatial dimension of the convolution rather than a channel), bail out
+    # here and leave the RequantizedGemm for another lowering pass to handle.
+    add, mul = node.inputs[2], node.inputs[3]
+    out_channels = matrixY.shape[-1]
+    if int(np.prod(mul.shape)) not in (1, out_channels) or int(np.prod(add.shape)) not in (1, out_channels):
+        return graph
 
     # Pointwise with HWC layout (channels_first == False)
 
@@ -408,13 +414,15 @@ def _requantized_gemm_to_pw_fun(graph: gs.Graph, match: Match, name: str):
     node.attrs['alpha'] = node.attrs.get('alpha', 1.0)
     node.attrs['beta'] = node.attrs.get('beta', 1.0)
 
-    # If transA is set then the matrix is of shape [B x K x M] and it needs to be transposed, otherwise its shape is  [B x M x K]
+    # If transA is set then the matrix is of shape [B x K x M] and it needs to
+    # be transposed, otherwise its shape is  [B x M x K]
     if node.attrs['transA'] == 1:
         perm = _swapLastTwoDimsPermutation(len(matrixA.shape))
         graph.nodes.append(_appendTranspose(matrixA, node, perm))
         matrixA = node.inputs[0]
 
-    # If transB is set then the matrix is of shape [N x K] and it doesn't need to be transposed, otherwise its shape is [K x N] and it has to be transposed
+    # If transB is set then the matrix is of shape [N x K] and it doesn't need
+    # to be transposed, otherwise its shape is [K x N] and it has to be transposed
     if node.attrs['transB'] == 0:
         perm = _swapLastTwoDimsPermutation(len(matrixB.shape))
         matrixB.values = matrixB.values.transpose(perm)
@@ -444,6 +452,7 @@ def _requantized_gemm_to_pw_fun(graph: gs.Graph, match: Match, name: str):
     matrixYSqueezeDimsNode, pwOut = _prependSqueezeDims(matrixY, name, squeezeDims)
     graph.nodes.append(matrixYSqueezeDimsNode)
 
+    n_levels = node.attrs['n_levels_out'] if 'n_levels_out' in node.attrs else node.attrs['n_levels']
     pwAttrs = {
         'channels_first': False,
         'dilations': [1, 1],
@@ -452,14 +461,11 @@ def _requantized_gemm_to_pw_fun(graph: gs.Graph, match: Match, name: str):
         'pads': [0, 0, 0, 0],
         'strides': [1, 1],
         'div': node.attrs['div'],
-        'n_levels_out': node.attrs['n_levels_out'],
+        'n_levels_out': n_levels,
         'shift': node.attrs['shift'],
         'signed': node.attrs['signed'],
     }
 
-    add = node.inputs[2]
-    mul = node.inputs[3]
-
     _inputs = [pwIn, pwWeight, mul, add]
 
     pw = gs.Node(op = 'RequantizedConv',

Deeploy/DeeployTypes.py

@@ -339,7 +339,7 @@ def has_live_aliases(self, ctxt: NetworkContext) -> bool:
         # Do a breadth-first search across the aliasing double-linked list
         live = self._live
         queue = set(self.aliases)
-        visited = set(self.name)
+        visited = {self.name}
         while len(queue) > 0:
             next = queue.pop()
             buffNext = ctxt.lookup(next)
@@ -2499,6 +2499,12 @@ def _createIOBindings(self, ctxt: NetworkContext, graph: gs.Graph):
             data_type = self.inputTypes[node.name]
             nb = ctxt.VariableBuffer(data_name, data_size)
             nb.is_input = True
+            # Global network I/O buffers are externally allocated and live for
+            # the entire inference. Marking them live ensures has_live_aliases
+            # protects them: a buffer that aliases a network input/output (e.g.
+            # a no-op Reshape view) must never be deallocated, or the free would
+            # lead to overwrite the still-needed I/O memory.
+            nb._live = True
 
             ctxt.add(nb, 'global')
             ctxt.annotateType(data_name, data_type)
@@ -2509,6 +2515,7 @@ def _createIOBindings(self, ctxt: NetworkContext, graph: gs.Graph):
             # WIESEP: The shape and type will be parsed from the graph
             nb = ctxt.VariableBuffer(data_name, data_size)
             nb.is_output = True
+            nb._live = True
             ctxt.add(nb, 'global')
 
         return ctxt

Deeploy/Targets/Neureka/Deployer.py

@@ -8,10 +8,10 @@
 
 from Deeploy.AbstractDataTypes import Pointer
 from Deeploy.CommonExtensions.OptimizationPasses.TopologyOptimizationPasses.LoweringOptimizationPasses import \
-    NCHWtoNHWCPass, PULPNCHWtoNHWCPass
+    PULPNCHWtoNHWCPass
 from Deeploy.DeeployTypes import DeploymentPlatform, TopologyOptimizer
 from Deeploy.Targets.Neureka.TopologyOptimizationPasses.Passes import ConvEngineDiscolorationPass, \
-    NeurekaOptimizationPass
+    NeurekaNCHWtoNHWCPass, NeurekaOptimizationPass
 from Deeploy.Targets.PULPOpen.Deployer import PULPDeployer
 
 
@@ -30,10 +30,10 @@ def __init__(self,
         super().__init__(graph, deploymentPlatform, inputTypes, loweringOptimizer, scheduler, name,
                          default_channels_first, deeployStateDir, inputOffsets)
 
-        if self.Platform.engines[0].enable3x3:
-            for idx in range(len(self.loweringOptimizer.passes)):
-                if isinstance(self.loweringOptimizer.passes[idx], PULPNCHWtoNHWCPass):
-                    self.loweringOptimizer.passes[idx] = NCHWtoNHWCPass(self.default_channels_first)
+        engine_name = self.Platform.engines[0].name
+        for idx in range(len(self.loweringOptimizer.passes)):
+            if isinstance(self.loweringOptimizer.passes[idx], PULPNCHWtoNHWCPass):
+                self.loweringOptimizer.passes[idx] = NeurekaNCHWtoNHWCPass(self.default_channels_first, engine_name)
 
         self.loweringOptimizer.passes += [
             ConvEngineDiscolorationPass(),

Deeploy/Targets/Neureka/Engine.py

@@ -31,11 +31,14 @@
         ConvLayer([NeurekaPWConv2DMapper, NeurekaDWConv2DMapper, NeurekaDenseConv2DMapper]),
 }
 
-_includeList = ["pulp_nnx_neureka.h", "pulp_nnx_util.h", "neureka_siracusa_bsp.h", "neureka.h", "neureka_task.h"]
+_includeList = [
+    "pulp_nnx_neureka.h", "pulp_nnx_util.h", "neureka_siracusa_bsp.h", "neureka.h", "neureka_task.h", "neureka_gvsoc.h"
+]
 
 _neurekaInitCode = r"""
 neureka_siracusa_conf_t conf = {.max_stall = 8};
 neureka_nnx_init(neureka_siracusa_get_dev(), &conf);
+// neureka_gvsoc_log_activate(neureka_siracusa_get_dev(), NEUREKA_GVSOC_LOG_LEVEL_ALL, NEUREKA_GVSOC_LOG_FORMAT_HEXADECIMAL);
 """
 
 
@@ -46,11 +49,9 @@ def __init__(self,
                  Mapping = NeurekaMapping,
                  initCode: str = _neurekaInitCode,
                  includeList: List[str] = _includeList,
-                 enable3x3: bool = False,
                  enableStrides: bool = False) -> None:
         super().__init__(name, Mapping, initCode, includeList)
 
-        self.enable3x3 = enable3x3
         self.enableStrides = enableStrides
 
     def isDenseConv(self, node) -> bool:
@@ -77,7 +78,4 @@ def isDWConv(self, node) -> bool:
             (node.attrs['strides'] == [1, 1] or self.enableStrides)
 
     def canExecute(self, node: gs.Node) -> bool:
-        if self.enable3x3:
-            return self.isPWConv(node) or self.isDWConv(node) or self.isDenseConv(node)
-        else:
-            return self.isPWConv(node)
+        return self.isPWConv(node) or self.isDWConv(node) or self.isDenseConv(node)

Deeploy/Targets/Neureka/Parsers.py

@@ -4,6 +4,7 @@
 
 from typing import Tuple
 
+import numpy as np
 import onnx_graphsurgeon as gs
 
 from Deeploy.DeeployTypes import NetworkContext
@@ -50,14 +51,18 @@ def parseNodeCtxt(self,
         #         and enforcing that the channels_first is false
         data_in = newCtxt.lookup(self.operatorRepresentation['data_in'])
         data_out = newCtxt.lookup(self.operatorRepresentation['data_out'])
-        weight = newCtxt.lookup(self.operatorRepresentation['weight'])
+        # MARCHIOA: weight depends on the type of convolution so it requires to be parsed by the child parsers
+        #           - PW -> 3-dim
+        #           - DW -> 4-dim
+        #           - Dense -> 4-dim
+        # weight = newCtxt.lookup(self.operatorRepresentation['weight'])
 
         if not all([
                 channels_first == False,
                 len(data_in.shape) == 4,
-                # LMACAN: weight shape should be equal to 3 because we have to do the neureka's
-                #         special weight encoding
-                len(weight.shape) == 3,
+                # # LMACAN: weight shape should be equal to 3 because we have to do the neureka's
+                # #         special weight encoding
+                # len(weight.shape) == 3,
         ]):
             return newCtxt, False
 
@@ -83,18 +88,36 @@ def parseNode(self, node: gs.Node) -> bool:
         if not super().parseNode(node):
             return False
 
-        ch_im_out = node.inputs[1].shape[0]
-        ch_im_in = node.inputs[1].shape[1]
+        weights = node.inputs[1]
 
+        # weigths reshaped by the weigths encoder into
+        # (cout, cinMajor, bits, weightBandwidthBytes)
+        # where:
+        # - cout: 1 by definition (it is cin from ONNX)
+        # - cinMajor: number of tiles over the channels
+        # - bits: weight bit width (only 8 is supported)
+        # - weightBandwidthBytes: which is 32 in Siracusa
         if not all([
                 self.operatorRepresentation['kernel_shape'] == [3, 3],
-                self.operatorRepresentation['group'] == ch_im_out,
-                self.operatorRepresentation['group'] == ch_im_in,
+                len(weights.shape) == 4,
+                weights.shape[0] == 1,  # ch_im_out
         ]):
             return False
 
         return True
 
+    def parseNodeCtxt(self, ctxt, node, channels_first = True):
+
+        newCtxt, ret = super().parseNodeCtxt(ctxt, node, channels_first)
+        if not ret:
+            return newCtxt, False
+
+        weight = newCtxt.lookup(self.operatorRepresentation['weight'])
+        if not (len(weight.shape) == 4):
+            return newCtxt, False
+
+        return newCtxt, True
+
 
 class NeurekaRQSDWConv2DParser(NeurekaDWConv2DParser, RQSParserInterface):
 
@@ -136,6 +159,18 @@ def parseNode(self, node: gs.Node) -> bool:
 
         return True
 
+    def parseNodeCtxt(self, ctxt, node, channels_first = True):
+
+        newCtxt, ret = super().parseNodeCtxt(ctxt, node, channels_first)
+        if not ret:
+            return newCtxt, False
+
+        weight = newCtxt.lookup(self.operatorRepresentation['weight'])
+        if not (len(weight.shape) == 3):
+            return newCtxt, False
+
+        return newCtxt, True
+
 
 class NeurekaRQSPWConv2DParser(NeurekaPWConv2DParser, RQSParserInterface):
 
@@ -155,9 +190,24 @@ def parseNodeCtxt(self,
         if not ret:
             return ctxt, False
 
-        inputs = ['data_in', 'weight', 'mul', 'add']
-        for idx, inputNode in enumerate(node.inputs):
-            self.operatorRepresentation[inputs[idx]] = ctxt.lookup(inputNode.name).name
+        data_in = ctxt.lookup(node.inputs[0].name)
+        weight = ctxt.lookup(node.inputs[1].name)
+        mul = ctxt.lookup(node.inputs[2].name)
+        add = ctxt.lookup(node.inputs[3].name)
+
+        # The Neureka PW conv's RQS unit only supports per-tensor or
+        # per-output-channel requantization: mul/add must have either 1
+        # element or one element per output channel (weight's dim 0).
+        out_channels = weight.shape[0]
+        for tensor in (mul, add):
+            size = int(np.prod(tensor.shape))
+            if size not in (1, out_channels):
+                return ctxt, False
+
+        self.operatorRepresentation['data_in'] = data_in.name
+        self.operatorRepresentation['weight'] = weight.name
+        self.operatorRepresentation['mul'] = mul.name
+        self.operatorRepresentation['add'] = add.name
 
         return newCtxt, True
 
@@ -176,6 +226,18 @@ def parseNode(self, node: gs.Node) -> bool:
 
         return True
 
+    def parseNodeCtxt(self, ctxt, node, channels_first = True):
+
+        newCtxt, ret = super().parseNodeCtxt(ctxt, node, channels_first)
+        if not ret:
+            return newCtxt, False
+
+        weight = newCtxt.lookup(self.operatorRepresentation['weight'])
+        if not (len(weight.shape) == 4):
+            return newCtxt, False
+
+        return newCtxt, True
+
 
 class NeurekaRQSDenseConv2DParser(NeurekaDenseConv2DParser, RQSParserInterface):
 

Deeploy/Targets/Neureka/Templates/ConvTemplate.py

@@ -225,7 +225,6 @@ def getCounters(
 
     @classmethod
     def getWeightStrides(cls, channel_in: int) -> Tuple[int, int, int]:
-        n_channel_in = _getNumTiles(channel_in, 28)
         _NEUREKA_WEIGHT_BANDWIDTH_BYTES = 32
         return _NEUREKA_WEIGHT_BANDWIDTH_BYTES, 0, 0
 
@@ -256,12 +255,12 @@ def getCounters(
             operatorRepresentation: OperatorRepresentation) -> Tuple[int, int, int, int, int, int, int, int, int, int]:
         _ = operatorRepresentation  # operatorRepresentation not accessed for now because it's just for pointwise kernels
 
-        n_channel_out_subtiles = _getNumTiles(channel_out, 28)
+        n_channel_out_subtiles = _getNumTiles(channel_out, 32)
         n_channel_in_subtiles = _getNumTiles(channel_in, 28)
         n_height_out_subtiles = _getNumTiles(height_out, 6)
         n_width_out_subtiles = _getNumTiles(width_out, 6)
 
-        channel_out_border = _getBorderTileSize(channel_out, 28)
+        channel_out_border = _getBorderTileSize(channel_out, 32)
         channel_in_border = _getBorderTileSize(channel_in, 28)
         height_out_border = _getBorderTileSize(height_out, 6)
         width_out_border = _getBorderTileSize(width_out, 6)