Skip to content

VISION_CONFORMANCE_REPORT.md

Latest commit

 

History

History
234 lines (167 loc) · 5.79 KB

VISION_CONFORMANCE_REPORT.md

File metadata and controls

234 lines (167 loc) · 5.79 KB

rustVX Vision Conformance Test Results

Date: April 4, 2026
Commit: c469193
Status: Testing Incomplete - Core Dump

Summary

Ran the full OpenVX Vision Conformance Test Suite. The test binary builds and runs but crashes during execution due to incomplete vision kernel implementations. The baseline tests (25 tests) continue to pass at 100%.

Test Suite Statistics

Metric Count
Total Tests 15,277
Test Cases 69
Tests Completed Before Crash ~240
Tests Passed ~62
Tests Failed ~179+
Crash Point During Image tests

Baseline Tests (Continue to Pass ✅)

All 25 baseline tests pass successfully:

  • ✅ GraphBase: 14/14
  • ✅ SmokeTestBase: 7/7
  • ✅ Logging: 1/1
  • ✅ TargetBase: 3/3

Vision Test Results Summary

Major Failure Categories

1. Graph Execution Failures

Tests fail because vision kernels are stubs (no actual algorithm implementation):

Failing Tests:

  • Graph.TwoNodes
  • Graph.GraphFactory
  • Graph.VirtualImage
  • Graph.VirtualArray
  • Graph.NodeRemove
  • Graph.NodeFromEnum
  • Graph.TwoNodesWithSameDst
  • Graph.Cycle
  • Graph.Cycle2
  • Graph.MultipleRun
  • Graph.MultipleRunAsync
  • Graph.NodePerformance
  • Graph.GraphPerformance

Root Cause: Vision kernel functions (vxSobel3x3, vxColorConvert, etc.) are exported but contain no actual algorithms

2. Kernel Lookup Failures

Tests fail to find kernels by enum:

Failing Tests:

  • Graph.KernelName/7/org.khronos.openvx.table_lookup
  • Graph.KernelName/8/org.khronos.openvx.histogram

Error: vxGetKernelByEnum returns NULL for some kernel enums

Root Cause: Kernel registry lookup by enum not working for all kernels

3. Image Cloning Failures

Tests fail to clone images:

Failing Tests:

  • Image.VirtualImageCreation
  • Image.VirtualImageCreationDims

Error:

ENGINE: Unable to make a clone of vx_image
Invalid OpenVX object "clone"
Expected: VX_TYPE_IMAGE object
Actual: NULL

Root Cause: vxCloneImage function not implemented or failing

4. Image Format Failures

Tests fail for specific image formats:

Failing Test:

  • Image.CreateImageFromHandle/9/VX_DF_IMAGE_NV12

Error:

Testing (tst) image to be exactly equal to (src)
16x16 NV12 images
Max difference (255) is found at offset 277:
Expected: 255
Actual:   0
Totally 128 bytes with different values

Root Cause: NV12 format handling incorrect in image creation

5. Memory Allocation Crash

Final Error:

Fatal glibc error: malloc.c:2599 (sysmalloc): assertion failed
(old_top == initial_top (av) && old_size == 0) || 
((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && 
((unsigned long) old_end & (pagesize - 1)) == 0)

Root Cause: Memory corruption due to improper image data handling

Specific Test Failures Sample

Test Category Tests Run Tests Passed Tests Failed Pass Rate
GraphBase 14 14 0 100%
SmokeTestBase 7 7 0 100%
Graph 50+ ~5 ~45 ~10%
Image ~150+ ~30 ~120 ~20%
Total (before crash) ~240 ~62 ~179 ~26%

Required Fixes for Vision Conformance

Critical Priority

  1. Implement Actual Vision Kernel Algorithms

    • Color conversion (RGB ↔ YUV, etc.)
    • Sobel edge detection
    • Gaussian filtering
    • Optical flow (Lucas-Kanade)
    • Harris corner detection
    • Histogram calculation
    • Table lookup (LUT)

  2. Fix vxCloneImage Function

    • Currently returning NULL or failing
    • Required for virtual image tests

  3. Fix Kernel Lookup by Enum

    • vxGetKernelByEnum returns NULL for some kernels
    • All kernels should be findable by enum

High Priority

  1. Fix Image Format Handling

    • NV12/NV21 format handling incorrect
    • YUV planar format support incomplete

  2. Fix Memory Management

    • Memory corruption in image handling
    • Need proper bounds checking

  3. Implement Missing Virtual Object Support

    • Virtual images need proper backing
    • Virtual arrays need implementation

What Was Fixed

✅ Completed for Vision Support:

  1. vxMapDistribution - Implemented
  2. vxUnmapDistribution - Implemented
  3. All distribution functions exported
  4. CTS builds and links successfully

Recommendations

To Pass Vision Conformance:

Phase 1: Core Vision Algorithms (Estimated: 4-6 weeks)

  1. Implement color space conversion kernels
  2. Implement filtering kernels (Gaussian, Sobel, etc.)
  3. Implement feature detection kernels
  4. Add numerical accuracy validation

Phase 2: Image Operations (Estimated: 2-3 weeks)

  1. Fix vxCloneImage implementation
  2. Fix image format conversions
  3. Fix memory management issues

Phase 3: Testing & Validation (Estimated: 2-3 weeks)

  1. Run full CTS suite
  2. Debug individual test failures
  3. Validate numerical accuracy against reference

Estimated Total Time: 8-12 weeks of focused development

Conclusion

While rustVX achieves 100% baseline conformance, vision conformance requires significant additional work. The infrastructure is in place (300 functions exported, CTS builds and runs), but actual vision algorithms need to be implemented.

The current status:

  • ✅ Baseline: 100% (25/25 tests passing)
  • ⚠️ Vision: ~26% (estimated from tests before crash)
  • ❌ Vision Complete: No (crashes during test run)

Next Steps

  1. Fix immediate crash - Memory corruption in image handling
  2. Implement vxCloneImage - Required for virtual image tests
  3. Implement one vision kernel - Start with color_convert as proof of concept
  4. Add algorithm accuracy tests - Compare against OpenCV or Khronos reference

Report generated: April 4, 2026
rustVX Commit: c469193