Skip to content

[Hackathon] feat: add image-native workflows#5107

Open
tanishqgandhi1908 wants to merge 1 commit into
apache:mainfrom
tanishqgandhi1908:feat/image-native-workflows
Open

[Hackathon] feat: add image-native workflows#5107
tanishqgandhi1908 wants to merge 1 commit into
apache:mainfrom
tanishqgandhi1908:feat/image-native-workflows

Conversation

@tanishqgandhi1908
Copy link
Copy Markdown

@tanishqgandhi1908 tanishqgandhi1908 commented May 16, 2026

Video submission

https://youtu.be/qkMTI_r6Z-I

What changes were proposed in this PR?

This hackathon submission focuses on making image workflows feel first-class inside Texera.

Motivation

Texera already works well for tabular data, but image-heavy workflows are harder to inspect and explain. A user may have a folder of images, want to build a workflow over them, and then inspect visual outputs, yet the experience often collapses back into raw binary values or table-only results.

The goal of this idea is to make visual data feel natural in a no-code workflow canvas.

User task Before After
Start from an image dataset No first-class path to treat a folder of images as structured input Load an image folder and get one structured row per image
Inspect image data Image bytes appear as opaque binary values Result tables render actual image thumbnails
Understand a visual result The user sees the final visual output but not the path behind it Clicking a visual result can open a Visual Journey side panel that explains the upstream flow

Main changes

  1. Support image folders as structured workflow input, with image bytes plus detected format and dimensions.
  2. Serialize recognized image binaries as image data URLs so the frontend can render them directly.
  3. Render image thumbnails in result tables instead of raw binary text.
  4. Add a reusable Visual Journey side panel for richer visual explanations of how an output was produced.
  5. Allow visualizer interactions, including iframe-origin clicks, to open the side panel reliably.

Why this matters

This makes Texera more useful for modern multimodal workflows such as image classification, visual QA, medical-image review, and report dashboards where the user needs to see both the data and the reasoning path through the workflow.

Image data in a table

Screenshot 2026-05-16 at 12 08 28 PM

HTML viewer of image

Screenshot 2026-05-16 at 12 08 47 PM

On click visual journey of data in side pane with steps

Screenshot 2026-05-16 at 12 09 40 PM Screenshot 2026-05-16 at 12 09 57 PM

This submission highlights the image-workflow slice of the broader implementation also discussed in #5094.

Any related issues, documentation, discussions?

How was this PR tested?

JAVA_HOME=$(/usr/libexec/java_home -v 17) sbt "testOnly org.apache.texera.amber.operator.source.scan.smart.CSVDialectSnifferSpec org.apache.texera.amber.operator.source.scan.smart.FormatDetectorSpec org.apache.texera.amber.operator.source.scan.smart.SmartFileSourceOpDescSpec org.apache.texera.amber.operator.source.scan.smart.SmartFileSourceOpExecSpec org.apache.texera.amber.operator.fileSplit.FileSplitOpDescSpec org.apache.texera.amber.operator.fileSplit.FileSplitOpExecSpec org.apache.texera.amber.operator.source.scan.file.FileScanSourceOpDescSpec org.apache.texera.web.service.ExecutionResultServiceSpec"

PATH="/Users/tanishqgandhi/.nvm/versions/node/v24.15.0/bin:$PATH" yarn ng test --watch=false --include='src/app/workspace/service/visual-trace/visual-trace.utils.spec.ts' --include='src/app/workspace/component/visual-trace-panel/visual-trace-panel.component.spec.ts' --include='src/app/workspace/component/result-panel/result-table-frame/result-table-cell.utils.spec.ts'

Manual verification:

  1. Loaded a folder-backed image dataset and confirmed it produced structured rows with image metadata.
  2. Confirmed result cells render visible image thumbnails instead of raw binary text.
  3. Opened an HTML visualizer, clicked a winner card, and confirmed the Visual Journey panel opens from visualizer interactions.

@github-actions github-actions Bot added feature engine dependencies Pull requests that update a dependency file frontend Changes related to the frontend GUI common agent-service labels May 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-service common dependencies Pull requests that update a dependency file engine feature frontend Changes related to the frontend GUI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant