[camera_web] Re: Support for camera stream on web#7950
Conversation
…d before expecting mock response
…d before expecting mock response
…d before expecting mock response
…d before expecting mock response
…d before expecting mock response
…d before expecting mock response
…m, and used window.animationFrames
…le height and width in takeFrame
…seOffScreenCanvas
DemoDummy example app source code 2026-04-03.19-47-24.mp4 |
|
Some lints have been reported here: https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8685251527310718641/+/u/Run_package_tests/analyze/stdout |
mdebbar
dismissed
ditman’s stale review
Many changes have been done since this review.
| ({int? audioBitrate, int? videoBitrate})? recorderOptions, | ||
| }) : recorderOptions = | ||
| recorderOptions ?? (audioBitrate: null, videoBitrate: null), |
There was a problem hiding this comment.
Why this change?
mockitowalks the constructor and inspects parameter defaults.- For a
constrecord literal, it goes throughsource_genconstant revival (reviveInstance). - That code path of the current version of the dependency does not handle record constants properly.
- As a result, it hits a null and throws Null check operator used on a null value.
|
Ready for review |
ditman
left a comment
ditman
left a comment
There was a problem hiding this comment.
This is very cool! I don't work on this nowadays, but here's some extra comments:
- Would
requestVideoFrameCallbackbe better than requestAnimationCallback? - Would it be possible to use
MediaStreamTrackProcessorif available inwindow? There's an example here: Video Frame Processing on the Web – WebAssembly, WebGPU, WebGL, WebCodecs, WebNN, and WebTransport
Anyway, very cool. Thanks for reviving this!
| // package:camera_platform_interface. | ||
| // https://github.com/flutter/flutter/issues/176148 | ||
| /// Used for running the camera frame stream at a specified FPS | ||
| final int cameraStreamFPS = 60; |
There was a problem hiding this comment.
Is this @visibleForTesting?
Also I think we should let the user know that this FPS is a maximum cap in the framerate, but not a guarantee. (Web devs should already know this, but maybe this is surprising for a mobile developer compiling for web)
There was a problem hiding this comment.
Making it @visibleForTesting would be a good idea. Let me do that.
I have updated the comment to say it out loud more explicitly.
| final num elapsed = timestamp - lastFrameTimestamp; | ||
|
|
||
| // If we're close to the next frame (~`_frameTimeToleranceMs`), do it. | ||
| if (fpsInterval - elapsed <= _frameTimeToleranceMs) { |
There was a problem hiding this comment.
Oh! So it is NOT a maximum cap on FPS. Does this mean that at 60fps we can trigger a new frame 8ms early?
There was a problem hiding this comment.
Yes, frames can trigger up to ~8ms early.
This means it's not really a maximum cap — it's more of a "try to hit 60fps with an 8ms tolerance window." You could actually see frame rates slightly exceed 60fps during burst scenarios.
| as web.OffscreenCanvasRenderingContext2D; | ||
|
|
||
| _offscreenCanvasContext!.drawImage(videoElement, 0, 0); | ||
| imageData = _offscreenCanvasContext!.getImageData(0, 0, width, height); |
There was a problem hiding this comment.
Can you please split the behavior of taking an offscreen frame vs the fallback into two smaller methods? _takeOffscreenCanvasFrame and _takeFallbackCanvasFrame (both return imageData). That way the takeFrame method becomes smaller?
There was a problem hiding this comment.
Valid point. I have addressed it.
| final ByteBuffer byteBuffer = imageData.data.toDart.buffer; | ||
|
|
||
| return CameraImageData( | ||
| format: const CameraImageFormat(ImageFormatGroup.unknown, raw: 0), |
There was a problem hiding this comment.
Would it make sense to extend the ImageFormatGroup to better describe what the browser is capturing here?
(There's also settings for getImageData that may be used here to capture frames in a "less unknown" format, maybe?)
There was a problem hiding this comment.
Supported pixelFormat values for CanvasRenderingContext2D.getImageData are:
rgba-unorm8, which corresponds to Flutter'sui.PixelFormat.rgba8888
2.rgba-float16
Neither is compatible with any current ImageFormatGroup value. For this PR, ImageFormatGroup.unknown is the appropriate value until ImageFormatGroup.rgba8888 exists. ImageFormatGroup.bgra8888 is also not suitable.
Here's the open issue for this flutter/flutter#151193. I have added this in the TODO comment.
|
Thanks for the valuable comments Ditman and the suggestion about MediaStreamTrackProcessor :) |
|
@TecHaxter – we love thoughtful, polite contributors. Thanks for helping out! |
7 participants
This PR aims to provide support for strartImageStream and stopImageStream on Web.
#92460
Based on #6944 from the archived plugins repository
Pre-launch Checklist
dart format.)[shared_preferences]pubspec.yamlwith an appropriate new version according to the [pub versioning philosophy], or this PR is [exempt from version changes].CHANGELOG.mdto add a description of the change, [following repository CHANGELOG style].///).POV: my first PR on a public repo
Contains required commits from the PR #6443, resolves unnecessary 202 commits