feat: Adjust saturation, contrast, gamma and Acg convergence#19
Open
d3vv3 wants to merge 16 commits into
Open
Conversation
Contributor
|
Please see https://libcamera.org/contributing.html and sign up to the mailing list to send patches for review. Patches must be atomic/individual patch per topic, and a signed-off by tag is required for all submissions or we won't be able to accept them. Looking through this change set you might also be interested in checking out |
Add a CameraSensorHelper for the OmniVision OV01A10 image sensor, used in Dell XPS 13 and other laptops with the Intel IPU6 camera subsystem. The analogue gain register (0x3508) uses a Q6.8 fixed-point format, with the minimum value OV01A10_ANAL_GAIN_MIN = 0x100 representing unity gain. This gives the linear model: gain = code / 256 Hans de Goede confirmed linear behaviour by monitoring the 18% grey patch of a Macbeth chart under controlled lighting while stepping the gain control. The black level of 0x40 at 10 bits (4096 scaled to 16 bits) was confirmed by dark frame measurement with the lens covered. Without this helper, libcamera's AGC algorithm cannot convert between gain codes and real gain values, causing auto-exposure oscillation and the following warning: IPASoft: Failed to create camera sensor helper for ov01a10 Signed-off-by: Stuart J Mackintosh <sjm@opendigital.cc>
Author
|
I broke my commit down into atomic parts and included the patches you linked. I like being smarter about Acg, but still thing the defaults should be configurable per sensor. Had to rebase and tweak the patches since the code has changed a bit since. Next I will look into how to contribute these via the mailing list, never done that. |
Author
Add configurable YAML parameters maxGainR, maxGainB, and speed to AWB. Replace the single hardcoded max gain (4.0x) with per-channel limits, and apply exponential moving average smoothing to reduce colour temperature oscillation between frames. Signed-off-by: d3vv3 <devve.3@gmail.com>
…m YAML Read default values for gamma, contrast, and saturation from the tuning file so sensors can specify different image processing defaults without code changes. Falls back to prior defaults (gamma 2.2, contrast 1.0, saturation 1.0) when not specified in YAML. Signed-off-by: d3vv3 <devve.3@gmail.com>
Add tuning data for the OmniVision OV01A10 sensor, calibrated from the factory-supplied Intel IPU6 AIQB binary (ov01a_1BG101N3_MTL.aiqb) shipped with the Dell XPS 9320 (Meteor Lake). The AIQB binary was parsed with some custom scripts to extract: - 8 colour correction matrices spanning 2856K-7500K (Incandescent to D75), all with rows summing to 1.0 (luminance-preserving) - AWB neutral locus: 9 achromatic R/G, B/G points defining the sensor's white balance response across illuminants - Sensor properties: 1280x800, 10-bit, GBRG colour order, base ISO 43 - Noise model: read noise variance 7.5e-5, shot noise slope 0.187 The AWB gain limits (maxGainR: 2.5, maxGainB: 3.2) are derived from the maximum gains required to neutralise the warmest and coolest illuminants in the locus, with 10% headroom. Signed-off-by: d3vv3 <devve.3@gmail.com>
The AGC's updateExposure() uses a fixed ~10% step per frame regardless of how far the current exposure is from optimal. With a hysteresis dead band of only +/-4%, the controller overshoots when the correct value falls within one step, causing visible brightness oscillation (flicker). Replace the fixed-step bang-bang controller with a proportional one where the correction factor scales linearly with the MSV error: factor = 1.0 + error * 0.04 At maximum error (~2.5), this gives the same ~10% step as before. Near the target, steps shrink to <1%, eliminating overshoot. The existing hysteresis (kExposureSatisfactory) still prevents hunting on noise. Tested on OV2740 behind Intel IPU6 ISYS (ThinkPad X1 Carbon Gen 10) where the old controller produced continuous brightness flicker. The proportional controller converges in ~3 seconds from cold start with no visible oscillation. Signed-off-by: Javier Tia <floss@jetm.me> Reviewed-by: Milan Zamazal <mzamazal@redhat.com> Tested-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com> Signed-off-by: d3vv3 <devve.3@gmail.com>
The SWSTATS_ACCUMULATE_LINE_STATS() macro divides the luminance value for the histogram to normalize it to 8-bit range, but does not apply the same normalization to the RGB sums. For 10-bit and 12-bit unpacked Bayer formats this means the sums are accumulated at native bit depth (0-1023 or 0-4095 per pixel) while the AWB algorithm subtracts an 8-bit black level from them, under-correcting by 4x or 16x respectively. Fix this by right-shifting the RGB sums in finishFrame() to normalize them to 8-bit scale, matching the histogram and the 8-bit black level used by AWB. A per-format sumShift_ value is set in configure(): 0 for 8-bit and CSI-2 packed formats (already 8-bit), 2 for 10-bit, and 4 for 12-bit unpacked formats. Signed-off-by: Javier Tia <floss@jetm.me> Reviewed-by: Milan Zamazal <mzamazal@redhat.com> Tested-by: Milan Zamazal <mzamazal@redhat.com> Reviewed-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com> Tested-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com> Signed-off-by: d3vv3 <devve.3@gmail.com>
Set blackLevel_ = 4096 (0x40 at 10-bit) in CameraSensorHelperOv2740. The OV2740 kernel driver programs BLC target register 0x4003 with 0x40 for the 180 MHz link frequency mode. This matches the same pattern used by OV5675 and other OmniVision sensors with a 10-bit black level of 64. Without this, the Simple pipeline falls back to auto-guessing the black level, which happens to arrive at the same value but isn't documented. More importantly, the CameraSensorHelper is the canonical location for sensor calibration data and is used across all pipeline handlers, not just Simple. Suggested-by: Robert Mader <robert.mader@collabora.com> Signed-off-by: Javier Tia <floss@jetm.me> Reviewed-by: Milan Zamazal <mzamazal@redhat.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> Signed-off-by: d3vv3 <devve.3@gmail.com>
Replace the hardcoded kExposureOptimal, kExposureSatisfactory, and kExpProportionalGain constants with member variables read from the tuning file as exposureTarget, hysteresis, and proportionalGain. Defaults match the previous values (2.5, 0.2, 0.04). The constants are renamed to match their YAML keys and to use standard control-theory terminology: - kExposureOptimal -> exposureTarget: "optimal" implies a single universally correct value; "target" is the conventional ISP/AGC term for the setpoint the controller drives towards and is sensor-dependent. - kExposureSatisfactory -> hysteresis: the old name described the effect (exposure is satisfactory within this band) rather than the mechanism. "hysteresis" is the standard term for a deadband that prevents oscillation around a setpoint. - kExpProportionalGain -> proportionalGain: drops the redundant kExp prefix and matches the YAML key name directly. Signed-off-by: d3vv3 <devve.3@gmail.com>
Add comments documenting all algorithm YAML keys (BlackLevel, Awb, Ccm, Adjust, Agc) to the uncalibrated template so sensor calibration authors can discover parameters without reading source code. Update ov01a10.yaml replacing stepDenominator (removed) with proportionalGain for the new proportional AGC controller. Signed-off-by: d3vv3 <devve.3@gmail.com>
Verify that each row of a colour correction matrix sums to 1.0 (luminance preservation property). Tests cover: - identity and known-bad inline matrices - a real OV01A10 D65 calibrated CCM - parsing a multi-entry CCM YAML table via Interpolator<Matrix> - detection of a bad entry in YAML Tolerance is 5e-4 to accommodate 4-decimal-place rounding in tuning files. Signed-off-by: d3vv3 <devve.3@gmail.com>
The CCM rows sum to 1.0, meaning it expects a white-balanced signal in [0,1]. Previously AWB gains were multiplied into combinedMatrix alongside the CCM, producing a combined matrix with row sums far from 1.0 (e.g. [1.62, 0.63, 2.23] at 4500K). A neutral white pixel then produced R=1.62, G=0.63, B=2.23 -- magenta -- before any clamping. Instead, store AWB gains in DebayerParams::gains only, leaving combinedMatrix to hold just the CCM (and saturation). The shader applies AWB gains separately with a clamp to [0,1] before the CCM multiply.
Verify that applying AWB gains with a clamp to [0,1] before the CCM prevents magenta highlights, while the old approach of baking AWB gains into the combined matrix produces magenta on bright neutral pixels. Five test cases: 1. Old baked-matrix approach produces magenta on a bright neutral pixel. 2. New clamp-then-CCM approach produces neutral white on the same pixel. 3. Mid-tones are identical between both approaches (no unintended shift). 4. Fix works across multiple CCM colour temperatures (4000K and 6500K). 5. Black pixels are unaffected.
Add a meteringPercentile tuning parameter that switches the AGC from mean-sample-value (MSV) metering to percentile metering. Instead of targeting the mean histogram bin, the AGC finds the Nth percentile pixel in the full 64-bin luminance histogram and targets it at exposureTarget. With meteringPercentile=1.0 (default) the original MSV behaviour is preserved. With values below 1.0, the top (1-N)% of pixels are allowed to clip while the subject is correctly exposed -- preventing bright windows or lights from pulling down the exposure of the main subject. Set exposureTarget near the top of the 5-bin scale (4.5) when using percentile metering so the Nth percentile pixel lands near full brightness. The OV01A10 tuning uses meteringPercentile=0.97 which allows the brightest 3% of pixels to clip.
When an application requests a specific frame rate via FrameDurationLimits (e.g. Firefox/WebRTC at 30fps), the sensor silently clips exposure to fit within the frame period. Without the IPA knowing about this cap, the AGC would see the scene as too bright (since exposure is capped by the sensor but the IPA thinks there is still headroom) and compensate by increasing analogue gain, resulting in a severely overexposed image. Fix this by: - Advertising FrameDurationLimits in the IPA controls map (with the sensor's absolute exposure range as min/max, and max as the default so that unconstrained clients like qcam get full exposure headroom) - Storing absolute frameDurationMin/Max in IPASessionConfiguration - Handling FrameDurationLimits in queueRequest() to dynamically update exposureMax based on the requested max frame duration The AGC already respects exposureMax, so no changes to agc.cpp are needed.
The previous AGC applied a fixed proportional step each frame (factor = 1 + error * proportionalGain), which produced discrete visible jumps in exposure, especially when the scene changed rapidly. Replace this with a damped exponential approach: correctionFull = exposureTarget / exposureMSV factor = 1 + damping * (correctionFull - 1) Each frame moves a fraction (damping) of the remaining distance to the target, producing a smooth asymptotic curve rather than discrete steps. At the default damping of 0.25, ~94% of the correction is applied within 10 frames. Also add an asymmetric EMA filter on the metered MSV before computing the correction: - msvFilterAlphaUp (default 0.2): slow response when MSV rises (scene gets darker), avoiding pumping exposure up on transient dark frames. - msvFilterAlphaDown (default 0.6): fast response when MSV falls (scene gets brighter), preventing overexposure on sudden bright scenes. Both damping and the EMA alphas are tunable via YAML.
ov01a10.yaml: - Switch to damped approach AGC (remove proportionalGain, add damping: 0.2) - Restore meteringPercentile: 0.98 and exposureTarget: 4.5 -- empirically correct for scenes with bright backgrounds (windows, ceiling lights). The top 2% of pixels are allowed to clip; the subject is well exposed. - Lower hysteresis to 0.15 (damped curve naturally settles without needing a large dead zone) - Add msvFilterAlphaUp/Down for smooth, asymmetric metering response uncalibrated.yaml: - Document all new AGC parameters: damping, meteringPercentile, msvFilterAlphaUp, msvFilterAlphaDown - Expand exposureTarget documentation to explain the percentile metering interaction
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I know this should go to https://git.libcamera.org/libcamera/libcamera.git but I do not see a way to create an account