Skip to content

media: Surface Pro 5 (IPU3) camera fixes: OV8865, OV5693, OV7251 IR illuminator - #181

Open
rylos wants to merge 11 commits into
linux-surface:v7.2-surface-develfrom
rylos:sp5-ipu3-cameras
Open

rylos wants to merge 11 commits into
linux-surface:v7.2-surface-develfrom
rylos:sp5-ipu3-cameras

Conversation

@rylos

@rylos rylos commented Oct 23, 2026

Copy link
Copy Markdown

Fixes for the three cameras of the Surface Pro 5 (1796, Kaby Lake / IPU3), tested on v7.2.12 with the current 7.2 patch set. Previously posted in linux-surface/linux-surface#2233. Most of them are generic or apply to the other IPU3 Surfaces with the same sensors (Book 2, Pro 6, Go).

OV8865 (rear)

  • OV8865_VTS_H() keeps bits 11:8 only, so any frame length above 4095 lines wraps around and long exposures give black frames. Generic.
  • VBLANK reset to the mode default on mode change (exposure range derived from it).
  • Analogue gain step 1 instead of 128 (the AGC was limited to integer 1x..16x).
  • SP5 only: 800x600 defaults to 30 fps; libcamera's IPU3 pipeline never changes VBLANK, so at 90 fps 640x480 was black indoors.

The "program the mode on stream start" fix I had for the OV8865 is dropped: Jurison Murati's patches already on this branch cover it.

OV5693 (front)

OV7251 (IR)

  • The illuminator hangs off the sensor's STROBE pad, which ACPI doesn't describe and the mode tables leave disabled. Enabled on stream start with an alternate-frame pattern (lit/unlit frames), pulse length following the exposure. On by DMI for the SP5, override with ov7251.surface_ir_strobe / surface_ir_pattern.

Orientation (SP5)

  • The SSDB reports 180° for front and rear, but both need a vertical flip only. ipu-bridge reports 0° for INT33BE/INT347A on SKU 1796/1807, and both drivers invert VFLIP by DMI.

Verification: 11 libcamera configurations (320x240 → full resolution, front and rear), 0 CSI-2 errors, correct frame rates, neutral colours; IR frames alternate between mean level 16 and 42; face login with howdy working.

@CmdrShepardsPie

Copy link
Copy Markdown

I tested this PR's head (76d2ae0) as-is on a Surface Pro 9 (IPU6, Alder Lake-P 8086:465d): linux-surface's Fedora 7.2.12-1 package plus these 10 commits, with the front ov5693 through libcamera's soft ISP (Snapshot and cam).

Works on IPU6:

  • Mode switch (ac26433): the first stream after a switch got 30/30 frames, 18 of 18 times full→binned and 18 of 18 binned→full. Without it, that stream got no frames in 5 of 5 tries in my earlier test.
  • Binned 1296x972 with binned_regs 0, 1 (default) and 2: 3 × 300/300 frames each at 28.64 fps, no frame-sync errors or FIFO overflows, correct colours.
  • Full 2592x1944: 3 × 300/300 frames at 28.64 fps.
  • The unconditional MIPI_CTRL00 0x2d write caused no errors on IPU6.

Regression: full-resolution colours (1ee078c). On this unit the sensor runs with horizontal_flip=1 and vertical_flip=1 (rotation 180). Full-resolution images come out purple and green in Snapshot. Raw BG10 frames from the ISYS node show the delivered Bayer order doesn't match what the subdev reports (SBGGR10) when hflip is on:

Mode hflip/vflip Delivered
full 0/0 BGGR
full 1/0 GBRG
full 0/1 BGGR
full 1/1 GBRG
binned 0/0 and 1/1 BGGR

So in full mode the horizontal-flip column offset is one column off on this sensor. The row handling is right, and binned mode is right with both flips. Stock 7.2.11 and 7.3 give correct full-resolution colours on this unit. I'm happy to test a revision, and I'll add a Tested-by once full mode is fixed.

@rylos

rylos commented Oct 25, 2026

Copy link
Copy Markdown
Author

Thanks for testing, and for the table — it pinned it down. In the full readout each flip moves the Bayer phase by one column, so the two cancel out when both are set; my patch only compensated the vertical one, which was enough on the SP5 (hflip stays 0 there) but not with the 180° rotation on the Pro 9.

Fixed in 1ee078c → dee4ccd (force-pushed): the output window is shifted when hflip XOR vflip, and reprogrammed when either flip changes. Binned mode is untouched.

On the Surface Pro 5, raw full-resolution frames from the CIO2 now come out BGGR for all four hflip/vflip combinations (before: GBRG whenever hflip was set), and libcamera still gives 0 errors and natural colours at every size. A re-test on the Pro 9 would be welcome.

@loafecho

Copy link
Copy Markdown

@rylos, a data point for the OV5693 binned mode from a different receiver: a Surface Pro 7 (Ice Lake, IPU4P).

Disclosure: this work was done with help from an AI coding assistant. I ran it on my own Surface Pro 7.

Setup: Ubuntu 24.04, linux-surface 7.3.0-rc7 with Sakari's IPU7 v4 series, ruslanbay's IPU4P-on-IPU6 RFC series and my SP7 patches. The front OV5693 is on CSI-2 port 2 (firmware source 7, two lanes), with his fixed receiver timing for that port.

What I tested: only the ov5693.c changes of this PR at head 5b3b61e7c6 (which includes your hflip-XOR-vflip window shift), applied cleanly to the 7.3 ov5693.c, with the defaults (binned_regs=1, binned_y_offset=2). The SP7 isn't in your DMI table, so the SP5 flip handling doesn't apply here.

Method: raw ISYS node, no libcamera. Each start sets the sensor pad and the CSI-2 pads to 1296x972 SBGGR10 with media-ctl (the sensor pad read back as 1296x972 every time), then captures 45 frames with v4l2-ctl. 10 fresh starts, each after the IPU had runtime-suspended.

ov5693 Clean starts (45/45 frames, no csi2-2 error lines)
This PR's ov5693 changes 10/10, 0 error lines
Without them (my 7.3 ov5693, binned modes not hidden, no 0x4800 write; see note below) 0/10, no frames, 200 csi2-2 error lines, mostly Frame sync error (rate-limited, so a lower bound)

The binned-mode patch from linux-surface#2252 on its own gave the same 10/10 here: linux-surface/linux-surface#2252 (comment)

Not measured: the ov7251, ov8865 and ipu-bridge changes, full resolution with this PR (so not the hflip/vflip Bayer fix), the frame rate, colours, mode switches (every start set the format after a suspend), and the other binned_regs values. I also didn't bisect the binned path (0x4800, PLL and readout registers, readout geometry, VTS), so I can't say which part the IPU4P needs.

Note on the "without" row: my ov5693 carries a local hflip fix, and at its default both horizontal flip bits are set, while the patched module is stock-based and clears them. So the 0/10 isn't a clean stock baseline. It's the binned mode as I had it.

@rylos

rylos commented Oct 26, 2026 •

Copy link
Copy Markdown
Author

@loafecho thanks, that's a useful third receiver: binned 1296x972 clean on IPU3 (SP5), IPU6 (Pro 9, @CmdrShepardsPie's test) and now IPU4P.

If you have time for one more run, full resolution would cover the part nobody has checked off this receiver yet: 2592x1944 SBGGR10 on the raw ISYS node with the four horizontal_flip/vertical_flip combinations, and whether the delivered Bayer order stays BGGR. The SP7 isn't in the DMI table, so the controls map straight to the sensor flips there. With this head every combination should come out BGGR. With the previous head (76d2ae0), every setting with hflip on came out GBRG.

A quick way to tell without libcamera: split a raw frame into its four 2x2 sub-planes. The green pair is the diagonal whose two planes are most alike, so (0,1)/(1,0) means BGGR/RGGB and (0,0)/(1,1) means GBRG/GRBG. That's how I checked it on the SP5.

@CmdrShepardsPie

Copy link
Copy Markdown

Re-tested at 5b3b61e on the Surface Pro 9 (Intel, Alder Lake-P; IPU6, front ov5693) on two kernels: linux-surface's Fedora 7.2.12-1 package plus this PR's 10 commits, and v7.3-surface (1ea399d) plus the same commits. "Before" is the 7.2.12-1 package as-is. Raw frames come from the ISYS node; long runs and mode switches use libcamera 0.5.2 cam (soft ISP).

Front ov5693 7.2.12-1 as-is 7.2.12-1 + #181 v7.3-surface + #181
Raw starts: full at the 4 flip combos, binned at 0/0 and 1/1, two passes 2 of 12 streamed 12/12 12/12
Full-res Bayer order at hflip/vflip 0/0, 1/0, 0/1, 1/1 no frames, GBRG, GBRG, no frames BGGR ×4 BGGR ×4
Binned Bayer order at 0/0 and 1/1 no frames BGGR BGGR
Full 2592x1944, 3 × 300 frames 300/300 once, then no frames twice 300/300 ×3, 28.64 fps 300/300 ×3, 28.64 fps
Binned 1296x972, 3 × 300 frames no frames 300/300 ×3, 28.64 fps 300/300 ×3, 28.64 fps
Mode switches, 6 each way no frames in either stream 12/12, 30/30 after the switch 12/12, 30/30 after the switch
isys "stream stop time out" (each followed by a close time-out) 50 0 0
Rear ov13858, 3 × 90 frames 90/90 ×3 90/90 ×3 90/90 ×3

The flips read back as set in every case. No frame-sync errors; the occasional FIFO overflow or inter-frame packet discard (up to 3 per switch pair) also shows up without the PR.

  • The mode switches at gap 0 weren't strictly inside the 1 s autosuspend window: runtime_suspended_time shows the sensor suspended for 10–17 ms before the second stream ended, so it most likely started from a fresh power-up. They show switching works, not ac26433 on its own. The in-window case is in my 6.19.8 run on cameras: Surface Pro 7+ — make the binned OV5693 stream on IPU6, drive the IR illuminator, map the int3472 sensor rails linux-surface#2252 (that PR's ov5693 changes: 0 of 50 in-window switches streamed; with ac26433 added, 14 of 14). The same goes for the 18 of 18 each way I reported earlier in this thread: that harness didn't record suspends either, so it shows switching works, not the in-window case.
  • Snapshot: the front camera works, with the slightly desaturated colours this unit has for a separate reason. The rear image is upside down, as it is without this PR (the orientation fix here is for the Pro 5 only).
  • Userspace, for the record: now that the front camera streams, switching cameras in Snapshot on the 7.3 build aborted WirePlumber once in 7 switches (libcamera 0.5.2, assertion data->queuedRequests_.empty() in PipelineHandler::stop() while stopping the front camera; no kernel camera errors at the time).

This covers the ov5693 changes and the rear camera still working; the OV8865/OV7251 and Pro 5 parts don't apply to this hardware.

Tested-by: Chris Simpson chris@nilk.net # Surface Pro 9 (IPU6), 7.2.12 and 7.3

Do you plan to send the ov5693 changes, including the stream-start mode programming (ac26433), to linux-media? If so, I'll test what you post on the Pro 9 and reply with a Tested-by on the list, and the Pro 9 mode-switch numbers are yours to cite.

@loafecho

Copy link
Copy Markdown

@rylos, here's the full-resolution run, on the same Surface Pro 7 (Ice Lake, IPU4P).

Disclosure: as before, this work was done with help from an AI coding assistant. I ran it on my own Surface Pro 7.

Setup: same as before: Ubuntu 24.04, linux-surface 7.3.0-rc7 with Sakari's IPU7 v4 series, ruslanbay's IPU4P-on-IPU6 RFC series and my SP7 patches. The front OV5693 is on CSI-2 port 2 (firmware source 7, two lanes).

What I tested: the ov5693.c changes of this PR at head 5b3b61e7c6, applied to the stock 7.3 ov5693.c as before. I also ran my installed ov5693 as a reference. It's the 7.3 driver with my local hflip fix, not stock.

Method: raw ISYS node, no libcamera.

  • One module load, then for each setting:
    • Wait until the IPU is runtime-suspended.
    • Set horizontal_flip/vertical_flip on the sensor subdev.
    • Set the sensor pad to 2592x1944 SBGGR10 and pass whatever code it reports on to the CSI-2 pads and the video node (it reported SBGGR10_1X10 every time).
    • Capture 45 frames, and run your 2x2 sub-plane test on frame 23.
  • "Alike" is measured two ways: Pearson r, and mean |a−b| divided by the pair's mean (lower = more alike).
hflip vflip Frames (0,1)/(1,0): r / diff (0,0)/(1,1): r / diff Green pair → order
0 0 45/45 0.9921 / 0.0638 0.9170 / 0.4596 (0,1)/(1,0) → BGGR/RGGB
1 0 45/45 0.9921 / 0.0640 0.9220 / 0.4743 (0,1)/(1,0) → BGGR/RGGB
0 1 45/45 0.9877 / 0.0717 0.9163 / 0.4740 (0,1)/(1,0) → BGGR/RGGB
1 1 45/45 0.9924 / 0.0632 0.9199 / 0.4680 (0,1)/(1,0) → BGGR/RGGB

So it stays on the BGGR/RGGB diagonal in all four, as you expected. That matches CmdrShepardsPie's Pro 9 (IPU6) re-test just above, which got BGGR at all four full-resolution flip settings on IPU6; this table shows the same diagonal on an IPU4P receiver. None of the four captures logged a csi2-2 error line. The diagonal test can't tell BGGR from RGGB, but the darker of the two non-green planes is at (0,0) every time, which fits blue there. That depends on the scene, so take it as consistent with BGGR, not proof.

Did the flips really happen? A second run, about two minutes after the one above, on the same scene (30 frames per setting, its own module load; I didn't check the scene for movement). I compared the green-plane column and row profiles of each setting against hflip=0/vflip=0:

  • hflip on matched the reversed column profile (r 0.9995, against 0.8599 as-is).
  • vflip on matched the reversed row profile (0.9978, against −0.7989 as-is).
  • Both on matched both reversed (0.9992 and 0.9865).

So the second run shows the controls reach the sensor, and in the first run the order held at all four settings.

Reference, my installed ov5693: GBRG/GRBG at vflip=0 (with hflip off or on) and BGGR/RGGB at vflip=1, while reporting SBGGR10 every time. A separate run right after the #181 one, same scene: green pair (0,0)/(1,1) at r 0.9921 with diff 0.0641 for hflip=0/vflip=0. So the test does separate the two diagonals on this sensor. My hflip fix inverts the control (at hflip=0 both horizontal flip bits are set), so its hflip rows aren't the same register state as this PR's, and it's a different driver from your 76d2ae0, so the pattern isn't comparable to yours.

Not measured: the binned mode with flips, frame rate, colours through libcamera, and the ov7251, ov8865 and ipu-bridge changes. Only one frame per setting went through the sub-plane test.

rylos and others added 10 commits October 27, 2026 08:08
Same issue as ov8865: the mode registers are only written on runtime PM
resume, so a format change on a sensor that stays powered never reaches
the chip. Program the mode before the controls on every stream start.

Tested-by: Chris Simpson <chris@nilk.net> # Surface Pro 9 (IPU6)
Signed-off-by: Marco Ziliani <rylos78@gmail.com>
Patchset: cameras
The SSDB of the Surface Pro 5 (SKU 1796, and 1807 for the LTE model)
reports a 180 degree rotation for both the front OV5693 and the rear
OV8865. Measured on the device, both sensors need a vertical flip only,
without mirroring, which is not a rotation: applying the reported 180
degrees leaves both images mirrored.

- ipu-bridge: add a DMI list of sensors that are not rotated despite
  what the SSDB says, and report 0 degrees for them.
- ov8865: on these machines invert the VFLIP control, so the default
  value gives an upright image and userspace (libcamera resets the
  flip controls) keeps working.

The front sensor gets the same treatment in a separate patch.

Signed-off-by: Marco Ziliani <rylos78@gmail.com>
Patchset: cameras
On the Surface Pro 5 the IR flood LED is driven by the OV7251 STROBE
pad, not by a GPIO or the TPS68470 LED outputs, and ACPI does not
describe it. The mode tables leave the pad disabled (0x3005 = 0x00,
0x3016..0x301c = 0xf0), so the LED never lights and the IR camera,
which sits behind an IR-pass filter, only sees black.

Enable the STROBE output on stream start on the Surface Pro 5 (SKU 1796
and 1807), with an alternate-frame pattern (0x3b81 = 0xaa) so userspace
gets lit and unlit frames. The module parameter surface_ir_strobe
overrides the DMI match (-1 auto, 0 off, 1 on).

Verified on a Surface Pro 5 (1796): with the strobe enabled the mean
level of consecutive 640x480 frames alternates between 16 and 42, with
it disabled it stays at 16.

Signed-off-by: Marco Ziliani <rylos78@gmail.com>
Patchset: cameras
The driver's 1296x972 binned mode has never streamed on IPU6 machines
(Surface Pro 7+/8/9, Go 4): the receiver reports "Transfer FIFO
overflow" or "Frame sync error" and no frame ever completes. Because of
that, every IPU6 setup has had to read the sensor at full 2592x1944 and
downscale, throwing away two stops of light and pinning the AGC at its
maximum analogue gain (127 vs ~15) - the familiar "working but unusably
noisy" IPU6 front camera. Fix the mode by porting what the vendor
(Windows) driver ov5693.sys programs differently, extracted from its
mode tables:

- Per-mode MIPI PLL (0x30b3: 0x70 binned / 0x83 full) and six per-mode
  analogue readout registers. Mainline writes only the full-resolution
  values, once, globally; the binned readout then overflows the
  receiver's FIFO.

- Readout geometry with spare pixels: cropping exactly the output size
  starves the sensor's internal ISP and every frame fails CSI-2 framing
  ("Frame sync error"). The vendor reads the whole pixel array, bins it
  to 1312x978, and places the 1296x972 output via the ISP window
  offsets. The parity of the Y offset selects the vertical Bayer phase
  (even = the advertised BGGR, odd = GRBG); it is exposed as a module
  parameter (binned_y_offset, default 2) because libcamera's software
  ISP wants the advertised phase while pipelines built around the
  Windows tuning (Intel's IPU6 HAL/PSYS) expect the vendor's odd
  offset.

- Mode-dependent vertical-flip bits: with binning, setting both FORMAT1
  flip bits corrupts the CSI-2 framing and the ISP bit alone does not
  flip; only the sensor-side bit both streams and flips. Re-apply the
  vflip control when the mode is configured, since the correct bits
  depend on the selected mode.

- Cap the frame rate at 30 fps: the auto-picked 60 fps VTS for the
  binned mode exceeds what the slower binned MIPI PLL can carry (the
  vendor runs this mode at VTS 1984 / 30 fps).

- Program MIPI_CTRL00 (0x4800) = 0x2d before stream-on. The 0x00
  power-on default is fine on IPU3 but leaves the IPU6 D-PHY unable to
  lock: the sensor streams yet no CSI-2 data arrives.

Also document the black-level pedestal next to the BLC init values: the
sensor keeps ~16 LSB (10-bit) at every analogue gain, while OEM .aiqb
calibrations state 64.9 because Windows' ISYS delivers RAW10 aligned to
12 bits.

Verified on a Surface Pro 7+ (IPU6, Tiger Lake): 1296x972 at 30 fps,
analogue gain in normal indoor light drops from 127 (maximum) to ~15,
both through libcamera's software ISP and through Intel's IPU6 HAL on
top of the hardware ISP. Reference setup and measurements:
https://github.com/dmanresa-saes/surface-ipu6-cameras

Signed-off-by: D. Manresa <d.manresa@electronica-submarina.com>
Tested-by: Chris Simpson <chris@nilk.net> # Surface Pro 9 (IPU6)
Signed-off-by: Marco Ziliani <rylos78@gmail.com>
Patchset: cameras
The front OV5693 of the Surface Pro 5 is mounted vertically mirrored:
the upright, unmirrored image needs the sensor vertical flip and no
horizontal flip. The firmware reports a 180 degree rotation, which makes
userspace enable both flips and leaves the image mirrored.

ipu-bridge now reports 0 degrees for this sensor; invert the VFLIP
control on the Surface Pro 5 (SKU 1796/1807) so that its default value
gives the upright image.

In the full 2592x1944 readout each flip also moves the Bayer phase by
one column: with exactly one of them enabled the receiver gets GBRG
instead of BGGR, which gives a magenta image through libcamera; with
both enabled the two shifts cancel out. The crop start only moves in
steps of two columns, so shift the output window by one column and
widen the crop accordingly when hflip XOR vflip, and reprogram the
window when either flip changes. The binned readout keeps its phase
with either flip.

Verified on a Surface Pro 5 (1796): raw full-resolution frames are
BGGR for all four hflip/vflip combinations; upright and unmirrored in
the full and in the binned 1296x972 mode, natural colours through
libcamera.

Tested-by: Chris Simpson <chris@nilk.net> # Surface Pro 9 (IPU6)
Signed-off-by: Marco Ziliani <rylos78@gmail.com>
Patchset: cameras
…rface Pro 5

set_fmt() updates the VBLANK range and default for the new mode but keeps
the current value when it is still in range. A value left over from the
previous mode then keeps that mode's timing: after the full-size mode
(VBLANK 22) the 800x600 mode runs faster than its nominal 90 fps, with
the exposure capped accordingly. Set VBLANK to the mode default on every
active format change, and derive the exposure range from it.

On the Surface Pro 5 the rear camera is used through libcamera's IPU3
pipeline, which does not change the frame length. With the 800x600 mode,
which libcamera picks for 640x480 and smaller requests, the exposure is
limited to ~11 ms and indoor images are black even at maximum analogue
gain. On this machine (SKU 1796/1807) default that mode to 30 fps by
tripling its frame length; higher frame rates remain available through
VBLANK.

Verified on a Surface Pro 5 (1796): 800x600 at 30.6 fps with the full
exposure range usable.

Signed-off-by: Marco Ziliani <rylos78@gmail.com>
Patchset: cameras
Signed-off-by: Marco Ziliani <rylos78@gmail.com>
Patchset: cameras
Signed-off-by: Marco Ziliani <rylos78@gmail.com>
Patchset: cameras
With the readout and MIPI PLL values of the 60 fps binned mode, the IPU3
CIO2 of the Surface Pro 5 loses several frames per stream ("frame sync
error", payloads of 778 or 831 lines instead of 972). The 30 fps binned
values (MIPI PLL multiplier 0x7d, analogue 0x3620-0x3622, 0x3708,
0x3709, 0x371f) stream cleanly: 90 of 90 frames, no errors, through
libcamera at 640x480.

Use them by default; the module parameter binned_regs selects the old
set (0) or the 30 fps values with the full-mode PLL (2). Also set 0x4307
to 0x30 and add a blc_target parameter for the black level target
(0x4009) for further tuning.

Tested-by: Chris Simpson <chris@nilk.net> # Surface Pro 9 (IPU6)
Signed-off-by: Marco Ziliani <rylos78@gmail.com>
Patchset: cameras
Signed-off-by: Marco Ziliani <rylos78@gmail.com>
Patchset: cameras
@rylos

rylos commented Oct 27, 2026

Copy link
Copy Markdown
Author

@CmdrShepardsPie @loafecho thanks, both. So the full-resolution Bayer order is now BGGR at all four flip settings on IPU3 (SP5), IPU6 (Pro 9) and IPU4P (SP7), and binned streams cleanly on all three.

I've added Tested-by: Chris Simpson <chris@nilk.net> # Surface Pro 9 (IPU6) to the four ov5693 commits (force-pushed, head 2bc8353; no code changes, trailers only). ac26433 is now e753f13.

Yes, I'll send the ov5693 changes to linux-media, starting with the stream-start mode programming, and I'll Cc you both. Your in-window mode-switch numbers on the Pro 9 (0 of 50 without the fix, 14 of 14 with it) are exactly what the commit message needs, so I'll cite them with a Reported-by/Tested-by as you suggested.

@dmanresa-saes

Copy link
Copy Markdown

@CmdrShepardsPie asked for in-window mode switches from the Pro 7+ before it goes. Here they are, with ac2643331 and without.

Short answer: on Tiger Lake it is 0/16 without the commit and 16/16 with it, and the register readback says why. But I could not use the method as given, and that part matters more than the number — please read it before quoting anything.

Why asym2.sh does not measure this on a Pro 7+

Run as supplied, the script said the unpatched module passed 12 of 12 in-window pairs, i.e. "no bug here". That is false, and here is the proof:

On this machine cam runs the simple pipeline handler with the software ISP. Asked for 1296x972 it does not switch the sensor mode at all — it leaves the sensor at full resolution and downscales in software. Taken 136 frames into a live cam -c3 -s width=1296,height=972:

pad0: Source  [stream:0 fmt:SBGGR10_1X10/2592x1944@1/30
OUTPUT_SIZE_X(0x3808)=2592  OUTPUT_SIZE_Y(0x380a)=1944
FORMAT1(0x3820)=0x42 FORMAT2(0x3821)=0x18 SUB_INC_X(0x3814)=0x11 SUB_INC_Y(0x3815)=0x11

Binning bits clear, subsample 1:1, full output size, mid-stream. ftrace over an in-window stream shows only ov5693_set_fmt and ov5693_s_stream running and ov5693_sensor_resume never running — so the premise of the bug is intact on this tree, the script just never asked the sensor to change mode. (This is consistent with the simple pipeline's own size matching: its debayer keeps a 2-pixel border, so a 1296-wide binned readout can serve at most 1292, and a request for exactly 1296 selects the full-resolution mode.)

@CmdrShepardsPie, you described your Pro 9 runs as libcamera cam with the soft ISP too. Your 0/20 and 0/30 failures suggest yours really was switching modes, so your setup probably differs from mine — but it would be worth saying which pipeline handler cam -l reports on the Pro 9, so nobody later repeats the no-op version of this test.

Second problem: the autosuspend delay is bypassed

Raising power/autosuspend_delay_ms to 5000 does nothing for this driver. ov5693_s_stream()'s stop path calls plain pm_runtime_put(), not pm_runtime_put_autosuspend(), so the delay is skipped at stream stop. With cam the sensor suspended in 158 to 233 ms on every pair regardless of the setting — all 12 pairs of my first run were out-of-window. That probably explains the 10 to 17 ms suspends you kept hitting on the Pro 9.

I pinned the sensor with echo on > power/control instead. That gives a verified 0 ms runtime_suspended_time delta across the pair and, by ftrace, no ov5693_sensor_resume between the two streams — the exact code path the bug describes.

The test I used

Direct V4L2, no libcamera: media-ctl on "ov5693 2-0036":0 and "Intel IPU6 CSI2 4":0/:1, then v4l2-ctl --stream-mmap --stream-count=20 on /dev/video33. After stream B each pair reads OUTPUT_SIZE_X (0x3808) back over i2c, which is ground truth for whether the switch reached the chip.

Results, in-window, 16 pairs per module, 0 discarded

Without ac2643331: OUT_X stuck at 1296 on all 16 pairs — the sensor never leaves the mode it powered up in. A stream gets frames if and only if the mode it asks for already matches. Counting only the streams that asked for a different mode (the 8 full-res A's in f>b, the 8 full-res B's in b>f): 0 of 16 produced a single frame. In your per-direction form that is binned→full 0/8; I would not report full→binned as 8/8, because in those pairs the chip was already binned and no register was written — the switch that genuinely had to happen there is the A stream, 0/8.

With ac2643331: 32 of 32 streams at 20/20, and OUT_X tracks the requested mode on every pair. full→binned 8/8, binned→full 8/8.

Out-of-window control (control=auto, 8 s gap, sensor really suspends, 67 to 82 ms): 10 pairs per module, every stream 20/20, OUT_X correct, base and patched indistinguishable. The harness is not just failing everything, and the commit causes no regression on the power-cycling path.

Errors: zero frame-sync errors anywhere, in any run. FIFO overflow (csi2-4 error: Transfer FIFO overflow) appears on pairs where the sensor power-cycles, 3 to 4 per pair on both modules, and on base in-window f>b pairs, 0 to 4; all 16 patched in-window pairs had none. The kernel rate-limits those messages (NN callbacks suppressed), so treat every FIFO count here as a lower bound, not an exact figure.

What makes this weaker than it looks

  • Not a like-for-like repeat of the Pro 9 procedure. Different capture path, different way of holding the sensor powered. Comparing my numbers to yours directly is not apples to apples.
  • control=on is not literally the autosuspend window. It forbids runtime PM rather than winning a race against the timer. The code path is identical and verified, but it is a constructed condition. On this tree the genuine race is in fact unreachable through the stated method, because of the pm_runtime_put() issue above.
  • Small sample, one unit, one session: 8 pairs per direction per module in-window. Mitigating only in that the outcome was completely deterministic, 0/16 against 16/16 with nothing in between, and the i2c readback gives a mechanical explanation rather than a statistical one.
  • The baseline is a rebuild, not the shipped module. Both variants were built from this machine's DKMS source; the installed module differs from it by one extra ACPI alias and nothing else in the symbol table. Base and patched came from the same source, so the A/B differs only by the patch, but the baseline is not byte-identical to what ships here.
  • The patch applied verbatim, Hunk #1 succeeded at 1171 (offset 195 lines), zero fuzz.

@rylos, if this goes to linux-media, you have a Tested-by from me for the Pro 7+ (Tiger Lake, IPU6 8086:9a19), in-window mode switches both directions. Say the word while the machine is still here and I will run whatever shape you send.

(Written with AI assistance, directed and reviewed by me.)

@rylos

rylos commented Oct 27, 2026

Copy link
Copy Markdown
Author

@loafecho @CmdrShepardsPie the linux-media series is ready: two ov5693 patches on top of the media tree's next, the stream-start mode programming and the full-resolution Bayer order with flips (the hflip XOR vflip window shift, without the Surface Pro 5 DMI parts). I'm running a last test of that exact code on the SP5 before sending.

Once it's out, I'll post the lore.kernel.org link here. Since I don't have an address for you, @loafecho, you can reply on the list from there (lore has a "reply" link with the right headers) with your Tested-by for the SP7, and Chris for the Pro 9.

@rylos

rylos commented Oct 27, 2026

Copy link
Copy Markdown
Author

@dmanresa-saes thank you, that's the clean measurement this needed: the register readback settles it. I've put your 0/16 against 16/16 in the commit message of the stream-start patch, instead of the Pro 9 figures I mentioned above: given what you found about cam with the simple pipeline, the readback is the number I'd rather stand behind. @CmdrShepardsPie, I hope that's fine with you; your Tested-by on the list still covers the Pro 9.

You're right about the stop path too, and it's going in the series as patch 3, with a Reported-by for you. One nuance on the mechanism, from the runtime PM core: pm_runtime_put() does go through autosuspend (rpm_idle() ends in rpm_suspend(dev, RPM_AUTO)), but the delay is counted from last_busy, which is only updated on runtime resume, i.e. at stream start. So at stream stop the delay is measured from when the stream started, not from the stop. That alone doesn't fully explain your 158 to 233 ms suspends with the delay at 5000 and short cam streams, so I'm describing it in the commit only as far as I've checked it in the code. Either way the fix is the same: pm_runtime_put_autosuspend() marks the device busy at stop, so the delay counts from there.

So the series is now three patches, not two. Here is the exact series I'm about to send to linux-media, against the media tree's next (d7e7bca). On a 7.2 tree only the client_info argument of the pad ops differs. If you can run it on the Pro 7+ while you still have it, a Tested-by on the list would be great; I'll post the lore link here once it's out, and you'll be in Cc.

  1. Program the mode registers on stream start (= ac26433)
  2. Keep the Bayer order when flipping at full resolution (the hflip XOR vflip shift, generic, no Surface Pro 5 DMI parts)
  3. Use autosuspend when stopping the stream

Note that patch 2 touches the readout window in ov5693_mode_configure(), so on a tree that carries the #2252 binned-mode patch it won't apply as is; patches 1 and 3 should.

Series (git am)
From 18eec3f7f3c9ad9a7507c8a02dc29abaf71a7b4b Mon Sep 17 00:00:00 2001
From: Marco Ziliani <rylos78@gmail.com>
Date: Thu, 15 Oct 2026 22:04:29 +0200
Subject: [PATCH 1/3] media: i2c: ov5693: Program the mode registers on stream
 start

The mode registers (readout window, binning and subsampling) are only
written from the runtime PM resume handler, and ov5693_set_fmt() merely
stores the requested mode. If the sensor stays powered between two
streams, the new mode never reaches the chip: the sensor keeps
streaming the previous mode while the CSI-2 receiver expects the newly
negotiated format.

This happens whenever the format is changed while the sensor is still
powered. On a Surface Pro 7+ (IPU6), with the sensor held powered
between two streams, OUTPUT_SIZE_X read back over I2C showed the sensor
stuck in the mode it was powered up in, and 0 of 16 streams that asked
for a different mode produced a frame. With this change, 32 of 32
streams completed and the register followed the requested mode.

Program the mode on every stream start, before the controls, so that
the mode registers don't clobber them. This matches what commit
4e1446ac169a ("media: i2c: ov8865: Program the sensor on stream start")
does for the ov8865.

Fixes: 89aef879cb53 ("media: i2c: Add support for ov5693 sensor")
Link: https://github.com/linux-surface/kernel/pull/181
Signed-off-by: Marco Ziliani <rylos78@gmail.com>
---
 drivers/media/i2c/ov5693.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/drivers/media/i2c/ov5693.c b/drivers/media/i2c/ov5693.c
index 56599f9ca..579aa55b7 100644
--- a/drivers/media/i2c/ov5693.c
+++ b/drivers/media/i2c/ov5693.c
@@ -1000,6 +1000,19 @@ static int ov5693_s_stream(struct v4l2_subdev *sd, int enable)
 			return ret;
 
 		mutex_lock(&ov5693->lock);
+
+		/*
+		 * The mode registers are otherwise only written on runtime PM
+		 * resume, so a format change on a sensor that stays powered
+		 * would never reach the chip. Program the mode first, then the
+		 * controls, so the mode registers don't clobber them.
+		 */
+		ret = ov5693_mode_configure(ov5693);
+		if (ret) {
+			mutex_unlock(&ov5693->lock);
+			goto err_power_down;
+		}
+
 		ret = __v4l2_ctrl_handler_setup(&ov5693->ctrls.handler);
 		if (ret) {
 			mutex_unlock(&ov5693->lock);
From 6d3cf2ee5517c853819e7f200080199b738c1cd4 Mon Sep 17 00:00:00 2001
From: Marco Ziliani <rylos78@gmail.com>
Date: Tue, 27 Oct 2026 08:12:37 +0100
Subject: [PATCH 2/3] media: i2c: ov5693: Keep the Bayer order when flipping at
 full resolution

In the full (unbinned) readout each of the horizontal and vertical
flips moves the Bayer phase by one column. With exactly one flip
enabled the CSI-2 receiver gets GBRG instead of the SBGGR10 the driver
reports, which gives a magenta and green image; with both enabled the
two shifts cancel out.

The crop start is only honoured in steps of two pixels, so compensate
by shifting the output window by one column when hflip XOR vflip, and
widen the crop by two columns, within the pixel array, so that the
window still fits. Reprogram the window when either flip changes. The
binned readout is left unchanged.

With this change, raw 2592x1944 frames come out BGGR at all four
hflip/vflip settings on the Surface Pro 5 (IPU3); the same change in
the linux-surface tree gave BGGR at all four settings on a Surface
Pro 7 (IPU4P) and a Surface Pro 9 (IPU6). Without it, the order is only
right with both flips off or both on.

Fixes: 89aef879cb53 ("media: i2c: Add support for ov5693 sensor")
Link: https://github.com/linux-surface/kernel/pull/181
Signed-off-by: Marco Ziliani <rylos78@gmail.com>
---
 drivers/media/i2c/ov5693.c | 56 ++++++++++++++++++++++++++++++++------
 1 file changed, 48 insertions(+), 8 deletions(-)

diff --git a/drivers/media/i2c/ov5693.c b/drivers/media/i2c/ov5693.c
index 579aa55b7..ca0968498 100644
--- a/drivers/media/i2c/ov5693.c
+++ b/drivers/media/i2c/ov5693.c
@@ -361,6 +361,49 @@ static const u8 ov5693_test_pattern_bits[] = {
 
 /* V4L2 Controls Functions */
 
+/*
+ * In the full (unbinned) readout each flip moves the Bayer phase by one
+ * column, so with exactly one of them enabled the CSI-2 receiver gets GBRG
+ * instead of the SBGGR10 the driver reports; with both enabled the two shifts
+ * cancel out. The crop start is only honoured in steps of two pixels, so
+ * compensate by shifting the output window by one column instead, and widen
+ * the crop so that the window still fits.
+ */
+static bool ov5693_bayer_shift(struct ov5693_device *ov5693)
+{
+	const struct ov5693_mode *mode = &ov5693->mode;
+	bool hflip, vflip;
+
+	if (mode->binning_x || mode->binning_y)
+		return false;
+
+	hflip = ov5693->ctrls.hflip && ov5693->ctrls.hflip->val;
+	vflip = ov5693->ctrls.vflip && ov5693->ctrls.vflip->val;
+
+	return hflip ^ vflip;
+}
+
+static int ov5693_window_x_configure(struct ov5693_device *ov5693)
+{
+	const struct ov5693_mode *mode = &ov5693->mode;
+	bool shift = ov5693_bayer_shift(ov5693);
+	u32 crop_end = mode->crop.left + mode->crop.width;
+	int ret = 0;
+
+	/* Widen the crop for the shifted window, within the pixel array */
+	if (shift)
+		crop_end = min(crop_end + 2,
+			       max_t(u32, crop_end, OV5693_NATIVE_WIDTH - 1));
+
+	/* Offset X */
+	cci_write(ov5693->regmap, OV5693_OFFSET_START_X_REG, shift, &ret);
+
+	/* Crop End X */
+	cci_write(ov5693->regmap, OV5693_CROP_END_X_REG, crop_end, &ret);
+
+	return ret;
+}
+
 static int ov5693_flip_vert_configure(struct ov5693_device *ov5693,
 				      bool enable)
 {
@@ -373,7 +416,7 @@ static int ov5693_flip_vert_configure(struct ov5693_device *ov5693,
 	if (ret)
 		return ret;
 
-	return 0;
+	return ov5693_window_x_configure(ov5693);
 }
 
 static int ov5693_flip_horz_configure(struct ov5693_device *ov5693,
@@ -388,7 +431,7 @@ static int ov5693_flip_horz_configure(struct ov5693_device *ov5693,
 	if (ret)
 		return ret;
 
-	return 0;
+	return ov5693_window_x_configure(ov5693);
 }
 
 static int ov5693_get_exposure(struct ov5693_device *ov5693, s32 *value)
@@ -566,16 +609,13 @@ static int ov5693_mode_configure(struct ov5693_device *ov5693)
 	cci_write(ov5693->regmap, OV5693_CROP_START_X_REG, mode->crop.left,
 		  &ret);
 
-	/* Offset X */
-	cci_write(ov5693->regmap, OV5693_OFFSET_START_X_REG, 0, &ret);
-
 	/* Output Size X */
 	cci_write(ov5693->regmap, OV5693_OUTPUT_SIZE_X_REG, mode->format.width,
 		  &ret);
 
-	/* Crop End X */
-	cci_write(ov5693->regmap, OV5693_CROP_END_X_REG,
-		  mode->crop.left + mode->crop.width, &ret);
+	/* Offset X and Crop End X follow the flips, see ov5693_bayer_shift() */
+	if (!ret)
+		ret = ov5693_window_x_configure(ov5693);
 
 	/* Horizontal Total Size */
 	cci_write(ov5693->regmap, OV5693_TIMING_HTS_REG, OV5693_FIXED_PPL,
From 0ad6a26f3533836f7b10b928fde0836d3d6a094c Mon Sep 17 00:00:00 2001
From: Marco Ziliani <rylos78@gmail.com>
Date: Tue, 27 Oct 2026 08:29:18 +0100
Subject: [PATCH 3/3] media: i2c: ov5693: Use autosuspend when stopping the
 stream

The driver enables runtime PM autosuspend with a 1 second delay at
probe, but drops the usage count at stream stop with pm_runtime_put(),
which doesn't update the device's last busy time. The autosuspend
delay is then counted from the runtime resume at stream start rather
than from stream stop, so after any stream longer than the delay the
sensor is powered off right away.

Use pm_runtime_put_autosuspend(), which marks the device busy, so that
the delay counts from stream stop as intended, and a following stream
started within it doesn't power the sensor up from scratch again. With
the mode now programmed on every stream start, keeping the sensor
powered between streams no longer leaves it in a stale mode.

Fixes: 89aef879cb53 ("media: i2c: Add support for ov5693 sensor")
Reported-by: Dario Manresa <d.manresa@electronica-submarina.com>
Closes: https://github.com/linux-surface/kernel/pull/181#issuecomment-6436016335
Signed-off-by: Marco Ziliani <rylos78@gmail.com>
---
 drivers/media/i2c/ov5693.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/i2c/ov5693.c b/drivers/media/i2c/ov5693.c
index ca0968498..ef5e7285c 100644
--- a/drivers/media/i2c/ov5693.c
+++ b/drivers/media/i2c/ov5693.c
@@ -1070,7 +1070,7 @@ static int ov5693_s_stream(struct v4l2_subdev *sd, int enable)
 		goto err_power_down;
 
 	if (!enable)
-		pm_runtime_put(ov5693->dev);
+		pm_runtime_put_autosuspend(ov5693->dev);
 
 	return 0;
 err_power_down:

@CmdrShepardsPie

Copy link
Copy Markdown

@dmanresa-saes the Pro 9 runs the simple pipeline handler with the soft ISP too (cam -l logs Adding camera '\_SB_.PC00.I2C3.CAMF' for pipeline handler simple; Fedora 43's libcamera 0.5.2, unchanged since before the 6.19.8 runs). Here, though, a 1296x972 request asks the sensor for its binned mode: cam adjusts the output to 1292x972 and keeps the 1296x972 sensor mode (Picked 1296x972-SBGGR10_1X10/Unset -> 1296x972-SBGGR10 for max stream size 1292x972), and media-ctl --get-v4l2 on the sensor pad during capture reads SBGGR10_1X10/1296x972, against 2592x1944 for a 2592x1944 request (checked today on 7.3.0-1 with the kernel's own ov5693, same libcamera).

I didn't read OUTPUT_SIZE_X back, so for the chip itself the evidence is indirect: in the 6.19.8 runs on linux-surface/linux-surface#2252, the stream after each in-window switch completed no requests in either direction (0 of 50; cam hung until killed), and adding only ac26433 made it 14 of 14. asym2.sh counts cam's seq: lines, which include empty buffers (bytesused: 0), so those counts show whether requests completed, not whether they carried frames; the 14 took 1.3–1.5 s for 30 frames each with no frame-sync errors.

@rylos that's fine with me. I'll test the series you send on the Pro 9 and reply on the list with the result.

@dmanresa-saes

Copy link
Copy Markdown

@rylos before you send: Simon Kruse sent an ov5693 patch to linux-media this
morning that overlaps your patch 1, and you would rather know now than after.

"media: i2c: ov5693: Use sub-device active state", 2026-10-27 07:49 UTC, same
base commit as yours (d7e7bca), 95 insertions / 141 deletions:
https://patchwork.linuxtv.org/project/linux-media/patch/20261027074933.21921-1-simon0x+git@proton.me/

Where it collides, from its own diff:

  • it drops the ov5693_mode_configure() call from ov5693_sensor_init() and
    adds one in ov5693_s_stream() -- the same change as your patch 1, reached
    from the other end;
  • it changes the signature to ov5693_mode_configure(ov5693, state) and
    derives binning locally (binning_x = format->width < crop->width), deleting
    mode->binning_x and mode->binning_y from the driver struct. Your
    ov5693_bayer_shift() reads both of those, so patch 2 would not compile on
    top of his;
  • it keeps OV5693_OFFSET_START_X_REG written as 0 inside the restructured
    function, which is the line your patch 2 replaces.

It also fixes the same v4l2-compliance control-events failure, through
v4l2_subdev_init_finalize(). Sakari asked Antoine Pichot for the same
active-state conversion on the ov7251 this week, so I would expect that
direction to be the one that lands.

That does not make your work redundant. The Bayer order with flips and the
autosuspend at stream stop are yours alone and are in neither patch; it is
patch 1 that is likely to be superseded and patch 2 that needs rebasing onto
his fields. Note he also has the binned-mode patch pending, which touches the
same function a third time.

I have no stake in who sends first -- I just happen to be talking to both. He
is simon0x+git@proton.me and reads linux-media.

On the PM point, you are right and I was wrong, and I would rather say so
before it reaches a commit message. I checked the core:

  • rpm_idle() ends in return retval ? retval : rpm_suspend(dev, rpmflags | RPM_AUTO); (drivers/base/power/runtime.c), so pm_runtime_put() does go
    through autosuspend, exactly as you said. My "the delay is skipped" was wrong;
  • pm_runtime_mark_last_busy() is called from rpm_resume() on the active
    path, and pm_runtime_put_autosuspend() calls it itself before
    __pm_runtime_put_autosuspend(). That is the whole difference between the two;
  • pm_runtime_autosuspend_expiration() computes last_busy + autosuspend_delay, so with the delay at 5000 ms and a stream shorter than
    that, the suspend should have been deferred rather than immediate.

So the mechanism is right about the code and still does not explain my
measurement. One thing I should flag about my own number: those 158-233 ms are
a delta of power/runtime_suspended_time across the gap, that is accumulated
suspended time, not the latency from stream stop to suspend. They say the
sensor was suspended for most of the gap between the two streams, which with
GAP=0 is just the four media-ctl and v4l2-ctl calls.

I still have the machine. I will measure the actual stop-to-suspend latency
against the stream duration and post the numbers here, so you can put a
verified sentence in the commit or drop the mechanism from it. If the stream
turns out to have run longer than the delay, your explanation covers it
completely.

The Tested-by offer stands for whatever you send, and I will test it against
Simon's patch as well if both are on the list.

@dmanresa-saes

Copy link
Copy Markdown

One small thing on patch 3, while it is still unsent: could you put the
Reported-by as

Reported-by: D. Manresa dmanresa@gmail.com

That is the address on my Signed-off-by and on every Tested-by I have given,
and the trailer outlives both of us in the history. The other one is a work
address that nothing else upstream points at.

@loafecho

Copy link
Copy Markdown

@rylos, here's your 3-patch series on the same Surface Pro 7 (Ice Lake, IPU4P). All three patches pass here, and I'll send a Tested-by on the list once the series is up.

Disclosure: as before, this work was done with help from an AI coding assistant. I ran it on my own Surface Pro 7.

Setup: same as before: Ubuntu 24.04, linux-surface 7.3.0-rc7 with Sakari's IPU7 v4 series, ruslanbay's IPU4P-on-IPU6 RFC series and my SP7 patches. Front OV5693 on CSI-2 port 2.

What I tested: the three patches from your comment above, applied with git am to the stock 7.3 ov5693.c (clean, no fuzz; 0 build warnings). Raw ISYS node, v4l2-ctl, no libcamera. This time every frame was saved and checked buffer by buffer: bytesused from v4l2-ctl --verbose, then the pixel data (an all-zero or constant frame, or one with zeroed rows, doesn't count). I checked that this frame check rejects all-zero, constant and half-zeroed buffers and buffers with bytesused 0, and passes real frames.

Test Your series Unpatched control
Mode switch with the sensor held powered (power/control=on, dmanresa-saes's method): OUTPUT_SIZE_X (0x3808) read over I2C during each stream 16/16 matched the requested width (8 full→binned, 8 binned→full); all 16 streams gave frames stock 7.3: 0x3808 stayed 2592 after each switch to binned; 0 of the 6 streams after the first switch gave a frame
Same switches, sensor runtime-suspended before each stream 10/10 matched (5 + 5), all streams gave frames —
Full res 2592x1944, all 4 hflip/vflip settings SBGGR10 reported and BGGR/RGGB diagonal at all 4, same blue/red plane order each time; profile check: all 3 flipped settings really mirrored; 45/45 buffers with full bytesused and non-constant data at each (this run used my first frame check, which had no zeroed-row test) —
Stop path: sensor runtime_status every 100 ms after a 60-frame stream (autosuspend delay 1 s) active through 1.2 s, suspended at 1.3 s (3 of 3) stock pm_runtime_put() (my installed driver): suspended at the first sample (3 of 3)
Binned 1296x972, 10 module loads × 2 captures 20/20 clean, 900/900 good buffers stock 7.3: 20/20 clean, 900/900 good

No csi2-2 error line in any of these captures.

Two things I noticed along the way:

  • Binned works on plain stock 7.3 here too (last row: fresh module loads, sensor suspended before each capture, so no in-window switch). So my earlier "without the fix: 0/10" row really was the hflip default of my own module, as noted at the time, not the stock binned mode. On this receiver the binned mode doesn't need a fix to stream.
  • First full-res buffer: in the switch runs, every 2592x1944 stream (15 of 15 with your series, and the one that ran on stock) started with a buffer in which more than 1% of the rows were all zero, then 44 good frames. Binned streams didn't show it. It happens with stock too, and with the sensor suspended before the stream as well as held powered, so I don't think it's your series. I haven't looked into it further.

On the stop-path mechanism (dmanresa-saes's last comment): my streams were 60 frames at 2592x1944, about 2 s, so longer than the 1 s delay. With stock pm_runtime_put() the sensor was already suspended at the first sample, taken right after v4l2-ctl exited; with patch 3 it stayed active through 1.2 s. That fits your last_busy explanation for streams longer than the delay. I didn't run streams shorter than the delay, so this doesn't cover dmanresa-saes's short-stream case.

I tested the three patches exactly as posted above, on the 7.3 base. If what goes to the list changes, for example rebased onto Simon Kruse's active-state patch, I'll re-run it before sending a Tested-by.

Not measured: frame rate, colours through libcamera, binned with flips, the ov7251/ov8865 changes. One scene only.

@CmdrShepardsPie

Copy link
Copy Markdown

@rylos I've tested your three patches on the Surface Pro 9 (Intel i7-1255U, IPU6 8086:465d, front ov5693). I applied them with git am to the ov5693.c of my 7.3.0-1 build, which is media next's at d7e7bca apart from the client_info pad-op arguments; git reported line offsets only, no fuzz. The control is the same file rebuilt unchanged, identical code to the installed module.

Method: direct V4L2 on the ISYS node (media-ctl + v4l2-ctl, no libcamera). In the mode-switch and flip runs, OUTPUT_SIZE (0x3808/0x380a) is read over I2C 1.5 s into each stream, and buffers are checked for complete, non-flat 10-bit data. "In-window" means the sensor is held powered with power/control=on, as in dmanresa-saes's run.

Unpatched Series
1: in-window mode switches, 8 pairs each way (32 streams) OUTPUT_SIZE stayed 2592x1944 in all 32; the 16 binned streams returned 0–2 of 180 buffers each, none an image; isys "stream stop time out" in all 16, frame-sync errors in 8 OUTPUT_SIZE matched the request in 32/32, mid-stream and after; every stream delivered image data, no frame-sync errors
Control: sensor suspended before each stream (12 streams) 12/12 streamed, size correct 12/12 streamed, size correct
2: full-res Bayer order at the 4 flip settings, set while suspended and while powered GBRG/GRBG with exactly one flip on BGGR/RGGB at all four in both cases; OFFSET_X 1 and CROP_END_X 2610 read back with one flip on
3: autosuspend_delay_ms=3000, stream stop to suspend, 3 runs each 5 s streams: 1–2 ms; 1 s streams: 2159–2443 ms 5 s streams: 3699–3733 ms; 1 s streams: 3385–3704 ms

FIFO overflow messages and inter-frame packet discards appeared in both builds at similar counts.

On patch 3: the unpatched times fit your last_busy explanation, the 3 s delay (plus timer slack) counting from the resume at stream start, so it had already run out after the 5 s streams. With the series, the extra 0.4–0.7 s over 3000 ms is consistent with the 25% slack rpm_suspend() gives the autosuspend timer (750 ms here). Times are measured from when a 50 ms poll saw v4l2-ctl exit, so they may read up to ~50 ms low.

Not tested: libcamera/cam, Simon Kruse's active-state patch, and binned mode with flips.

These results are for the series as posted; as I said above, I'll test what reaches the list and reply there.

@rylos

rylos commented Oct 27, 2026

Copy link
Copy Markdown
Author

The series is on the list:
https://lore.kernel.org/linux-media/20261027-ov5693-fixes-v1-0-3c8ab1ae104d@gmail.com/

The code is byte for byte what you tested from my comment above; only the commit messages changed. They now carry the Pro 7+ and Pro 9 numbers from this thread plus an A/B run on the SP5 (IPU3), against media next stock:

SP5 next next + series
Mode switches with the sensor held powered (2592x1944 ↔ 1920x1080 crop), 16 pairs OUTPUT_SIZE_X stuck at 2592, 0 of 16 streams in the new size gave a frame OUTPUT_SIZE_X followed the request, 32/32 streams 10/10 frames
Full-res Bayer order, 4 flip settings GBRG with exactly one flip BGGR at all four
Stop to suspend after a 2 s stream, 1 s delay 0 ms ~1.1 s

No CSI-2 errors in either run.

@dmanresa-saes the Reported-by on patch 3 uses dmanresa@gmail.com as you asked, and you're in Cc there. Simon Kruse is in Cc too, and the cover letter points at his active-state patch.

@CmdrShepardsPie @loafecho since it's the code you already ran, a Tested-by as a reply on the list is all it needs. @loafecho, lore's "reply" link at the bottom of each message gives the git send-email command or the headers to reply with any mail client (plain text only; vger drops HTML mail). Replying to the cover letter covers the whole series.

Thanks all three of you: the series is a lot stronger for your measurements.

@Simon0x

Simon0x commented Oct 28, 2026

Copy link
Copy Markdown

Hi all, Simon Kruse here (the active-state patch on linux-media). @dmanresa-saes, thanks for pointing out the overlap.

Plan, also posted on the list as a reply to the cover letter:

  • @rylos, your 1/3 is the same change as the mode move in my patch, and it stands better on its own, so I'm dropping that part from mine.
  • I'll rebase v2 of the active-state conversion on top of your whole series rather than the other way around, so your series can go in as it is. The conversion will then also move ov5693_window_x_configure() / ov5693_bayer_shift() from ov5693->mode to the active state.
  • v2 keeps the driver mutex as the state lock, as Dan asked in his review, and its cover letter will say it depends on your series.
  • I'll run your series on my Surface Pro 9 and reply on the list with the result.

I'll post here when v2 is on the list. It adds the conversion on top of whatever version of Marco's series is current then.

Disclosure: like the others here, I work on this with an AI coding assistant (the patch carries an Assisted-by: tag). I run the tests on my own Surface Pro 9.

@technosf

technosf commented Nov 4, 2026

Copy link
Copy Markdown

Tested on a Surface Pro 4 SP4 (DMI SKU Surface_Pro_4, i5-6300U, IPU3; front OV5693 = ACPI INT33BE), stock linux-surface kernel 7.2.15-1-surface, Secure Boot on.

Build: ov5693.c and ov8865.c from the 7.2.15 kernel source, with this PR's 181.patch applied (all 10 patches, applied cleanly, no offset or fuzz), built as out-of-tree modules for 7.2.15-1-surface and installed in updates/ signed with the machine's own key. Before patching I built the unpatched ov5693.c the same way; its srcversion matched the distribution module (29B7F301…), so the source is the same. The stock ipu3-cio2 receiver was used unchanged. The rear and IR cameras were not tested.

Test: fresh boot, front camera only, raw capture of 5 frames per step through the CIO2 video node with v4l2-ctl, no gap between steps so the sensor stayed powered (autosuspend is 1 s): 1296x972, 2592x1944, 1296x972, 2592x1944.

Without the patches (stock 7.2.15): all four steps returned buffers of the expected size, but the kernel logged ten times ipu3-cio2 0000:00:14.3: payload length is 6469632, received 1617408 (two bursts of five, matching the two full-resolution steps). The full-resolution buffers were empty after the first 1,617,408 bytes: the sensor kept sending the binned mode after the format change.

With this PR: no payload length messages and nothing else from ipu3-cio2 or ov5693 in dmesg during the run. All four quarters of the full-resolution buffers contain pixel data in both full-resolution steps.

Caveats: the images are close to saturation (mean about 1000 of 1023) because the capture used default exposure with no AEC, so I make no claim about image quality or colour. One run each.

Orientation (front camera): the firmware reports camera_sensor_rotation = 180 for the front OV5693 (INT33BE). The raw frames, viewed unprocessed, are both upside down and mirrored, which looks like a plain vertical flip and not a 180° rotation (checked by eye on a live scene; I did not test with a reference pattern). If that is right, this is the same situation as the Pro 5: applying the reported 180° leaves the final image mirrored. The Pro 5 DMI entries in this PR (the ipu-bridge "not rotated" list and the VFLIP inversion in ov5693) do not match Surface_Pro_4, so the Pro 4 would need Surface_Pro_4 added to the same lists for the front camera. I have not checked the rear OV8865 on the Pro 4.

The test scripts and this summary were prepared with an AI coding assistant (Claude); I ran the tests on my Surface Pro 4.

@technosf

technosf commented Nov 4, 2026

Copy link
Copy Markdown

Tested on a Surface Pro 8 SP8 (IPU6; front OV5693 = ACPI INT33BE), kernel 7.2.15-1-surface, with the same build as in my Surface Pro 4 comment above (ov5693 and ov8865 from the 7.2.15 source plus this PR, loaded from updates/). No clock_ncont option exists in this build, and none was set.

Raw capture of 5 frames per step, no gap between steps: 1296x972, 2592x1944, 1296x972, 2592x1944, after a reboot, twice, and once more with a 3 s gap between steps so that the sensor suspended in between. All three runs delivered 5 frames of the requested size on all four steps, with pixel data (images near saturation, default exposure). So with this PR, which writes MIPI_CTRL00 = 0x2d on every stream start, the front camera streams in both modes on the Pro 8 without any clock-lane option.

Things I saw: the receiver logs csi2-4 Transfer FIFO overflow and inter-frame short/long packet errors at stream start in every run (10–20 lines) and the first frame of one or two steps is black (mean at the black level); in the first run there was also one stream stop time out and one stream close time out (about 2 s each), which did not recur in the second run. I did not run the stock 7.2.15 module on this machine, so I have no baseline here.

This overlaps with #180, which sets only bit 5 of the same register when clock_ncont is on: both change ov5693_enable_streaming().

The test scripts and this summary were prepared with an AI coding assistant (Claude); I ran the tests on my Surface Pro 8.

@rylos

rylos commented Nov 4, 2026

Copy link
Copy Markdown
Author

@technosf Thanks a lot for both runs, the Pro 4 one in particular: it is the first IPU3 data point besides my Pro 5, and the payload length messages are a clear before/after.

Pro 4 orientation. "Upside down and mirrored" in the raw frames is what a real 180° rotation looks like, and the SSDB says 180°, so the firmware may well be right on the Pro 4. On the Pro 5 the raw frames were upside down without the mirroring, which is why it needs the DMI quirk. Could you check it end to end? With this PR as it is (no Surface_Pro_4 entry), open the front camera in qcam (or cam -C and save a frame) and hold some printed text in front of it: readable text → no quirk needed; mirrored text → I'll add Surface_Pro_4 to both lists. The same check on the rear OV8865 would settle that one too.

MIPI_CTRL00. Right, the 0x2d write comes from @dmanresa-saes' binned-mode commit and overlaps with #180. As agreed in linux-surface/linux-surface#2252, once #180 is in I'll drop the write from this PR, so that the clock-lane setup lives in one place. Your Pro 8 run is useful there too: it shows the two approaches give the same result on that machine.

@technosf

technosf commented Nov 5, 2026

Copy link
Copy Markdown

I ran the end-to-end check on the Pro 4 for both cameras (stock modules, so "the PR as it is": its DMI entries don't match Surface_Pro_4). Both sensors report camera_sensor_rotation 180. I held a phone showing a large letter F in front of each camera with the exposure raised or lowered as needed, and converted a raw frame from the CIO2 node to a picture.

Front (OV5693):

  • Raw frame: the F is upside down with the stem on the left and the arms at the bottom, i.e. a vertical flip only.
  • Rotated by 180° (what libcamera applies for rotation 180): the stem is on the right and the arms at the top, i.e. a mirrored F.

Rear (OV8865, 3264x2448):

  • Same result: the raw frame is a vertical flip only, and after the 180° rotation it is a mirrored F.
  • The capture itself was clean (full byte count, no CIO2 payload-length errors, stock ov8865 on 7.2.15-1-surface). The exposure control had to be raised well above its default of 32 to get a usable image.

So, as on the Pro 5, the reported 180° leaves the final image mirrored on both cameras, and the Pro 4 needs the same treatment: Surface_Pro_4 added to the ipu-bridge "not rotated" list (for INT33BE and the OV8865 entry) and to the VFLIP inversion list of each sensor driver (ov5693 and ov8865).

rylos added a commit to rylos/kernel that referenced this pull request Nov 5, 2026
The Surface Pro 4 (IPU3, DMI SKU "Surface_Pro_4") has the same
cameras as the Pro 5 and the same mounting: the SSDB reports a 180
degree rotation for the front OV5693 and the rear OV8865, but both
sensors only need a vertical flip. Applying the reported rotation
leaves both images mirrored.

Add the Pro 4 to the ipu-bridge list of sensors that are not rotated
and to the VFLIP inversion lists of ov5693 and ov8865, as for the
Pro 5.

Checked on a Surface Pro 4 with a letter held in front of each camera:
the raw frames of both sensors show a vertical flip only, and after
the reported 180 degree rotation the letter is mirrored.

Reported-by: technosf <technosf@hit-on.com>
Link: linux-surface#181 (comment)
Signed-off-by: Marco Ziliani <rylos78@gmail.com>
Patchset: cameras
@rylos

rylos commented Nov 5, 2026

Copy link
Copy Markdown
Author

@technosf Thanks, that settles it: same mounting as the Pro 5 on both cameras.

Pushed as a separate commit, bb48b22 ("media: Surface Pro 4: same camera orientation quirks as the Pro 5"): Surface_Pro_4 is now in the ipu-bridge "not rotated" list for INT33BE and INT347A, and in the VFLIP inversion lists of ov5693 and ov8865. I left it out of the ov8865 30 fps default, since nobody has run libcamera on the Pro 4's rear camera yet. The three files build without warnings (W=1).

If you get the chance, a run with this PR on the Pro 4 through libcamera (qcam, or cam -C and a saved frame) with the same letter test would confirm the final image is upright and not mirrored on both cameras.

@technosf

technosf commented Nov 5, 2026

Copy link
Copy Markdown

Patch tested with libcamera-ipa, cam -c on SP4.

Before (same machine, without bb48b22): the F was upright but mirrored (back-to-front) on both the front and the rear camera, as expected from the raw frames.

After (with bb48b22): the F is upright and not mirrored on both the front (OV5693) and the rear (OV8865) camera. I.e. It is correct

Thank you @rylos

@rylos

rylos commented Nov 5, 2026

Copy link
Copy Markdown
Author

Thanks @technosf, that settles it for the Pro 4: upright and unmirrored on both cameras through libcamera.

I'll add your Tested-by to bb48b22 the next time I rework this branch, together with dropping the 0x4800 write once #180 is in.

The Surface Pro 4 (IPU3, DMI SKU "Surface_Pro_4") has the same
cameras as the Pro 5 and the same mounting: the SSDB reports a 180
degree rotation for the front OV5693 and the rear OV8865, but both
sensors only need a vertical flip. Applying the reported rotation
leaves both images mirrored.

Add the Pro 4 to the ipu-bridge list of sensors that are not rotated
and to the VFLIP inversion lists of ov5693 and ov8865, as for the
Pro 5.

Checked on a Surface Pro 4 with a letter held in front of each camera:
the raw frames of both sensors show a vertical flip only, and after
the reported 180 degree rotation the letter is mirrored.

Reported-by: technosf <technosf@hit-on.com>
Link: linux-surface#181 (comment)
Tested-by: technosf <technosf@hit-on.com> # Surface Pro 4
Signed-off-by: Marco Ziliani <rylos78@gmail.com>
Patchset: cameras
@rylos

rylos commented Nov 5, 2026

Copy link
Copy Markdown
Author

Added @technosf's Tested-by now rather than waiting for #180, so it isn't lost if this goes in first. Only the commit message changed: bb48b22 is now 8e38339, same tree. The 0x4800 write stays until #180 is in.

namastecoconut-imagined pushed a commit to namastecoconut-imagined/room-linux that referenced this pull request Nov 12, 2026
On the Surface Pro 5/6 the IR lights hang off the OV7251's STROBE pin, which
the driver leaves off (linux-surface/kernel#181, not merged). "lit" writes
the same sensor registers over I2C while the camera streams; the sensor
forgets them when the stream stops. The lights flash every other frame, so
the photo shows lit, unlit, and lit minus unlit. All photos are now turned
a quarter anticlockwise, the way the sensor is mounted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XWRVQj81Vw5DNLETcaUeEh
@T-Caster

Copy link
Copy Markdown

Some results from a Surface Pro 7+ (IPU6 Tiger Lake 8086:9a19), in case IPU6 data helps here. Base is linux-surface 7.3.4 plus the clock-lane gate backport from #193.

ov5693: Program the mode registers on stream start (e753f13): cherry-picked cleanly. With it I can switch between 2592x1944 and 1296x972 back to back with no pause between the captures (raw capture from the ISYS node, four switches in a row) and every capture delivers frames with image content.

ov5693 binned mode: I did not use the commit in this PR but D. Manresa's later variant of the same patch, ov5693-binned-no-mipictrl.patch from his surface-ipu6-cameras repository, which has no 0x4800 write and is meant for trees that carry the upstream gate. He noted it had not been run on hardware in that configuration, so for the record, on the Pro 7+:

1296x972, flips before with the patch
hflip=0 vflip=0 streams streams
hflip=1 vflip=0 no frames, Frame sync error streams
hflip=0 vflip=1 no frames, Frame sync error streams
hflip=1 vflip=1 no frames, Frame sync error streams

The sensor's colour bars come out clean and in the right colours with flips 0/0 and 1/1, so the Bayer phase stays BGGR. Through libcamera and PipeWire the 640x480 and 1280x720 streams, which use this mode with both flips set, went from black to a normal picture. That may be useful when the 0x2d write in this PR's version is reconciled with #193.

ov8865: Allow fine analogue gain steps (7dc915f): cherry-picked cleanly. With the step of 128, libcamera's software ISP got stuck at a low gain in a dark room (exposure at maximum, gain 256 of 2048). With a step of 1 the gain ramps up to the maximum as expected.

ov8865: Fix the VTS high byte mask (7b2ab04): cherry-picked cleanly and the rear camera streams normally with it, but I did not exercise frame lengths above 4095 lines, so I cannot confirm the fix itself.

I have not tested the ov7251 commits from this PR. For the IR camera I used Antoine Pichot's v3 series from linux-media instead, which works on this machine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants