Skip to content

hid: ipts: Bound the stop path and free the context by reference - #187

Open
loafecho wants to merge 9 commits into
linux-surface:v7.3-surface-develfrom
loafecho:ipts-bound-stop-path
Open

loafecho wants to merge 9 commits into
linux-surface:v7.3-surface-develfrom
loafecho:ipts-bound-stop-path

Conversation

@loafecho

@loafecho loafecho commented Nov 6, 2026

Copy link
Copy Markdown

The first commit here is #178 itself, unchanged and with its author kept, because this series depends on it and it is not merged yet. It drops out of this PR once #178 lands.

The IPTS driver's stop path had no bound: it waited for the receiver
thread with wait_for_completion(), and a thread that died in an oops or
slept forever on a stuck message lock hung every later stop, and with it
reboot and suspend, since ipts_remove() runs on both. When a stop failed
it also unloaded with its HID device still registered, so the next open
of its hidraw node ran freed module code. And the context was allocated
with devm, so any stop that gave up turned into a use-after-free.

This series reworks the stop path on top of linux-surface/kernel #178:

  • the context is reference-counted first, so the later patches can keep
    what a stuck task still uses without devres freeing it, and it holds
    the MEI device for as long as it lives;
  • every wait in the driver is bounded; a GET_FEATURE request that waits
    for a reply is woken on stop, and the other waiters run to their own
    timeouts;
  • the message-list lock is held only around a list add or delete, never
    across a sleep;
  • a thread that died (PF_EXITING) counts as stopped;
  • the HID device is destroyed once no HID callback and no HID input of
    the receiver is running, even if the receiver thread does not stop;
  • the DMA buffers are freed only once the device has confirmed its stop
    and no HID request outlived the stop's wait for it; otherwise they are
    leaked, and an EDS1 device is not restarted;
  • remove waits for the EDS1 restart work only up to a bound derived from
    the bounds of a start and a stop.

AI-assisted: Claude Code found the bugs, wrote these patches and their
messages, and ran the plain device tests. OpenCode wrote the test-only
fault-injection builds and ran those tests. AI models from several labs
reviewed the series. I directed the work on my own device and take
responsibility for it.

Only one device is available for testing, a Surface Pro 7 (EDS2, poll
mode) on Ubuntu 24.04, running linux-surface 7.3.0-rc7 with unrelated
camera patches. The EDS1 paths, including the mode-switch restart, are
untested. It needs linux-surface/kernel #178 applied first. Every patch
builds on its own and checkpatch --strict is clean.

Tested on that Surface Pro 7 with the final series. The touch check
passed at the end of every test run:

  • 200 module loads while four processes sent HID requests;
  • 10 reboots and 50 suspend/resume cycles;
  • with test-only fault-injection builds (not part of this series) that
    make the stop path stall or fail in five ways, 231 rmmod runs and 9
    reboots: no rmmod and no reboot hung, and the 225 runs that checked
    the kernel log found no oops;
  • in 20 of those runs another task held the message lock during rmmod,
    and a use-after-free detector in the test build never fired. On an
    earlier, unfixed version of this rework the same test fired it in 8
    of 12 runs;
  • with a test build that keeps the device from confirming its stop, 3
    rmmod runs took 5-6 s (about 0.15 s without it) and leaked the DMA
    buffers as designed, with no oops, and touch came back after each
    reload.

The test patches and scripts are available on request.

Known limits:

  • rmmod while a task is stuck forever in driver code is out of scope:
    a receiver thread that never returns, a HID callback that never
    returns or dies, a task that holds the message lock forever, or an
    EDS1 restart that never finishes. The driver frees none of its own
    memory that such a task can still use, the context keeps the MEI
    device for it, and reboot and suspend still finish. But after rmmod, a
    receiver thread that wakes up runs freed module code. A HID callback
    that never returned keeps the HID device registered, because
    destroying it could block on HID core locks the stuck task holds, so
    opening its hidraw node after rmmod runs freed module code too.
    Closing this needs module pinning, which this series does not do.
  • After a stop at remove gives up on the receiver thread, the context is
    never freed, even if the thread exits later: nothing stops it again.
  • Remove waits, bounded, for the other users of the context to return.
    That narrows the window in which a user still runs module code after
    rmmod, but does not close it: the last user wakes remove from inside
    its own put.
  • A send is bounded while it waits for the write to complete, but
    mei_cldev_send_timeout() can still wait without a bound for the MEI
    bus lock, and for room in a full transmit queue.
  • If the MEI receive work blocks while it holds the message lock, remove
    hangs in the MEI core: mei_cldev_disable() waits for that work with
    cancel_work_sync(), and the driver cannot bound that wait.
  • A device that never confirms its stop leaks one set of DMA buffers per
    stop, on every suspend and every restart. An EDS1 restart then does
    not start the device again, so touch stays off until the next probe.
    A HID request that outlives the stop's wait for it leaks one set too,
    with the same effect on an EDS1 restart.

Josh Spaulding and others added 9 commits November 6, 2026 00:00
ipts_mei_search() looks up a matching response under the read side of
message_lock, drops the lock, and then re-acquires it for writing to
unlink the entry. Two concurrent callers waiting for the same command
code (typically the receiver poll thread and a userspace hidraw raw
request, both inside ipts_control_send_feedback()) can each find the
same entry during their read-locked walk and then both unlink and free
it. The second list_del() hits LIST_POISON and, with CONFIG_DEBUG_LIST,
oopses the poll kthread:

  list_del corruption, ...->next is LIST_POISON1 (dead000000000100)
  kernel BUG at lib/list_debug.c:56!
  RIP: __list_del_entry_valid_or_report.cold
  Call Trace:
   ipts_mei_search+0x83/0x150 [ipts]
   ipts_mei_recv+0x61/0x190 [ipts]
   ipts_cmd_recv_timeout+0x2f/0x90 [ipts]
   ipts_control_send_feedback+0x81/0xc0 [ipts]
   ipts_receiver_poll_loop.cold+0x89/0x11c [ipts]
   ipts_thread_runner+0x21/0x40 [ipts]

Without CONFIG_DEBUG_LIST the same race shows up as a general
protection fault on 0xdead000000000108 in ipts_mei_search(). The other
caller is then left blocked in D state inside the driver, which makes
every later suspend fail with "tasks refusing to freeze". Seen on
resume from s2idle on a Surface Pro 7 while iptsd was (re)started.

Hold the write lock across the search and the unlink so that a message
can only be claimed by one caller.

Signed-off-by: Josh Spaulding <git@spaulding.app>
The context is allocated with devm_kzalloc(), so devres frees it when
the device is released, whoever still uses it. The following changes
bound the stop path, and a stop that gives up must keep everything that
a receiver thread or a HID device which is still there can touch. With a
devm context, devres frees it anyway, and the next touch is a
use-after-free.

Reference-count the context instead, and free it only when the last
reference is dropped:

- Probe holds one reference, dropped by a devres action. Remove and
  every later probe failure disable the MEI device before that, which
  cancels its receive callback, so no message arrives while the context
  is freed. If adding the devres action fails, the context is freed at
  once, before the MEI device is disabled; the receive callback finds no
  context then, because drvdata is not set yet.
- The receiver thread and the HID device hold one each, for their whole
  life.
- The context holds a reference to the MEI device until it is freed: a
  user that is still there after remove keeps sending through it and
  logging against it.

ipts_remove() then waits, bounded, for the remaining users to return.
That narrows the window in which the module code is freed under one
that is about to return, but does not close it: the last user wakes
remove from inside its own put, before it has left the module code.
After a failed stop the receiver thread or the HID device is still there
on purpose, and remove does not wait for it.

A failed probe used to return without disabling the MEI device, so its
receive callback stayed registered after the context was gone. Every
probe failure after mei_cldev_enable() now disables it.

The messages are now plain kzalloc() instead of devm: they are freed on
the error paths of the receive callback, and any left over is freed with
the context.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
ipts_remove() runs on rmmod, and also on every reboot, power-off and
suspend: mei_me_shutdown() and mei_me_pci_suspend() both call
mei_stop(), which removes every ME client device. ipts_thread_stop()
waited for the receiver thread with wait_for_completion() and no
timeout. If the thread died in an oops, or never got out of a wait, the
stop never returned, and reboot or suspend hung with it.

Wait for the thread with a timeout instead, which the caller passes in.
While waiting, check whether the thread is exiting: a thread that died
runs no driver code anymore and counts as stopped, without calling
kthread_stop() on it, which could wait for its exit forever. A thread
that is still running after the timeout is reported to the caller, which
then frees none of the driver's memory that the thread can still touch:
the receiver keeps the thread's context reference, so the context stays,
and the context keeps the MEI device. Only a later stop drops that
reference. After remove there is none, so the context then stays for
good.

Until later changes in this series remove the HID device on those
paths, a stop that gives up here, or finds the thread dead, still
returns before the HID device is removed: rmmod with a stuck or dead
receiver then leaves the HID device registered, and opening its hidraw
node runs freed module code. Before this change, that rmmod hung for
good instead.

Keep a reference on the task_struct from creation until the stop is
done, so that a thread which exits on its own early can still be looked
at, and never store a kthread_create() error as the thread. The return
value of the thread function moves to thread->exit_code; the receiver
reports it as before.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
Every command waits for its reply in ipts_mei_recv(), which slept in
wait_event() with ipts_mei_search() as the wait condition. The search
takes message_lock, so a waiter blocked on the lock inside the wait
condition, and a task that held the lock without releasing it (one that
died while holding it) blocked every later command for good: the
receiver thread, the HID requests, and the shutdown handshake on the
stop path.

The lock only ever covers a list add or delete, so nobody has a reason
to hold it for long. Never block on it:

- ipts_mei_search() only tries the lock, and reports a busy lock to the
  caller.
- ipts_mei_recv() searches with the task running and sleeps until a new
  message is queued (a sequence counter bumped under the lock) or the
  time is up. While the lock is busy, it tries again after a short
  sleep. The time limit covers both.
- ipts_mei_incoming() gives up after a bounded wait for the lock and
  drops the response.
- Sends use mei_cldev_send_timeout(). An interrupted send is not
  retried, because the message is already queued and would go out
  twice. The timeout bounds the wait for the write to complete; the
  MEI core can still wait without a bound for its bus lock, and for
  room in a full transmit queue, which the driver cannot change.

Holding the lock also holds a context reference, taken and dropped by
the new lock helpers, so the context is never freed under a holder that
never lets go. The lock is released before the reference, because
dropping the last reference frees the context, and the release function
takes the lock itself to free the messages.

The "negative timeout waits forever" case could never be reached with a
u64 timeout and is gone; a zero timeout still checks once.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
When the receiver thread returned an error from its shutdown handshake,
_ipts_control_stop() and ipts_control_stop() returned early and never
called ipts_hid_free(). The module then unloaded with its hid_device
still registered, and the next open or ioctl of its hidraw node ran
freed module code: a page fault in hidraw_open() or inside
__hid_hw_raw_request(), with "[last unloaded: ipts]" in the oops.

Remove the HID device on every path once the receiver thread is gone.
The only exception is a receiver thread that is still running after the
bounded stop: it still feeds the HID device and uses the buffers, so
nothing is freed and the error is passed on.

Until a later change in this series counts the receiver's HID input,
that exception still lets rmmod with a stuck receiver leave the HID
device registered, with the hidraw problem above. And a receiver that
died inside hid_input_report() may hold HID core locks, so destroying
the HID device after it can block; the same later change makes the stop
wait for running HID input first, and keep the HID device in that case.

If the thread died, or the device did not confirm the stop, the device
may still write to its DMA buffers. Leak them instead of freeing them;
the next start allocates new ones.

The stop now reads the thread's exit code itself. A stop that finds no
thread, because a start failed before it started one, reports a clean
exit instead of the exit code of the thread before.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
…firmation

Once a stop is requested, the receiver loops still finished every buffer
of the current round before they looked at the stop flag again, and the
poll loop walked through every doorbell value it had missed, however far
the doorbell had moved. Check the stop flag inside the inner loops too,
and never walk back further than the buffers that can hold new data.

The poll loop sent QUIESCE_IO only when it saw the stop flag at the top
of a round. A stop requested while it handled buffers ended the loop at
the bottom check without sending it, and the device was never told to
stop. Send it after the loop instead, as the event loop does, and then
give back the buffers the loop did not get to, as before: in poll mode
the device confirms the stop only once every filled buffer was given
back.

At the end of a loop, the stop handshake gave up on the QUIESCE_IO
confirmation as soon as the last READY_FOR_DATA reply was missing, and
reported a missing reply as success. Starting the device again before it
has stopped can leave its firmware unresponsive until the next boot, and
freeing buffers it still writes to corrupts memory. Collect the last
READY_FOR_DATA reply, then wait for the QUIESCE_IO confirmation for a
bounded time in any case, and return success only when the device
confirmed the stop. Otherwise the stop keeps the DMA buffers.

That is a trade-off: a device that never confirms its stop leaks one set
of DMA buffers per stop, and that repeats on every suspend and every
restart, with a warning each time. Memory the device may still write to
is never freed.

Every step after the stop request is bounded now, apart from the waits
inside the MEI core that a send can still make, so the time a stop
waits for the receiver thread is the sum of those bounds instead of a
guess. A slow device then still gets its stop confirmed, instead of
leaving a thread behind that keeps the buffers. The bounds move to
receiver.h, so that the waits of the stop path can be derived from them
too.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
On EDS1 devices, a HID request that switches the mode restarts the
device from inside the HID callback, and nothing serialized that restart
with ipts_remove(): both could stop the receiver thread and free the
buffers at the same time. A stop that waits for running HID callbacks,
as the next change makes it do, would also wait for the very callback
that called it.

Run the restart from a work item instead, under a mutex that remove
takes for its own stop. A queued restart holds a context reference until
it has run. Remove marks the context as going away first, so no new
restart is scheduled, and a restart that is already running checks that
mark again before it starts the device.

Remove never waits for the work without a bound: the worker can be the
task that is stuck, or that died with the mutex held, and reboot and
suspend must still finish. It cancels a queued restart without waiting,
before and after its stop, and takes the mutex with a bounded wait. The
bound is derived from the bounds of a start and a stop, the way the
receiver's stop timeout is: it covers whatever a running restart can
still do once remove has marked the context. If a restart holds the
mutex past that bound, it is stuck, and remove skips the stop and frees
nothing that restart can still use: the restart holds a context
reference, and the context holds the MEI device.

A restart whose stop was not confirmed by the device, so that the DMA
buffers were leaked, no longer starts the device again: starting a
device that has not stopped can leave its firmware unresponsive until
the next boot. The stop reports that case with a return value of its
own. Touch input then stays off until the driver is probed again.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
Stopping only cleared hid_active, which new HID callbacks check on
entry. A callback that had already passed that check kept running while
the stop freed the buffers it used: a SET_FEATURE request writing the
hid2me buffer, or a GET_FEATURE request waiting for a reply in a buffer
that was about to go away.

Count the callbacks that run driver code, and make the stop wait for the
count to drop to zero, with a timeout. A pending GET_FEATURE request is
released at once instead of waiting for its own timeout, and one that
starts while the stop runs returns at once.

The receiver thread's HID input is counted the same way. Once the count
has dropped to zero, the receiver can no longer reach the HID device,
so the HID device is destroyed even when the receiver thread does not
stop. Before, a receiver that did not stop kept the HID device
registered, and after rmmod its hidraw node ran freed module code.

The receiver thread is asked to stop in any case. A callback that
returns while it stops was only slow, and the stop goes on as usual,
except that the buffers are leaked instead of freed: the callback may
have sent the device a command after the device confirmed its stop, and
the device may answer it by writing to them. A callback that still runs
after the timeout and the receiver stop is most likely stuck for good,
in a task that died inside it, possibly with HID core locks held. That
includes a receiver that died inside hid_input_report(). It can still
touch the buffers, and destroying the HID device could block forever on
those locks, so in that case nothing is freed and the error is passed
on. A later stop then leaks the buffers instead of freeing them,
although it finds no receiver thread anymore.

So rmmod after a HID callback died, or while one never returns, still
leaves the HID device registered, and its hidraw node runs freed module
code. Destroying it would trade that for a remove that can block, and
with it reboot and suspend.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
When a step of ipts_control_start() failed, it returned at once and left
behind whatever the earlier steps had set up: the buffers, and a
receiver thread that kept running against a device that was never told
to stop. An allocation error from hid_allocate_device() was even kept in
ipts->hid, where later code took it for a device.

Tear down on failure what the start set up. Buffers the device was
never given are freed. Buffers it may have been given are leaked, since
it was never told to stop: that covers a GET_DESCRIPTOR request that
got no reply, because the device may still write the descriptor. Once
the receiver thread runs, the normal stop path stops it, and keeps what
a thread that does not stop can still use. On an EDS1 restart, that stop
also removes the HID device that probe created. A HID device that failed
to register is no longer destroyed in ipts_hid_init(), while the
receiver may already use it, but by that stop, once HID input has
stopped.

The receiver thread already runs while ipts_hid_init() sets up the HID
device, so the device is now set up in a local variable and published
in ipts->hid only once it is complete. The receiver never sees an error
pointer or a half set-up device.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
@loafecho

loafecho commented Nov 7, 2026

Copy link
Copy Markdown
Author

@antoniovazquezblanco thanks for running #178 on the SP4.

Saw you closed intel-precise-touch#35 in favour of this PR. The code does cover it, ipts_thread_start() holds a reference on the task from creation until the stop is done and it doesn't leave an error pointer behind, same two changes as yours. That's in 224f261. But I only have a Surface Pro 7, so none of this has ever run on EDS1, which is where you actually hit that use-after-free. So I can say it should be fixed, I can't say it is.

Would you be able to try this branch on your SP4? Mainly I'd want to know if the refcount warnings from your trace are gone when iptsd switches modes. That restart runs from a work item now, so more changed on that path than just the reference. Also worth watching dmesg for "Device did not confirm the stop" or "not restarting IPTS". If the device takes longer than about 3 s to confirm a stop, touch stays off until the next probe, and I honestly don't know how often that happens on EDS1.

You also might not want to drop yours because of this. Yours is one file and you tested it on the hardware that hits the bug, this is 9 commits and could sit for a while. If you reopen it against v7.3-surface-devel and it lands first I'll just rebase on it, no problem. And since you reported this back in August, lmk if you want a Reported-by on that commit, I'd just need the name and address you want on it.

One more thing, I saw my comment on #178 got linked from a lot of the intel-precise-touch threads. That was only my read from the commit history, not something a maintainer told me, so qzed or StollD would be the ones to say if that repo should be archived.

@jaydenmilne

Copy link
Copy Markdown

Looks like my claude ended up duplicating some of this in #188 - so that's independent confirmation I guess. I've been testing on a Surface Book 1 / ESD, so I'll pull your branch here and see how it goes. I've got three more fixes that aren't in your branch though, how would you feel about including those with your PR? I can also do a followup if you think that better, but I'd rather test the full patch set rather than a subset.

The three additional fixes are:

  1. Drop stale responses at start (sometimes the firmware responds late, after the driver has given up. the old response stays queued up and then causes chaos later on)
  2. Restart after a failed mode switch (switching to multitouch on this device means restarting the touch device, which happens on every resume. sometimes the restart fails, so we retry a few times instead of leaving touch dead)
  3. Reconnect to the management engine before each retry (after a timeout, the ME gets out of sync and gets stuck sending one reply behind (the reply only comes after the next command is sent). So we reconnect to the ME which clears whatever broken state is happening

@loafecho

loafecho commented Nov 9, 2026

Copy link
Copy Markdown
Author

@jaydenmilne nice, that's good confirmation, especially the "one reply behind" part, because I've been chasing the same thing on my SP7 (EDS2) and found at least one way it gets there.

FEEDBACK replies are matched to their sender by command code only. The receiver loop (giving back data buffers) and HID feature requests (through the one shared hid2me buffer, and SET_FEATURE isn't locked at all) can have FEEDBACKs in flight at the same time. In my traced runs, once three or more were in flight at once, a sender took another one's reply and returned early, and from then on the ME stayed one reply behind, only letting the newest reply out when the next command arrived. A QUIESCE_IO sent in that state never gets confirmed. I traced it with bpftrace on the MEI send/recv calls: every unconfirmed stop had exactly one reply owed when QUIESCE_IO went out, every confirmed stop had none, and in 25 of 25 idle gaps the owed reply never came out on its own.

My fix serializes each FEEDBACK with its reply under one lock (which also covers the hid2me buffer) and drops a FEEDBACK reply that's already queued before sending, since under the lock that can only be a late one. Reloading the driver while 4 processes send feature requests, 200 cycles: 108 unconfirmed stops before, 0 after. On a KASAN + lockdep build, 160 before, 3 after, no reports. The FEEDBACK reply also echoes the buffer index in its first 4 bytes, so replies could be matched exactly if we want to go further.

Your last three look complementary to that. Mine keeps the ME from getting one reply behind, your reconnect gets it out if it happens anyway, and your mode switch retry covers the EDS1 case #187 still leaves open (touch stays off until the next probe). I only have the SP7, so I can't test EDS1 myself. Your first three overlap with #187 (no retry on -EINTR, the thread reference, tearing down after a failed stop). One difference there: when the device didn't confirm the stop, #187 removes the HID device but keeps the DMA buffers instead of freeing them, since the device may still write to them.

Since you want to test the full set, I can put one branch together: #187, my FEEDBACK fix, and your last three rebased onto v7.3, with your authorship and sign-off kept on yours. We both run it, then decide with the maintainers whether it goes into #187 or a follow-up. I'd lean follow-up so #187 doesn't grow, but I'm fine either way.

Separate question: do you see "QUIESCE_IO: recv failed" on the SB1 when it suspends? I still get a rare unconfirmed stop on suspend, about 1 in 100, with or without my fix, and nothing owed in the ones I traced. Haven't found why yet.

@loafecho

Copy link
Copy Markdown
Author

@jaydenmilne the combined branch is up: https://github.com/loafecho/kernel/tree/ipts-combined-test (branch ipts-combined-test)

It's #187 as it is now (6e15adc on v7.3-surface-devel) plus four commits:

  1. My FEEDBACK fix, one step further than what I described before: one FEEDBACK at a time under a lock, drop any late reply before sending, and now also take only the reply whose first payload word names the buffer we sent (the device echoes the buffer index there), dropping others with a warning, all inside the usual 1 s limit.
  2. Your three, with your authorship and sign-off kept. Each has a [loafecho: ...] line saying what I changed to rebase it onto hid: ipts: Bound the stop path and free the context by reference #187:

Nothing was dropped.

Results on my SP7 (EDS2, Ubuntu 24.04, 7.3.0-rc7 from v7.3-surface-devel), same tests as before:

Test #187 alone This branch
Reload 200x while 4 processes send feature requests, normal kernel 108/200 stops unconfirmed 0/200, 200/200 passed (ran it twice, 0 both times)
Same on a KASAN + lockdep + DEBUG_LIST kernel 160/200 3/200, no reports, lockdep still on
Suspend/resume 100x 1/100 unconfirmed 100/100 resumed, still 1/100 unconfirmed

It's installed as my daily driver now. The suspend one is the rare unconfirmed stop I asked you about before; it's the same with or without the fix, so it's something else.

What I can't test: your mode-switch retry and the ME reconnect only run on EDS1, so on my SP7 they compile and stay out of the way, nothing more. Could you run it on the SB1? Two things I'd look for:

  • whether the FEEDBACK reply echoes the buffer index on EDS1 too. If it doesn't, every FEEDBACK would wait out its 1 s and fail, and dmesg would fill with "FEEDBACK: dropped a reply for buffer". On EDS2 I never saw one in those 700 cycles.
  • a failed mode switch coming back through the retries, if you have a way to make one fail.

My commit only has an Assisted-by tag for now; I'll add my Signed-off-by before anything goes into a PR. Once you've run it, we can decide with the maintainers whether it goes into #187 or a follow-up (I'd still lean follow-up).

@loafecho

Copy link
Copy Markdown
Author

@jaydenmilne follow-up PR is up: #196. It's the combined branch you saw (my FEEDBACK fix plus your three, with your authorship and sign-off kept) plus one new commit for the rare unconfirmed stop on suspend.

Cause: IPTS is a fixed-address MEI client, so on a stop the ME sends the READY_FOR_DATA and QUIESCE_IO replies back to back. When both land before the bus's rx work runs, the bus calls our callback once, we read one reply, and the QUIESCE_IO reply just sits in the bus queue. The fix keeps reading until nothing's ready. 300 suspends each, alternating: 3 unconfirmed without it, 0 with it.

Your reconnect commit describes the ME getting stuck one response behind with the cause unknown. That might be this same thing (a reply stranded in the bus queue until the next one arrives), though I haven't proven it. If you can still hit that state on the SB1, it'd be worth seeing whether it still happens with the new commit.

I still can't test the EDS1 paths here, so a run on the SB1 before merge would be great.

@jaydenmilne

jaydenmilne commented Nov 19, 2026 •

Copy link
Copy Markdown

OK - ran the soak test. Claude found a regression and says it fixed a few things, they're 4 new commits on the tip of my branch. The fully patched series ran very well.

I can paste more of claude's notes if that helps. I can't reproduce the "one response behind" state with the complete branch, so it's possible we can drop the recovery when it gets behind

Driver iptsd still running after systemctl restart iptsd
stock 7.3.4 4/4
#187 alone 0/4
#196 0/6
#196 + fixes 6/6, and 273/273 in the soak below
Check Result
Freezer failures / kernel warnings 0 / 0
Leaked HID devices 0 (exactly one in all 273 cycles)
SIGTERM during mode switch 273/273 recovered
Immediate systemctl restart iptsd 273/273 iptsd kept running
Reload tests 68 (340 reloads, 58,206 feature requests), 0 failed, slowest rmmod 1.2 s
Unconfirmed / leaked stops 0
Stuck thread / HID request / lock / restart 0
QUIESCE_IO recv failures 0
ME reply timeouts (-11) 0 (earlier soaks of my old series: 7 in 314 cycles, 9 in 339)
Restart retries / give-ups / reconnects 0 / 0 / 0 (never triggered)
FEEDBACK calls (kprobe) 5, all succeeded, 0 "dropped a reply for buffer"

@loafecho

Copy link
Copy Markdown
Author

@jaydenmilne thanks for running it, and for catching the iptsd restart one. That's from my "Run the EDS1 mode-switch restart from a work item" in #187, as your commit message says, and the SP7 is EDS2 so I never saw it.

I read your four commits and built them on top of #196: no new warnings, and checkpatch only flags the Co-Authored-By form. On the SP7 they don't change anything, since the restart path only runs on EDS1. Three notes:

  1. "Stop the device before every restart attempt" covers a receiver thread that didn't stop (the thread pointer stays set, so the retry's stop times out again and blocks) and a stuck HID callback (-EBUSY again, or buffers_unconfirmed is still set). It doesn't cover the "device did not confirm its stop" case the comment mentions. The failed start's own stop already leaked the buffers and cleared buffers_unconfirmed, and ipts_receiver_stop() resets receiver_loop.exit_code when there's no thread, so the retry's stop returns 0 and the device gets started again.
    The same thing can happen with a queued restart: a mode switch that comes in while restart_work is stopping the device requeues the work. If that stop leaks, blocked: sets restart_blocked, but the requeued run never checks it, its stop returns 0 for the same reason, and it starts the device. restart_blocked is only read in ipts_hid_enter_restarting(). So I think restart_work needs to check it at the top, and a failed start whose stop leaked needs to set it. Or the comment and message should only claim the thread and HID cases. Your call, since a set_mem_window timeout already leaks and retries on purpose.

  2. Your table has retries / give-ups / reconnects at 0 / 0 / 0, so the soak never ran the retry path that commits 1 and 2 change, or a remove during one (commit 4). If it's easy, one forced failure of ipts_control_request_data() on a restart would run the retry end to end. It won't hit the case in 1, which also needs that start's own stop to go unconfirmed.

  3. Dropping the reconnect: probably right, if IPTS is a fixed-address MEI client on the SB1 too. On the SP7 it is (fixed in its /sys/bus/mei/devices entry is non-zero), and for those mei_cldev_disable()/mei_cldev_enable() never talk to the ME. They only reset host-side state: unregister the callback, flush the client's queues and relink it. That would clear the kind of stranded reply that "Read every response that is ready" now handles at the source. Your 0 ME timeouts in 273 cycles fits that, though the 7/314 and 9/339 were your older series, so it's a hint, not proof. If fixed is non-zero on the SB1, I'd drop the reconnect commit and the reconnect part of the remove bound with it. The 2 x 30 s in commit 4 doesn't apply to a fixed-address client anyway: mei_cl_connect() and mei_cl_disconnect() skip the wait for those.

Since #187 alone breaks iptsd restarts on EDS1, it shouldn't go in by itself. Happy to pull your commits into #196 with your authorship and sign-off once you've settled 1 and 3, or you can open them on top, whichever you prefer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants