Skip to content

hid: ipts: Follow-ups to #187: FEEDBACK replies, EDS1 restarts, suspend stop - #196

Open
loafecho wants to merge 14 commits into
linux-surface:v7.3-surface-develfrom
loafecho:ipts-followup
Open

loafecho wants to merge 14 commits into
linux-surface:v7.3-surface-develfrom
loafecho:ipts-followup

Conversation

@loafecho

Copy link
Copy Markdown

This builds on #187 and should go in after it. The first nine commits are #187 as it is now (head 6e15adc, with #178 first); please review only the last five. They drop out of this diff once #187 lands.

The five commits:

  • hid: ipts: Keep each FEEDBACK reply with the command it answers (mine). The receiver loop and HID feature requests both send FEEDBACK, and replies were matched to their sender by command code only. With several in flight, a sender could take another's reply, and the device then stayed a reply behind, so the QUIESCE_IO sent on stop was never answered. Now one FEEDBACK runs at a time under a new lock, a late reply is dropped before sending, and only the reply that echoes the buffer index that was sent is taken.
  • Jayden Milne's three fixes from IPTS sleep improvements #188, rebased onto hid: ipts: Bound the stop path and free the context by reference #187, with his authorship and Signed-off-by kept. Each has a [loafecho: ...] note under his trailers saying what changed in the rebase:
    • hid: ipts: Drop stale responses when starting the device
    • hid: ipts: Restart the device if switching modes failed
    • hid: ipts: Reconnect to the ME when restarting after a failed mode switch
  • hid: ipts: Read every response that is ready when one arrives (mine). About one suspend in a hundred left the stop unconfirmed with every build I tested on my one device: hid: ipts: Bound the stop path and free the context by reference #187 alone, the FEEDBACK fix, and the first four commits. IPTS is a fixed-address ME client, so when stopping in poll mode the ME sends the READY_FOR_DATA and QUIESCE_IO responses back to back. When both complete before the MEI bus runs its receive work, the bus calls the driver once, the driver read one response, and the QUIESCE_IO response was never read. The receive callback now reads until no response is ready within a millisecond (msecs_to_jiffies(1), so at least one jiffy), and each run of it ends with that short wait. Responses only answer commands, so the loop ends one such wait after the last answer; by the time remove waits for the receive work in mei_cldev_disable(), the stop has sent its last command.

Tested on one device, a Surface Pro 7 (EDS2, poll mode) on Ubuntu 24.04 with linux-surface 7.3.0-rc7 from v7.3-surface-devel plus unrelated camera patches:

Test #187 alone First four commits All five
Reload the driver 200 times while four processes send feature requests 108/200 stops unconfirmed 0/200, run twice 0/200
The same on a KASAN, lockdep and DEBUG_LIST kernel 160/200 3/200 0/200, no reports, lockdep still on
Suspend and resume 1/100 unconfirmed 6/600 0/300

The suspend numbers for the last two columns: 300 traced suspends with the first four commits left 3 stops unconfirmed, and in each the two responses completed together and only READY_FOR_DATA was read. Then 300 suspends with each build, alternating in blocks of 50 on one boot: 3 unconfirmed with the first four commits, none with all five. Every unconfirmed stop in the second set looked the same in the trace. In the runs with all five, that timing did not come up at all, so they show the rate, not the fix catching the case in the act; that it handles it rests on the code path, since the callback now keeps reading what the bus has completed.

One thing I can't explain: once, a reboot with the fifth commit loaded, right after a 200-reload run, froze after the journal stopped, and the Surface needed a hard reset. The same sequence again (a 200-reload run on the debug kernel, then a reboot with the fix loaded and the kernel set to panic on a hung task or soft lockup) was normal, and the 300 suspends above go through the same ipts_remove() path, since mei_stop() removes the client on suspend too. A reboot also runs mei_me_shutdown()'s own steps and the firmware restart, which suspend does not, so I can't rule it in or out.

Not tested: Jayden's mode-switch retry and ME reconnect run only on EDS1 devices, so on my EDS2 device they build and stay idle. The buffer index echo was seen on EDS2 only; if an EDS1 device does not echo it, every FEEDBACK there would wait out its one second and fail, and the kernel log would fill with "FEEDBACK: dropped a reply for buffer". A run on an EDS1 device (Jayden's Surface Book) before merging would settle both. Each commit builds on its own, and checkpatch --strict is clean apart from the Co-Authored-By form in Jayden's trailers.

AI-assisted: Claude Code (an LLM) found both bugs, wrote my two commits and their messages, rebased Jayden's three onto #187, and ran the device tests on my Surface Pro 7. I directed the work, reviewed it, and take responsibility for it. My commits carry an Assisted-by: LLM tag.

Josh Spaulding and others added 14 commits November 6, 2026 00:00
ipts_mei_search() looks up a matching response under the read side of
message_lock, drops the lock, and then re-acquires it for writing to
unlink the entry. Two concurrent callers waiting for the same command
code (typically the receiver poll thread and a userspace hidraw raw
request, both inside ipts_control_send_feedback()) can each find the
same entry during their read-locked walk and then both unlink and free
it. The second list_del() hits LIST_POISON and, with CONFIG_DEBUG_LIST,
oopses the poll kthread:

  list_del corruption, ...->next is LIST_POISON1 (dead000000000100)
  kernel BUG at lib/list_debug.c:56!
  RIP: __list_del_entry_valid_or_report.cold
  Call Trace:
   ipts_mei_search+0x83/0x150 [ipts]
   ipts_mei_recv+0x61/0x190 [ipts]
   ipts_cmd_recv_timeout+0x2f/0x90 [ipts]
   ipts_control_send_feedback+0x81/0xc0 [ipts]
   ipts_receiver_poll_loop.cold+0x89/0x11c [ipts]
   ipts_thread_runner+0x21/0x40 [ipts]

Without CONFIG_DEBUG_LIST the same race shows up as a general
protection fault on 0xdead000000000108 in ipts_mei_search(). The other
caller is then left blocked in D state inside the driver, which makes
every later suspend fail with "tasks refusing to freeze". Seen on
resume from s2idle on a Surface Pro 7 while iptsd was (re)started.

Hold the write lock across the search and the unlink so that a message
can only be claimed by one caller.

Signed-off-by: Josh Spaulding <git@spaulding.app>
The context is allocated with devm_kzalloc(), so devres frees it when
the device is released, whoever still uses it. The following changes
bound the stop path, and a stop that gives up must keep everything that
a receiver thread or a HID device which is still there can touch. With a
devm context, devres frees it anyway, and the next touch is a
use-after-free.

Reference-count the context instead, and free it only when the last
reference is dropped:

- Probe holds one reference, dropped by a devres action. Remove and
  every later probe failure disable the MEI device before that, which
  cancels its receive callback, so no message arrives while the context
  is freed. If adding the devres action fails, the context is freed at
  once, before the MEI device is disabled; the receive callback finds no
  context then, because drvdata is not set yet.
- The receiver thread and the HID device hold one each, for their whole
  life.
- The context holds a reference to the MEI device until it is freed: a
  user that is still there after remove keeps sending through it and
  logging against it.

ipts_remove() then waits, bounded, for the remaining users to return.
That narrows the window in which the module code is freed under one
that is about to return, but does not close it: the last user wakes
remove from inside its own put, before it has left the module code.
After a failed stop the receiver thread or the HID device is still there
on purpose, and remove does not wait for it.

A failed probe used to return without disabling the MEI device, so its
receive callback stayed registered after the context was gone. Every
probe failure after mei_cldev_enable() now disables it.

The messages are now plain kzalloc() instead of devm: they are freed on
the error paths of the receive callback, and any left over is freed with
the context.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
ipts_remove() runs on rmmod, and also on every reboot, power-off and
suspend: mei_me_shutdown() and mei_me_pci_suspend() both call
mei_stop(), which removes every ME client device. ipts_thread_stop()
waited for the receiver thread with wait_for_completion() and no
timeout. If the thread died in an oops, or never got out of a wait, the
stop never returned, and reboot or suspend hung with it.

Wait for the thread with a timeout instead, which the caller passes in.
While waiting, check whether the thread is exiting: a thread that died
runs no driver code anymore and counts as stopped, without calling
kthread_stop() on it, which could wait for its exit forever. A thread
that is still running after the timeout is reported to the caller, which
then frees none of the driver's memory that the thread can still touch:
the receiver keeps the thread's context reference, so the context stays,
and the context keeps the MEI device. Only a later stop drops that
reference. After remove there is none, so the context then stays for
good.

Until later changes in this series remove the HID device on those
paths, a stop that gives up here, or finds the thread dead, still
returns before the HID device is removed: rmmod with a stuck or dead
receiver then leaves the HID device registered, and opening its hidraw
node runs freed module code. Before this change, that rmmod hung for
good instead.

Keep a reference on the task_struct from creation until the stop is
done, so that a thread which exits on its own early can still be looked
at, and never store a kthread_create() error as the thread. The return
value of the thread function moves to thread->exit_code; the receiver
reports it as before.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
Every command waits for its reply in ipts_mei_recv(), which slept in
wait_event() with ipts_mei_search() as the wait condition. The search
takes message_lock, so a waiter blocked on the lock inside the wait
condition, and a task that held the lock without releasing it (one that
died while holding it) blocked every later command for good: the
receiver thread, the HID requests, and the shutdown handshake on the
stop path.

The lock only ever covers a list add or delete, so nobody has a reason
to hold it for long. Never block on it:

- ipts_mei_search() only tries the lock, and reports a busy lock to the
  caller.
- ipts_mei_recv() searches with the task running and sleeps until a new
  message is queued (a sequence counter bumped under the lock) or the
  time is up. While the lock is busy, it tries again after a short
  sleep. The time limit covers both.
- ipts_mei_incoming() gives up after a bounded wait for the lock and
  drops the response.
- Sends use mei_cldev_send_timeout(). An interrupted send is not
  retried, because the message is already queued and would go out
  twice. The timeout bounds the wait for the write to complete; the
  MEI core can still wait without a bound for its bus lock, and for
  room in a full transmit queue, which the driver cannot change.

Holding the lock also holds a context reference, taken and dropped by
the new lock helpers, so the context is never freed under a holder that
never lets go. The lock is released before the reference, because
dropping the last reference frees the context, and the release function
takes the lock itself to free the messages.

The "negative timeout waits forever" case could never be reached with a
u64 timeout and is gone; a zero timeout still checks once.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
When the receiver thread returned an error from its shutdown handshake,
_ipts_control_stop() and ipts_control_stop() returned early and never
called ipts_hid_free(). The module then unloaded with its hid_device
still registered, and the next open or ioctl of its hidraw node ran
freed module code: a page fault in hidraw_open() or inside
__hid_hw_raw_request(), with "[last unloaded: ipts]" in the oops.

Remove the HID device on every path once the receiver thread is gone.
The only exception is a receiver thread that is still running after the
bounded stop: it still feeds the HID device and uses the buffers, so
nothing is freed and the error is passed on.

Until a later change in this series counts the receiver's HID input,
that exception still lets rmmod with a stuck receiver leave the HID
device registered, with the hidraw problem above. And a receiver that
died inside hid_input_report() may hold HID core locks, so destroying
the HID device after it can block; the same later change makes the stop
wait for running HID input first, and keep the HID device in that case.

If the thread died, or the device did not confirm the stop, the device
may still write to its DMA buffers. Leak them instead of freeing them;
the next start allocates new ones.

The stop now reads the thread's exit code itself. A stop that finds no
thread, because a start failed before it started one, reports a clean
exit instead of the exit code of the thread before.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
…firmation

Once a stop is requested, the receiver loops still finished every buffer
of the current round before they looked at the stop flag again, and the
poll loop walked through every doorbell value it had missed, however far
the doorbell had moved. Check the stop flag inside the inner loops too,
and never walk back further than the buffers that can hold new data.

The poll loop sent QUIESCE_IO only when it saw the stop flag at the top
of a round. A stop requested while it handled buffers ended the loop at
the bottom check without sending it, and the device was never told to
stop. Send it after the loop instead, as the event loop does, and then
give back the buffers the loop did not get to, as before: in poll mode
the device confirms the stop only once every filled buffer was given
back.

At the end of a loop, the stop handshake gave up on the QUIESCE_IO
confirmation as soon as the last READY_FOR_DATA reply was missing, and
reported a missing reply as success. Starting the device again before it
has stopped can leave its firmware unresponsive until the next boot, and
freeing buffers it still writes to corrupts memory. Collect the last
READY_FOR_DATA reply, then wait for the QUIESCE_IO confirmation for a
bounded time in any case, and return success only when the device
confirmed the stop. Otherwise the stop keeps the DMA buffers.

That is a trade-off: a device that never confirms its stop leaks one set
of DMA buffers per stop, and that repeats on every suspend and every
restart, with a warning each time. Memory the device may still write to
is never freed.

Every step after the stop request is bounded now, apart from the waits
inside the MEI core that a send can still make, so the time a stop
waits for the receiver thread is the sum of those bounds instead of a
guess. A slow device then still gets its stop confirmed, instead of
leaving a thread behind that keeps the buffers. The bounds move to
receiver.h, so that the waits of the stop path can be derived from them
too.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
On EDS1 devices, a HID request that switches the mode restarts the
device from inside the HID callback, and nothing serialized that restart
with ipts_remove(): both could stop the receiver thread and free the
buffers at the same time. A stop that waits for running HID callbacks,
as the next change makes it do, would also wait for the very callback
that called it.

Run the restart from a work item instead, under a mutex that remove
takes for its own stop. A queued restart holds a context reference until
it has run. Remove marks the context as going away first, so no new
restart is scheduled, and a restart that is already running checks that
mark again before it starts the device.

Remove never waits for the work without a bound: the worker can be the
task that is stuck, or that died with the mutex held, and reboot and
suspend must still finish. It cancels a queued restart without waiting,
before and after its stop, and takes the mutex with a bounded wait. The
bound is derived from the bounds of a start and a stop, the way the
receiver's stop timeout is: it covers whatever a running restart can
still do once remove has marked the context. If a restart holds the
mutex past that bound, it is stuck, and remove skips the stop and frees
nothing that restart can still use: the restart holds a context
reference, and the context holds the MEI device.

A restart whose stop was not confirmed by the device, so that the DMA
buffers were leaked, no longer starts the device again: starting a
device that has not stopped can leave its firmware unresponsive until
the next boot. The stop reports that case with a return value of its
own. Touch input then stays off until the driver is probed again.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
Stopping only cleared hid_active, which new HID callbacks check on
entry. A callback that had already passed that check kept running while
the stop freed the buffers it used: a SET_FEATURE request writing the
hid2me buffer, or a GET_FEATURE request waiting for a reply in a buffer
that was about to go away.

Count the callbacks that run driver code, and make the stop wait for the
count to drop to zero, with a timeout. A pending GET_FEATURE request is
released at once instead of waiting for its own timeout, and one that
starts while the stop runs returns at once.

The receiver thread's HID input is counted the same way. Once the count
has dropped to zero, the receiver can no longer reach the HID device,
so the HID device is destroyed even when the receiver thread does not
stop. Before, a receiver that did not stop kept the HID device
registered, and after rmmod its hidraw node ran freed module code.

The receiver thread is asked to stop in any case. A callback that
returns while it stops was only slow, and the stop goes on as usual,
except that the buffers are leaked instead of freed: the callback may
have sent the device a command after the device confirmed its stop, and
the device may answer it by writing to them. A callback that still runs
after the timeout and the receiver stop is most likely stuck for good,
in a task that died inside it, possibly with HID core locks held. That
includes a receiver that died inside hid_input_report(). It can still
touch the buffers, and destroying the HID device could block forever on
those locks, so in that case nothing is freed and the error is passed
on. A later stop then leaks the buffers instead of freeing them,
although it finds no receiver thread anymore.

So rmmod after a HID callback died, or while one never returns, still
leaves the HID device registered, and its hidraw node runs freed module
code. Destroying it would trade that for a remove that can block, and
with it reboot and suspend.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
When a step of ipts_control_start() failed, it returned at once and left
behind whatever the earlier steps had set up: the buffers, and a
receiver thread that kept running against a device that was never told
to stop. An allocation error from hid_allocate_device() was even kept in
ipts->hid, where later code took it for a device.

Tear down on failure what the start set up. Buffers the device was
never given are freed. Buffers it may have been given are leaked, since
it was never told to stop: that covers a GET_DESCRIPTOR request that
got no reply, because the device may still write the descriptor. Once
the receiver thread runs, the normal stop path stops it, and keeps what
a thread that does not stop can still use. On an EDS1 restart, that stop
also removes the HID device that probe created. A HID device that failed
to register is no longer destroyed in ipts_hid_init(), while the
receiver may already use it, but by that stop, once HID input has
stopped.

The receiver thread already runs while ipts_hid_init() sets up the HID
device, so the device is now set up in a local variable and published
in ipts->hid only once it is complete. The receiver never sees an error
pointer or a half set-up device.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
The receiver loop, which gives back data buffers, and HID feature requests,
which use the shared hid2me buffer, both send FEEDBACK. Replies are matched
to their sender by the command code only, so with several FEEDBACKs in
flight a sender can take the reply to another one and return while its own
command is still pending. The device then stays a reply behind: it holds
its newest reply until the next command arrives, and a QUIESCE_IO sent in
that state is never answered, so stopping the device times out.

Allow one FEEDBACK at a time with a new feedback_lock, held from the send
until the reply is taken. The lock also guards the hid2me buffer, so a
second feature request cannot overwrite it while the device may still read
it. GET_FEATURE releases it before waiting for its report, because the
receiver loop needs it to give back the buffer the report arrives in. Lock
order is feature_lock before feedback_lock.

Under the lock, drop any FEEDBACK reply that is already queued before
sending: it answers a sender that gave up waiting. Then take only the reply
whose first payload word names the buffer that was sent (the device echoes
the buffer index there), dropping others with a warning, within the usual
one second limit. Two hid2me replies look alike, but the lock allows only
one hid2me FEEDBACK at a time.

On a Surface Pro 7 (EDS2), reloading the driver 200 times while four
processes send feature requests left 108 stops unconfirmed before this
change and none after it. On a KASAN and lockdep build: 160 before, 3
after, with no reports. The buffer index echo was checked on EDS2 only.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
When sending a command is interrupted, or waiting for its response times
out, the ME may still answer later. That response stays in the list of
received messages until the device is removed, where it can be mistaken
for the response to a later command with the same code.

This is not just a leak. After a restart, a stale READY_FOR_DATA response
makes the event mode receiver process a buffer that has no data, and if
its status is an error, the next shutdown of the receiver fails:

  IPTS running in event mode
  Failed to process buffer: -19
  FEEDBACK: recv failed: -11
  Failed to send feedback: -11
  Stopping IPTS
  READY_FOR_DATA: cmd failed: 5
  Failed to wait for data: -53

Drop all unconsumed responses when starting the device, before sending
the first command.

Signed-off-by: Jayden Milne <linux-surface@jayd.ml>
Patchset: ipts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[loafecho: rebased onto linux-surface#187: ipts_mei_flush() takes the message lock
 with the bounded ipts_mei_lock_timeout() and frees with kfree(),
 because linux-surface#187 allocates messages with kzalloc()]
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
On EDS v1 devices, switching modes restarts the device. If the restart
fails (e.g. because the calling task has a signal pending), the device
stays stopped and the HID device inactive. Since ipts_hid_raw_request()
rejects requests while the HID device is inactive, userspace cannot
retry, and touch input stays broken until the module is reloaded or the
MEI client device is probed again.

When the restart fails, restore the previous mode and restart the device
from a work item instead, retrying a few times. Restoring the previous
mode means that the device comes back in the mode that works without
iptsd, and that iptsd can switch modes again once it is restarted.

Serialize restarts with a mutex, and make sure that no restart is queued
or running when the device is removed.

Signed-off-by: Jayden Milne <linux-surface@jayd.ml>
Patchset: ipts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[loafecho: rebased onto linux-surface#187, which already restarts the EDS1 device from
 a work item serialized by ctl_lock, cancelled on remove without waiting.
 The retries and the return to the previous mode now live in that work
 item: a failed start restores the old mode and queues a retry after
 2 s, up to 5 attempts; a retry skips the stop. No second lock or work
 item: taking a lock in the HID callback would deadlock with the stop,
 which waits for HID callbacks.]
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
…itch

After a command times out, the ME client can get stuck one response
behind: each response is delivered only when the next command is sent.
Every later command then times out as well. The restart attempts cycle
between failing on GET_DEVICE_INFO and failing on SET_MODE until they give
up, and the device stays stopped.

Waiting does not clear this, but reloading the module does. Disconnect
from the ME client and connect again before each restart attempt, like a
re-probe does. The cause of the stuck response is not known.

Signed-off-by: Jayden Milne <linux-surface@jayd.ml>
Patchset: ipts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[loafecho: rebased onto linux-surface#187: the reconnect runs before each retry in
 linux-surface#187's restart work item, as it did before each background attempt;
 a failed reconnect counts as a failed attempt]
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
The MEI bus calls the receive callback from a work item that it schedules
when a response arrives, and the callback read one response. IPTS is a
fixed-address ME client, so the ME can send the next response before that
work has run. Scheduling it again then does nothing: the callback runs
once, and the second response waits with the bus until another arrives.

When stopping in poll mode, QUIESCE_IO gets two responses: first the one
for the READY_FOR_DATA sent at start, then its own, usually about half a
millisecond later. When both arrive together, the QUIESCE_IO response is
never read and the stop is not confirmed.

Read responses in the callback until none is ready within a millisecond
(at least one jiffy), so each run of it now ends with that short wait.
The wait is bounded, so a call whose response an earlier call already
read returns too, and since responses only answer commands, the loop ends
one wait after the last answer.

On a Surface Pro 7 (EDS2), 300 traced suspends left 3 stops unconfirmed,
each one this way. Over 300 more suspends per driver, alternated in
blocks of 50, the stop was left unconfirmed 3 times without this change
and never with it. In the runs with this change the two responses never
arrived together, so they show the rate, not this change catching it.

Fixes: 820c589 ("hid: Add support for Intel Precise Touch and Stylus")
Patchset: ipts
Assisted-by: LLM
Signed-off-by: Tony Garcia <180770769+loafecho@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant