Repository navigation
[stable-v2.15] audio: arbiter, kbp, mfcc, microwakeword: Backport wov-upstream plus add SDW single wov-topologies - #11413
Open
singalsu wants to merge 63 commits into
Conversation
The "flags" argument of notifier_register() is unused, remove it. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com> (cherry picked from commit 9820953)
When comp_prepare() returns a negative error code in pipeline_comp_prepare(), emit a pipe_err message including the component ID and error code. This greatly accelerates diagnosing topology and IPC parameter configuration failures during pipeline setup. Co-developed-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
In copier_dai_free(), guard cd->dd[i] against NULL before accessing cd->dd[i]->gain_data and calling dai_common_free(). This prevents a NULL pointer dereference if endpoint allocation failed during initialization or during teardown of partially configured copier instances. Also reset pointers to NULL after freeing. Co-developed-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
In dai_common_free() and dai_free(), guard dd, dd->dai, dd->dma, and dd->dai_spec_config against NULL before dereferencing or releasing channels, and reset pointers to NULL after freeing. Also add NULL pointer guards to dai_put() and sof_dma_put(). This prevents NULL pointer dereferences and duplicate frees during teardown of partially configured or failed DAI pipelines. Co-developed-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
When pipelines are cancelled via pipeline_schedule_cancel() during pause or stop triggers, sa_set_panic_on_delay() is called to enable panic on delay. In host testbench and library builds, the system agent timer is not initialized, leaving the sa pointer NULL. Add a NULL check in sa_set_panic_on_delay() before dereferencing the sa pointer so host testbench runs do not crash with a segmentation fault. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
When a capture stream is starved or waiting for detection audio, logging every 20 ms floods the trace buffer and causes diagnostic log messages to be dropped. Increase SOF_MIN_NO_BYTES_INTERVAL_MS from 20 ms to 1000 ms so that trace transport bandwidth is conserved during idle capture periods. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
When freeing an EDF scheduled task, ensure any pending delayed work associated with the task is cancelled before marking the task free and clearing callbacks. This prevents delayed work from triggering on freed task data during pipeline resets or teardown. Co-developed-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
CONFIG_ZEPHYR_DP_SCHEDULER can be enabled on cAVS, but only the ACE platform_init() registered the DP scheduler. Creating a DP module on cAVS then failed with "unable to bind scheduler to task (type 3)" and module init returned -ENODEV. Call scheduler_dp_init() after the LL scheduler is set up, as ACE does. Co-developed-by: Liam Girdwood <liam.r.girdwood@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The DP lock is taken by the LL thread on every tick in scheduler_dp_ll_tick() and by each DP thread after every run. It was a k_sem, which has no priority inheritance: when a DP thread holding it was preempted by other DP threads, the cooperative high priority LL thread blocked until the holder got CPU time again. With several busy DP modules (3 MFCC + 3 MWW on cAVS 2.5) single LL ticks took up to ~40 ms, overrunning the DMIC DMAs of unrelated streams. With a k_mutex the holder inherits the LL priority: LL max drops from ~1.5M to <30k timer cycles, with no LL overruns or DMA xruns. Co-developed-by: Liam Girdwood <liam.r.girdwood@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical issues include unsafe component teardown and notification payload lifetimes, configuration-dependent link failures, and a microphone topology with an unresolved route and no input.
6 open issues
What changed in this PR
This PR backports Wake-on-Voice support into SOF, adds multi-slot DMIC and single-slot SoundWire topologies, and extends MFCC feature processing for microWakeWord.
Changes:
- Integrate WoV arbitration, ECNS, microWakeWord, PCAN, and AGC8 through module metadata, build wiring, and platform configuration.
- Extend topology classes, production targets, controls, and audio-processing presets.
- Update scheduling, notifications, buffering, and component cleanup; add capture tooling and unit-test infrastructure.
This pull request requires changes.
Approval blockers include unsafe component teardown and reset-result handling, remote-notification payload lifetime, a missing square-root build dependency, and an unresolved SoundWire microphone route. Additional findings concern KPB platform defaults and duration selection, ineffective PCAN smoothing configuration, capture-control handling, and missing linear-Mel reference tests.
| File | Description |
|---|---|
zephyr/lib/vregion.c |
Include LLEXT symbol definitions. |
zephyr/Kconfig |
Add DP-to-DP binding configuration. |
zephyr/edf_schedule.c |
Cancel delayed work during task freeing. |
zephyr/CMakeLists.txt |
Adjust extension linking and Xtensa compilation flags. |
uuid-registry.txt |
Register ECNS, MWW, and WoV arbiter UUIDs. |
tools/wov_capture/wov_blocking_read_tinyalsa.c |
Add a tinyalsa blocking-capture utility. |
tools/wov_capture/Makefile |
Add capture-tool build and installation targets. |
tools/topology/topology2/sof-hda-generic.conf |
Include WoV classes and conditional MFCC support. |
tools/topology/topology2/production/tplg-targets-sdca-generic.cmake |
Register a single-slot SoundWire WoV target. |
tools/topology/topology2/production/tplg-targets-hda-generic.cmake |
Register an ACE3 multi-slot WoV target. |
tools/topology/topology2/production/tplg-targets-cavs25.cmake |
Register a four-channel TGL WoV target. |
tools/topology/topology2/platform/intel/dmic-generic.conf |
Add multi-slot selection and adjust capture-object gating. |
tools/topology/topology2/platform/intel/dmic-default.conf |
Define the WoV DAI channel-count default. |
tools/topology/topology2/include/pipelines/cavs/wov-kpb-be.conf |
Add a configurable KPB pipeline class. |
tools/topology/topology2/include/pipelines/cavs/dai-mixin-be.conf |
Add a configurable DAI capture pipeline class. |
tools/topology/topology2/include/pipelines/cavs/dai-kpb-be.conf |
Add DMIC defaults, formats, and KPB routing. |
tools/topology/topology2/include/components/wov.conf |
Replace initialization bytes with switches and add an output pin. |
tools/topology/topology2/include/components/wov-arbiter.conf |
Define the arbiter widget and active-slot control. |
tools/topology/topology2/include/components/mww/score_enum.conf |
Define probability-score enum labels. |
tools/topology/topology2/include/components/mww.conf |
Define the MWW widget and fake-wake switch. |
tools/topology/topology2/include/components/mfcc/mel80.conf |
Refresh the 80-band Mel preset. |
tools/topology/topology2/include/components/mfcc/mel80_pcan_compress.conf |
Add compressed 80-band PCAN features. |
tools/topology/topology2/include/components/mfcc/mel80_compress.conf |
Refresh compressed 80-band features. |
tools/topology/topology2/include/components/mfcc/mel80_compress_dtx.conf |
Refresh the 80-band compression/DTX preset. |
tools/topology/topology2/include/components/mfcc/mel40.conf |
Add a 40-band Mel preset. |
tools/topology/topology2/include/components/mfcc/mel40_compress.conf |
Add compressed 40-band features. |
tools/topology/topology2/include/components/mfcc/mel40_10ms.conf |
Add a 10-ms-hop Mel preset. |
tools/topology/topology2/include/components/mfcc/mel40_10ms_pcan_compress.conf |
Add streaming PCAN features with VAD. |
tools/topology/topology2/include/components/mfcc/mel40_10ms_compress.conf |
Add streaming compressed Mel features. |
tools/topology/topology2/include/components/mfcc/mel40_10ms_agc8_compress.conf |
Add streaming AGC8 features with VAD. |
tools/topology/topology2/include/components/mfcc/default.conf |
Refresh default MFCC parameters. |
tools/topology/topology2/include/components/mfcc/ceps13_compress_dtx.conf |
Refresh the cepstral compression/DTX preset. |
tools/topology/topology2/include/components/mfcc.conf |
Change VAD switch inversion. |
tools/topology/topology2/include/components/kpb.conf |
Add history-duration controls and a static IPC4 UUID. |
tools/topology/topology2/include/components/host-copier.conf |
Document D0i3 capture compatibility. |
tools/topology/topology2/include/components/eqiir/highpass_50hz_40db_48khz.conf |
Add a 50-Hz, 40-dB, 48-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_50hz_40db_16khz.conf |
Add a 50-Hz, 40-dB, 16-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_50hz_30db_48khz.conf |
Add a 50-Hz, 30-dB, 48-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_50hz_30db_16khz.conf |
Add a 50-Hz, 30-dB, 16-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_40hz_40db_48khz.conf |
Add a 40-Hz, 40-dB, 48-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_40hz_40db_16khz.conf |
Add a 40-Hz, 40-dB, 16-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_40hz_30db_48khz.conf |
Add a 40-Hz, 30-dB, 48-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_40hz_30db_16khz.conf |
Add a 40-Hz, 30-dB, 16-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_30hz_40db_48khz.conf |
Add a 30-Hz, 40-dB, 48-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_30hz_40db_16khz.conf |
Add a 30-Hz, 40-dB, 16-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_30hz_30db_48khz.conf |
Add a 30-Hz, 30-dB, 48-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_30hz_30db_16khz.conf |
Add a 30-Hz, 30-dB, 16-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_20hz_40db_48khz.conf |
Add a 20-Hz, 40-dB, 48-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_20hz_40db_16khz.conf |
Add a 20-Hz, 40-dB, 16-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_20hz_30db_48khz.conf |
Add a 20-Hz, 30-dB, 48-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_20hz_30db_16khz.conf |
Add a 20-Hz, 30-dB, 16-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_100hz_40db_48khz.conf |
Add a 100-Hz, 40-dB, 48-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_100hz_40db_16khz.conf |
Add a 100-Hz, 40-dB, 16-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_100hz_30db_48khz.conf |
Add a 100-Hz, 30-dB, 48-kHz filter preset. |
tools/topology/topology2/include/components/eqiir/highpass_100hz_30db_16khz.conf |
Add a 100-Hz, 30-dB, 16-kHz filter preset. |
tools/topology/topology2/include/components/ecns.conf |
Define the ECNS widget and boost/test controls. |
tools/topology/topology2/include/common/common_definitions.conf |
Add SoundWire WoV and KPB defaults. |
tools/topology/topology2/cavs-sdw.conf |
Include WoV classes and single-slot feature support. |
tools/topology/topology2/cavs-rt5682.conf |
Include multi-slot WoV component classes. |
tools/rimage/config/wcl.toml.h |
Include new component manifests for WCL. |
tools/rimage/config/tgl.toml.h |
Include new component manifests for TGL. |
tools/rimage/config/ptl.toml.h |
Include new component manifests for PTL. |
tools/rimage/config/nvl.toml.h |
Include new component manifests for NVL. |
test/ztest/unit/math/advanced/pcan/testcase.yaml |
Register native-simulation PCAN tests. |
test/ztest/unit/math/advanced/pcan/ref_pcan_lut.h |
Add reference gain lookup tables. |
test/ztest/unit/math/advanced/pcan/ref_pcan_func.h |
Add function reference vectors. |
test/ztest/unit/math/advanced/pcan/ref_pcan_corners.h |
Add boundary-case reference vectors. |
test/ztest/unit/math/advanced/pcan/prj.conf |
Configure the standalone PCAN test application. |
test/ztest/unit/math/advanced/pcan/pcan_wide_dynamic_func.m |
Model fixed-point gain interpolation. |
test/ztest/unit/math/advanced/pcan/pcan_shrink.m |
Model piecewise compression. |
test/ztest/unit/math/advanced/pcan/pcan_process.m |
Model frame-level PCAN processing. |
test/ztest/unit/math/advanced/pcan/pcan_noise_estimate.m |
Model temporal noise smoothing. |
test/ztest/unit/math/advanced/pcan/pcan_gain_lookup.m |
Generate reference gain tables. |
test/ztest/unit/math/advanced/pcan/CMakeLists.txt |
Build PCAN tests with microfrontend sources. |
test/ztest/unit/audio/mfcc_agc8/testcase.yaml |
Register native-simulation AGC8 tests. |
test/ztest/unit/audio/mfcc_agc8/prj.conf |
Configure the standalone AGC8 test application. |
test/ztest/unit/audio/mfcc_agc8/CMakeLists.txt |
Build AGC8 unit tests. |
test/cmocka/src/notifier_mocks.c |
Update the notifier mock signature. |
test/cmocka/src/math/auditory/CMakeLists.txt |
Link square-root support into auditory tests. |
submanifests/tflm.yml |
Pin machine-learning workspace dependencies. |
src/schedule/zephyr_dp_schedule.c |
Replace DP semaphores with mutexes. |
src/schedule/ll_schedule_xtos.c |
Update notifier registration arguments. |
src/schedule/dma_single_chan_domain.c |
Update notifier registration arguments. |
src/samples/audio/detect_test.toml |
Increase detector instance capacity. |
src/platform/intel/cavs/platform.c |
Initialize the DP scheduler when enabled. |
src/math/pcan/CMakeLists.txt |
Integrate PCAN and microfrontend sources. |
src/math/numbers.c |
Export CRC32 for extensions. |
src/math/Kconfig |
Add the PCAN math-library option. |
src/math/CMakeLists.txt |
Conditionally build the PCAN library. |
src/math/auditory/mel_filterbank_32.c |
Add linear Mel magnitude output. |
src/lib/notifier.c |
Simplify registration and change remote notification delivery. |
src/lib/dma.c |
Make DMA release tolerate NULL. |
src/lib/dai.c |
Make DAI release tolerate NULL. |
src/lib/cpp_new_export.cpp |
Export C++ allocation and runtime-support wrappers. |
src/lib/CMakeLists.txt |
Build C++ runtime exports. |
src/ipc/ipc4/handler-kernel.c |
Broadcast D0ix state notifications. |
src/ipc/ipc-helper.c |
Attempt stop/reset before component destruction. |
src/include/user/mww.h |
Define the model-blob ABI and feature metadata. |
src/include/user/mfcc.h |
Add PCAN and AGC8 configuration fields. |
src/include/sof/math/auditory.h |
Declare the linear Mel output API. |
src/include/sof/lib/notifier.h |
Add WoV/D0ix events and simplify registration. |
src/include/sof/lib/ams_msg.h |
Add whitespace only. |
src/include/sof/lib/agent.h |
Guard an uninitialized system agent. |
src/include/sof/audio/wov_arbiter.h |
Define arbiter controls and notification payloads. |
src/include/sof/audio/pipeline.h |
Expose pipeline scheduling cancellation. |
src/include/sof/audio/mfcc/mfcc_vad.h |
Adjust VAD thresholds and noise-floor adaptation. |
src/include/sof/audio/mfcc/mfcc_comp.h |
Add output modes and PCAN/AGC8 processing state. |
src/include/sof/audio/mfcc/mfcc_agc8.h |
Define the soft-AGC interface and defaults. |
src/include/sof/audio/kpb.h |
Make history duration and client capacity configurable. |
src/include/sof/audio/ecns.h |
Define ECNS pins, frame sizes, and channel maps. |
src/include/sof/audio/component.h |
Declare new component initialization functions. |
src/include/sof/audio/audio_buffer.h |
Avoid resetting shared secondary buffers twice. |
src/include/ipc4/kpb.h |
Add the history-duration configuration parameter. |
src/include/ipc4/detect_test.h |
Add a trailing enum comma. |
src/drivers/dw/dma.c |
Update notifier registration arguments. |
src/debug/debug_stream/debug_stream_text_msg.c |
Include LLEXT symbol definitions. |
src/audio/wov_arbiter/wov_arbiter.toml |
Define the arbiter module manifest. |
src/audio/wov_arbiter/CMakeLists.txt |
Add arbiter build integration. |
src/audio/pipeline/pipeline-schedule.c |
Expose cancellation and guard missing tasks. |
src/audio/pipeline/pipeline-params.c |
Log component preparation failures. |
src/audio/module_adapter/module_adapter.c |
Adjust DP preparation and buffer synchronization. |
src/audio/microwakeword/mww.toml |
Define the microWakeWord module manifest. |
src/audio/microwakeword/mww_model.h |
Declare classifier and model-management APIs. |
src/audio/microwakeword/llext/llext.toml.h |
Define the loadable MWW manifest. |
src/audio/microwakeword/Kconfig |
Add model, tracing, profiling, and test options. |
src/audio/mfcc/mfcc_agc8.c |
Implement soft AGC and int8 quantization. |
src/audio/mfcc/llext/CMakeLists.txt |
Include AGC8 in loadable MFCC builds. |
src/audio/mfcc/Kconfig |
Add PCAN and tracing options. |
src/audio/mfcc/CMakeLists.txt |
Integrate PCAN headers and AGC8 sources. |
src/audio/kpb.toml |
Expand KPB instances and declare both output pins. |
src/audio/Kconfig |
Add arbiter/ECNS options and configurable KPB sizing. |
src/audio/host-zephyr.c |
Reduce repetitive no-data logging. |
src/audio/host-legacy.c |
Update notifier registration arguments. |
src/audio/ecns/ecns.toml |
Define the ECNS module manifest. |
src/audio/ecns/CMakeLists.txt |
Add ECNS build integration. |
src/audio/dai-zephyr.c |
Update registration and harden partial-initialization cleanup. |
src/audio/dai-legacy.c |
Update notifier registration arguments. |
src/audio/copier/copier.c |
Update microphone-privacy notifier registration. |
src/audio/copier/copier_dai.c |
Guard endpoint cleanup and clear released pointers. |
src/audio/CMakeLists.txt |
Include new audio component directories. |
src/audio/buffers/ring_buffer.c |
Release retained virtual-region references. |
src/audio/buffers/audio_buffer.c |
Support shared DP-to-DP secondary buffers. |
src/arch/host/configs/library_defconfig |
Enable PCAN in host-library builds. |
scripts/tensorflow-clone.sh |
Remove the standalone dependency-cloning script. |
scripts/mww_shim_flatten.py |
Rename shim ELF sections to reduce Xtensa call distances. |
scripts/llext_link_helper.py |
Propagate linker and objcopy failures. |
scripts/host-testbench.sh |
Clear GTK_PATH before host testing. |
README.md |
Link WoV hardware-verification instructions. |
app/wov-d0i3-fake-wake.conf |
Add a synthetic-wake test overlay. |
app/configs/wcl/modules.conf |
Add WCL modular-component settings. |
app/configs/ptl/modules.conf |
Add PTL modular-component settings. |
app/boards/intel_adsp_cavs25.conf |
Enable WoV components and adjust memory/logging settings. |
app/boards/intel_adsp_cavs25_tgph.conf |
Enable PCAN and extend the IDC timeout. |
app/boards/intel_adsp_ace40_nvl.conf |
Enable WoV and revise module/resource defaults. |
app/boards/intel_adsp_ace30_wcl.conf |
Enable WoV and revise module/resource defaults. |
app/boards/intel_adsp_ace30_ptl.conf |
Enable WoV and revise module/userspace defaults. |
Review effort: Balanced
notifier_event() sends events to other cores as non-blocking IDC messages. Each target core has a single notify_data slot and the IDC work item is reused, so two events sent back to back to the same core, such as the MWW detection followed by the KPB drain request, can overwrite the first event's data or resubmit a work item that is still queued. Send remote notifications as blocking IDC so the next event to the same core is not issued before the previous one has been handled. This adds latency on the notifying core, as it now waits for the remote callback to complete. Verified on PTL with WoV detection on core 1 notifying the arbiter and KPB on core 0. The blocking wait is bounded by the IDC timeout, so mark each target slot pending until the remote core has run the callbacks, and drop and log a new event instead of overwriting a slot that is still queued. Events raised from an interrupt handler, such as the mic privacy ones, are sent without blocking. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add PCAN (Per-Channel AGC Normalization) fixed-point math library for normalizing mel filterbank energies. PCAN estimates per-channel noise background, applies non-linear wide-dynamic-range AGC compression with fixed-point LUT gain lookup, and delivers scaled features for streaming keyword-spotting networks. Include ztest test suite with MATLAB reference models covering gain lookup, noise estimation, dynamic shrinking, stream processing, and corner cases. Co-developed-by: Liam Girdwood <liam.r.girdwood@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com> Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
singalsu
force-pushed
the
upstream_stable-v2.15_with_wov-upstream_backport
branch
from
November 9, 2026 11:14
70e1ef5 to
658f055
Compare
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical issues include KPB-disabled build failures, missing MFCC extension exports, overwritten pending notification payloads, and detection being disabled by default.
5 open issues
5 resolved since last review
Review effort: Balanced
singalsu
force-pushed
the
upstream_stable-v2.15_with_wov-upstream_backport
branch
from
November 9, 2026 13:08
658f055 to
316cc9f
Compare
singalsu
force-pushed
the
upstream_stable-v2.15_with_wov-upstream_backport
branch
from
November 9, 2026 13:38
316cc9f to
e5d2c4c
Compare
singalsu
marked this pull request as ready for review
November 9, 2026 13:39
singalsu
requested review from
abonislawski,
dbaluta,
iuliana-prodan,
kv2019i,
lyakh and
ranj063
as code owners
November 9, 2026 13:39
The WCL DMIC and RT721 multi-slot WoV topologies are byte-for-byte identical to the PTL ones, only reachable under a different name. Drop the duplicated TPLGS entries and list them in SYMLINKS instead, so they are built as symlinks to the PTL topologies rather than separately compiled duplicates. The WCL RT722 targets stay as separate TPLGS entries since they enable INCLUDE_SDW_JACK_WOV, which the PTL ones do not. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The hda-generic naming convention declares in case NHLT blob is included: hda-generoc-[DSP plaform]- PTL and WCL are ACE3, they use the same blob for NHLT, the name of the topology must be: sof-hda-generic-ace3-4ch-wov-multi.tplg Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
platform/intel/wcl.conf was a verbatim copy of ptl.conf and the WCL
manifests only differed from the PTL ones by PLATFORM="wcl". WCL is an
ACE3 platform exactly like PTL; the SOF topology layer does not need a
separate platform key for it, as the existing sof-wcl-dmic-*-id* symlinks
already show.
Remove wcl.conf and the "wcl" IncludeByKey.PLATFORM entries, delete the
duplicated WCL DMIC multi-WoV manifests and generate sof-wcl-dmic-wov-multi
and sof-wcl-dmic-wov-multi-4ch as symlinks to the PTL topologies instead.
The WCL RT722 targets were the only ones enabling INCLUDE_SDW_JACK_WOV;
PTL and WCL are identical from the topology's point of view, so enable
it on the PTL RT722 targets and turn the WCL ones into symlinks as well.
Update the README to match, including the stale
sof-{ptl,wcl}-hda-generic-4ch-wov-multi target names which were renamed
to sof-hda-generic-ace3-4ch-wov-multi earlier.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The ECNS high-pass filters support switching between boosted (+30 dB Pin 0,
+20 dB Pin 1) and 0 dB responses via the "ECNS capture boost" control.
Also support enabling test signal on KPB mono output pin (Pin 0).
In wov-multi topologies, instantiate unique per-instance controls
("DMIC ECNS boost" / "Jack In ECNS boost" and "DMIC ecns_kpb_test_signal" /
"Jack In ecns_kpb_test_signal") so a topology with both DMIC and Jack
inputs has distinct control names.
Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
…rase Detect ID When both DMIC and SoundWire headset WoV paths are instantiated in the same topology (e.g. rt722 wov-multi), both wov-arbiter widgets previously generated an ALSA mixer enum control with the identical name 'wov_active_slot', causing a naming collision in the mixer. Disambiguate the active slot controls per input path: - Change default wov-arbiter enum name to 'DMIC Phrase Detect ID'. - Override wov-arbiter enum name in sdw-jack-wov-multi.conf to 'Jack In Phrase Detect ID'. - Update integration guides and documentation accordingly. Co-developed-by: Naveen Manohar <naveen.m@intel.com> Signed-off-by: Naveen Manohar <naveen.m@intel.com> Co-developed-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com> Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Enhance sample keyword detector for multi-slot Wake-on-Voice testing: - Support multi-slot keyword detection with dedicated DP worker threads for 20ms audio batch processing. - Subscribe to NOTIFIER_ID_WOV_CTRL notifications from the WoV arbiter to pause and resume detection based on slot arbitration. - In detect_test.toml: increase instance count to 4. Co-developed-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Co-developed-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com> Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
… Lake Add Panther Lake ACE 3.0 Intel OTC debug key modulus and exponent mapping in sof_ri_info.py to support manifest and signature decoding on PTL binaries. Co-developed-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…i3 verification Document multi-slot Wake-on-Voice architecture and reproduction procedures: - Update tools/topology/topology2/README.md with comprehensive hardware reproduction guide, NHLT requirements, tinyalsa capture commands, and power benchmark procedures across PTL, WCL, and TGL. - Update top-level README.md referencing the WoV test documentation, required kernel/firmware branches, and 20/20 test results. Co-developed-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com> Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com> Co-developed-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Add single-detector SoundWire DMIC Wake-on-Voice topology support for Panther Lake with Realtek RT713 headset jack and RT1320 smart amplifiers. The capture pipeline downmixes SoundWire microphone audio through micsel, resamples to 16 kHz with SRC, and feeds a 2000 ms KPB ring buffer. KPB pin 0 feeds the detector copier directly, which converts the 32-bit samples to 16-bit for the DP MFCC and microWakeWord modules. KPB pin 1 feeds the WoV arbiter, which forwards the drained 32-bit history to the host capture copier on PCM 12. Also add the sof-sdw-wov feature topology build target with this pipeline, for dynamic feature loading alongside SoundWire base topologies such as sof-ptl-rt713-l3-rt1320-l12.tplg. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add a Kconfig choice COMP_MWW_MODEL to select the built-in wake word model. The default choice is COMP_MWW_MODEL_PIPELINE_ID, which preserves the default behavior of selecting models dynamically via pipeline ID (compiling all default model headers into the firmware image). Add Kconfig options to force a single built-in keyword model at build time, omitting all other model headers from the build to reduce firmware footprint: - COMP_MWW_MODEL_HI_INTEL (mww_model_data_hi_intel.h) - COMP_MWW_MODEL_HEY_JARVIS (mww_model_data_hey_jarvis.h) - COMP_MWW_MODEL_STRAWBERRY (mww_model_data_strawberry.h) - COMP_MWW_MODEL_ORANGE (mww_model_data_orange.h) - COMP_MWW_MODEL_BANANA (mww_model_data_banana.h) Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
…build Align intel_adsp_ace40_nvl.conf with the static build configuration of intel_adsp_ace30_ptl.conf. This enables the static modules needed for Wake-on-Voice and audio feature processing (WOV Arbiter, KPB, ECNS with test signal, MFCC with PCAN, microWakeWord with PCAN and debug trace, DRC, Multiband DRC, TDFB, Aria, MUX, S24_3LE converter). Disables HiFi4 nnlib acceleration for microWakeWord (CONFIG_COMP_MWW_HIFI_NNLIB=n) to fall back to portable TFLM kernels on the HiFi5 DSP core. Disables LLEXT and the dynamic library manager in favor of a static build. Expands heap and malloc arena sizes, sets early P4WQ init for core 0 IDC worker threads, sets 100ms IDC timeout, enables telemetry, and configures DMA LLI pool and L2 MMU tables while preserving Nova Lake specific settings. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Include module toml configurations for WOV arbiter, ECNS, and microWakeWord (MWW) when enabled by Kconfig or when forcing all modular builds. This allows rimage to populate the module entries in the firmware manifest for Nova Lake (NVL) and prevents firmware load failures. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add the lean SSP0 NoCodec Wake-on-Voice topology sof-ptl-nocodec-wov.tplg
(and sof-tgl-nocodec-wov.tplg) defined in cavs-nocodec-wov.conf based on
the single-slot WoV pipeline from sdw-dmic-wov-single.conf.
The topology provides:
- Playback (passthrough): PCM 0 ("Port0", 2ch, 48 kHz, S16_LE / S32_LE) ->
host-copier -> dai-copier -> SSP0 with LBM mode (quirk 64) enabled.
- Capture (Wake-on-Voice): SSP0 DAI copier (loopback, 48 kHz 2ch S16_LE) ->
micsel (stereo downmix to mono 48 kHz 16-bit) -> SRC (48 kHz to 16 kHz
mono 16-bit) -> KPB (2000 ms 16 kHz mono 16-bit).
- Detection slot: KPB pin 0 -> detector copier (16-bit pass-through) ->
MFCC (10 ms hop PCAN) -> microWakeWord (MWW) -> WOV Arbiter pin 1.
- History drain: KPB pin 1 -> WOV Arbiter pin 0 (16-bit) -> host-copier
(PCM 12, 16 kHz mono S16_LE, D0i3-compatible).
This enables automated stress tests and keyword recognition accuracy testing
by playing audio into the SSP0 playback stream and receiving the looped-back
signal on the keyword detection and WoV capture path.
Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
All builds, topologies and models use the int8 Mel features from MFCC, so the int32 Q9.23 input path with the in-component soft AGC and int8 requantization is not used anymore. Remove it together with the COMP_MWW_PCAN option that selected between the two paths. The soft AGC will be added to MFCC as an alternative int8 output mode to PCAN, so MWW only needs to copy the int8 features to the model. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
MWW accepts only int8 Mel features, so the MWW_PCAN=false choice of 32-bit Mel frames does not work anymore. Replace the true/false switch with MWW_FEATURES that names the int8 feature type from MFCC, for now only "pcan". This makes it possible to add other int8 feature types later. The generated WoV topologies are binary identical before and after the change. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add an output mode to the MFCC state that is set once in setup from the configuration blob: cepstral coefficients, int32 Mel or int8 PCAN Mel. The output preparation and the output sample size now use it instead of checking the PCAN enable flag under the Kconfig option in several places. This prepares adding more output data types. The output is bit-exact with the previous version for the cepstral, Mel and PCAN benchmark topologies in testbench. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add AGC8, an int8 Mel output mode that is a light alternative to PCAN. A single gain is added to all Q9.23 Mel log values of a hop. It attacks instantly to keep the hop peak at or below +1.0 and releases towards a target, slower when the VAD reports speech. The values are then clamped and converted to int8 Q1.7. It needs no per-channel noise estimate and no tflite-micro code, and is always built, so one MFCC build supports cepstral, int32 Mel, PCAN and AGC8 output. The algorithm is moved from the removed microwakeword int32 input path and is bit-exact with it. AGC8 is enabled with the new enable_agc8 blob flag and tuned with agc_target, agc_floor, agc_release and agc_release_speech, which take the previously reserved blob space. Zero values select the defaults, so the blob size and existing blobs are unchanged. Setup rejects a blob that enables both PCAN and AGC8, or either of them with cepstral output. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add the mel40_10ms_agc8_compress blob for microwakeword: 40 Mel bins with 10 ms hop, soft AGC int8 output, and VAD enabled for the AGC release and the MWW gating. The blob uses log10 Mel with scale 0.4, so the int8 range covers 50 dB below the AGC normalized peak, and a 3.0 gain target, so quiet input is boosted. The release in silence is 3 dB/s to recover quickly after loud sounds, and 0.1 dB/s during speech. With these settings the features of keywords in background noise are the same for input peak levels from -50 to -6 dBFS. Add the "agc8" choice for MWW_FEATURES in the WoV topologies. The default stays "pcan" and the WoV topologies are binary identical with it. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
EXPORT_SYMBOL() is defined in zephyr/llext/symbol.h. Without LLEXT nothing includes it indirectly, so a static build with the debug stream text messages fails. Include the header directly; the macro is a no-op when LLEXT is disabled. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The model output is a LOGISTIC with the fixed scale 1/256, so the probability is the raw output minus the zero point in Q0.8. Use that directly for the detection average, threshold and the live score instead of a float probability. The detection decisions and the logged values are identical to the float version. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Twister rejects the space separated tags string in the PCAN test case description, so the test could not be run. Use a YAML list like the other math unit tests. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The PCAN strength and offset were passed as floats only to be converted back to Q1.15 and Q8.7 inside the gain lookup function. The MFCC component in turn converted its Q1.15 and Q8.7 blob values to float to call the library. Change the PCAN configuration and the LUT computation API to take the fixed-point values directly. This removes the last floating point use from the MFCC and PCAN code. The output is bit-exact with the previous version. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The size field of the struct sof_abi_hdr in front of a model from the bytes control was used without checking it against the received blob size. A too large value made the model parser read beyond the blob. Reject the blob when the size exceeds the received payload. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The model blob was a bare TFLite flatbuffer with no information about the model. Add struct sof_mww_model_hdr in front of the flatbuffer, for both the built-in models and the bytes control. It has a version and header size for later additions, the model size and CRC32 that are checked before the model is used, the keyword name, the feature type and geometry that are checked against the component, and the trigger threshold and averaging length, so each model carries its own trigger tuning. The arena size field lets a model that needs more than the static tensor arena be rejected, and reserved words leave room for future use. A bare flatbuffer is still accepted and uses the previous default trigger settings. crc32() in numbers.c is exported for this, so that the check also works when the component is built as a loadable module. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Retrain the hi intel, strawberry, orange and banana models, and store all built-in models and sof-ctl blobs, including the upstream hey jarvis model, with the struct sof_mww_model_hdr model header. The models use int8 PCAN features and int8 streaming state, which removes 18 dequantize and quantize operations per inference and makes the models about 10 kB smaller. The trigger settings in the headers are threshold 0.5 and average of 4 probabilities. The hi intel model is trained with negative class weight 3.0 and about 6000 Mozilla Common Voice clips as continuous speech negatives. Keyword recall stays at 99.8 % while false triggers on the Common Voice clips drop from 59.4 % to 2.4 %. It triggers on one of the 11 held-out ODAQ dialog clips, where the previous model triggered on 10 of them. The strawberry and orange models use a new TTS speaker for every clip, the other fruit words as unknown speech, about 6000 Common Voice clips as speech negatives, and augmented recordings of a few real speakers as additional positives. On 120 held-out LibriTTS-R speakers at -40..-10 dBFS strawberry detection is 99-100 % and orange 96-100 %. Orange does not trigger on the other fruits in 960 clips and strawberry triggers on 10 of them. False triggers on the Common Voice clips are 0.3 % for strawberry and 1.4 % for orange. The banana model also uses Whisper screening of the TTS clips, which raises its detection from 82-86 % to 95-98 %. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add native_sim unit tests for the MFCC soft AGC int8 output: default and blob parameter init, Q1.7 conversion and rounding, attack with saturation, the gain floor, the silence and speech release steps and the release stop at the target gain. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
With IPC4 the kernel assigns its own pipeline IDs, so the MWW slot is derived from the module instance ID. A topology with a single detector gets instance 0 and slot 2, which selected the orange model. The single-slot SoundWire and NoCodec WoV topologies are meant to detect "hi intel". For SOF v2.15.1, swap the default models of slots 1 and 2 so that slot 2 uses hi intel and slot 1 uses orange. The three-slot DMIC topology still loads all three models, but the Phrase Detect ID control now reports Slot 2 for orange and Slot 3 for hi intel. A proper fix is to select the model from topology data instead of the instance ID. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The MFCC component calls FFT, DCT, matrix, auditory, window and PCAN functions from the math library, but none of them were exported. With CONFIG_COMP_MFCC=m the module could not resolve these symbols when it was loaded. Export them, along with the two TFLM microfrontend functions that the PCAN inline helpers call, from SOF code since the TFLM sources are not modified. Also add the missing mfcc_vad.c and mfcc_ipc4.c to the MFCC LLEXT build so the module contains the same code as the built-in version. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
singalsu
force-pushed
the
upstream_stable-v2.15_with_wov-upstream_backport
branch
from
November 9, 2026 15:47
e571733 to
d18bb66
Compare
kv2019i
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.