Repository navigation
Conversation
Add west submanifest for TensorFlow Lite Micro (TFLM) to integrate ML inference directly into the SOF build tree and replace obsolete out-of-tree tensorflow-clone.sh script. Co-developed-by: Liam Girdwood <liam.r.girdwood@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com> Co-developed-by: Naveen Manohar <naveen.m@intel.com> Signed-off-by: Naveen Manohar <naveen.m@intel.com> Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add PCAN (Per-Channel AGC Normalization) fixed-point math library for normalizing mel filterbank energies. PCAN estimates per-channel noise background, applies non-linear wide-dynamic-range AGC compression with fixed-point LUT gain lookup, and delivers scaled features for streaming keyword-spotting networks. Include ztest test suite with MATLAB reference models covering gain lookup, noise estimation, dynamic shrinking, stream processing, and corner cases. Co-developed-by: Liam Girdwood <liam.r.girdwood@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com> Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The testbench build enables MFCC with PCAN, which compiles the PCAN gain control, noise reduction and log scale sources from the TFLM microfrontend library. The testbench workflow only checks out the SOF repository, so these sources are missing and the build fails. Add a step that fetches only the microfrontend library directory of tflite-micro next to sof/, at the revision pinned in submanifests/tflm.yml, using a shallow sparse checkout. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Extend MFCC (Mel-Frequency Cepstral Coefficients) audio component with 8-bit Per-Channel AGC Normalization (PCAN) feature extraction: - Add CONFIG_COMP_MFCC_PCAN Kconfig option and CMake configuration. - Implement 8-bit quantized PCAN feature extraction paths alongside standard 16-bit cepstral outputs. - Add Topology2 component definition macros for 40-mel and 80-mel PCAN compressed feature extraction. - Lower high-frequency PCAN tracing to debug level to prevent IPC trace buffer saturation during continuous listening. Co-developed-by: Liam Girdwood <liam.r.girdwood@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com> Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add an output mode to the MFCC state that is set once in setup from the configuration blob: cepstral coefficients, int32 Mel or int8 PCAN Mel. The output preparation and the output sample size now use it instead of checking the PCAN enable flag under the Kconfig option in several places. This prepares adding more output data types. The output is bit-exact with the previous version for the cepstral, Mel and PCAN benchmark topologies in testbench. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add AGC8, an int8 Mel output mode that is a light alternative to PCAN. A single gain is added to all Q9.23 Mel log values of a hop. It attacks instantly to keep the hop peak at or below +1.0 and releases towards a target, slower when the VAD reports speech. The values are then clamped and converted to int8 Q1.7. It needs no per-channel noise estimate and no tflite-micro code, and is always built, so one MFCC build supports cepstral, int32 Mel, PCAN and AGC8 output. The algorithm is moved from the removed microwakeword int32 input path and is bit-exact with it. AGC8 is enabled with the new enable_agc8 blob flag and tuned with agc_target, agc_floor, agc_release and agc_release_speech, which take the previously reserved blob space. Zero values select the defaults, so the blob size and existing blobs are unchanged. Setup rejects a blob that enables both PCAN and AGC8, or either of them with cepstral output. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Twister rejects the space separated tags string in the PCAN test case description, so the test could not be run. Use a YAML list like the other math unit tests. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The PCAN strength and offset were passed as floats only to be converted back to Q1.15 and Q8.7 inside the gain lookup function. The MFCC component in turn converted its Q1.15 and Q8.7 blob values to float to call the library. Change the PCAN configuration and the LUT computation API to take the fixed-point values directly. This removes the last floating point use from the MFCC and PCAN code. The output is bit-exact with the previous version. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add native_sim unit tests for the MFCC soft AGC int8 output: default and blob parameter init, Q1.7 conversion and rounding, attack with saturation, the gain floor, the silence and speech release steps and the release stop at the target gain. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The MFCC component calls FFT, DCT, matrix, auditory, window and PCAN functions from the math library, but none of them were exported. With CONFIG_COMP_MFCC=m the module could not resolve these symbols when it was loaded. Export them, along with the two TFLM microfrontend functions that the PCAN inline helpers call, from SOF code since the TFLM sources are not modified. Also add the missing mfcc_vad.c and mfcc_ipc4.c to the MFCC LLEXT build so the module contains the same code as the built-in version. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.