Repository navigation
[BUG][APL] "Input/output error" occurred when do multiple-pipeline-capture test. #2749
Description
Describe the bug
"Input/output error" occurred when multiple-pipeline-capture test. After error occured,all the pipeline can not work,reboot can recovery.
Used ubuntu@sh-apl-up2-pcm512x-02:
ubuntu@sh-apl-up2-pcm512x-02:~$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: sofpcm512x [sof-pcm512x], device 0: Port5 (*) []
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: sofpcm512x [sof-pcm512x], device 1: HDMI1 (*) []
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: sofpcm512x [sof-pcm512x], device 2: HDMI2 (*) []
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: sofpcm512x [sof-pcm512x], device 3: HDMI3 (*) []
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: sofpcm512x [sof-pcm512x], device 4: Media Playback 4 (*) []
Subdevices: 1/1
Subdevice #0: subdevice #0
card 1: Device [USB Audio Device], device 0: USB Audio [USB Audio]
Subdevices: 1/1
Subdevice #0: subdevice #0
ubuntu@sh-apl-up2-pcm512x-02:~$ arecord -l
**** List of CAPTURE Hardware Devices ****
card 0: sofpcm512x [sof-pcm512x], device 0: Port5 (*) []
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: sofpcm512x [sof-pcm512x], device 5: DMIC (*) []
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: sofpcm512x [sof-pcm512x], device 6: DMIC16kHz (*) []
Subdevices: 1/1
Subdevice #0: subdevice #0
card 1: Device [USB Audio Device], device 0: USB Audio [USB Audio]
Subdevices: 1/1
Subdevice #0: subdevice #0
To Reproduce
1.sudo reboot
2.cd sof-test
3.cd test-case
4.export TPLG=/lib/firmware/intel/sof-tplg/sof-apl-pcm512x.tplg
5.. ./multiple-pipeline-capture.sh -c 4 -l 50 (https://github.com/thesofproject/sof-test/blob/master/test-case/multiple-pipeline-capture.sh)
Reproduction Rate
Test for 50 times, failed at 10th time.
Expected behavior
There is no error occured when do multiple-pipeline-capture test.
Impact
"Input/output error" occurred when do multiple-pipeline-capture test.
Platform: APL UP2 with I2S PCM512x
Firmware: 939c14 https://github.com/thesofproject/sof/commits/master
Kernel: 77b49b3 https://github.com/thesofproject/linux/commits/topic/sof-dev
Topology: file: tools/topology/sof-apl-pcm512x.tplg same as firmware
no error info in dmesg:
sof_error_trace9.log
sof_trace9.log
dmesg9.log
Activity
sinahuang commentedon Apr 9, 2020
Bisect result:Caused by firmware commit:8b7f1fa
Liviali155 commentedon Apr 9, 2020
related with #2735
lgirdwood commentedon Apr 9, 2020
Seems related to slave SSP support which can be fixed for v1.6
lgirdwood commentedon Apr 9, 2020
@plbossart fyi.
YvonneYang2 commentedon May 27, 2020
Issue cannot be reproduced with latest sof-dev(commit: 866e4d5) and master(commit: 6cfd3e5) .
So close it.
Liviali155 commentedon May 28, 2020
Error occured again with latest sof-dev(0d22f66)+Master(4689780) when run Pause/Release: simultaneous playback/capture - 100x in CI stress test
1.sudo reboot
2.cd sof-test
3.cd test-case
4.export TPLG=/lib/firmware/intel/sof-tplg/sof-apl-pcm512x.tplg
5. ./multiple-pause-resume.sh -r 25
dmesg_error.log
sof_error.log
dmesg.txt
etrace.txt
slogger.txt
Reopen it
lgirdwood commentedon May 29, 2020
DMIC is not getting emptied fast enough. @mmaka1 @tlauda do we need more overhead for some DMA buffers (an extra period here) ?
YvonneYang2 commentedon Jun 8, 2020
Issue can still be reproduced as #2749 (comment) said on latest topic/sof-dev(1af84d0) and master(428804e)
Liviali155 commentedon Jul 7, 2020
On TGL chromebook with codec 98357 in I2S mode also can reproduce this issue when do multiple-pipeline-playback test
after error occured,all the pipeline can work
1.sudo reboot
2.cd sof-test
3.cd test-case
4.export TPLG=sof-tgl-max98357a-rt5682.tplg
5../multiple-pipeline-playback.sh -c 8 -l 300
Failed at 2th when arecord via headset(0,1)
dmesg_error.log
sof_error.log
dmesg0707.log
sof-logger_0707.log
Branch name and commit hash of the 2 repositories: sof (firmware/topology) and linux (kernel driver).
Kernel: Kernel:https://github.com/thesofproject/linux/commits/topic/sof-dev (commit: 1b2f260)
+PR#2238 for max98373-sdw jack switch
SOF: https://github.com/thesofproject/sof/tree/tgl-007-drop-stable v7.0-rc6,commit: a40b46c
Name of the topology file
Topology: sof-tgl-max98357a-rt5682.tplg
Name of the platform(s) on which the bug is observed.
Platform: TGL chromebook with codec 98357 in I2S mode
5 remaining items
ClarexZhou commentedon Jul 14, 2020
move to P1 since TGL also failed.
Liviali155 commentedon Jul 14, 2020
The issue of tgl has creat another issue #3177 to track.this issue is only used to trace apl issue
[-][BUG][APL/TGL] "Input/output error" occurred when do multiple-pipeline-capture test.[/-][+][BUG][APL] "Input/output error" occurred when do multiple-pipeline-capture test.[/+]Liviali155 commentedon Jul 24, 2020
Issue still can be reproduced with latest sof-dev(e1a25077)+master(44a65d8)on apl pcm512 when run multiple-pipeline-capture
dmesg_error.log
sof_error.log
dmesg0724.log
sof-logger0724.log
plbossart commentedon Jul 29, 2020
@Liviali155 I get very different results depending on which scripts are used on APL-Up2 w/ PCM512x
firmware: #3245
kernel: topic/sof-dev b31892bc2404c4ecc59d863a1d5cadba6740be5b
export TPLG=/lib/firmware/intel/sof-tplg/sof-apl-pcm512x.tplg
/multiple-pipeline-capture.sh -c 4 -l 50
-> no issues, seems to work fine (no idea what the -c 4 option means, removing it works fine as well). I could not make this test break with my setup.
./multiple-pause-resume.sh
no IPC errors in dmesg but same dmic_irq_handler issue. 100% reproducible problem, this never worked on my side.
Can you give it a try on your side and try to narrow to a specific problem? This issue has too many details on different tests, we can't make progress if we don't focus on a common/easy to reproduce issue.
@singalsu @ranj063 @marc-hb FYI if this rings a bell.
ClarexZhou commentedon Jul 29, 2020
@plbossart Verified with PR #3245 , issue still can be reproduced.
plbossart commentedon Jul 29, 2020
@ClarexZhou there's got to be some device variation, I just reran the same test with 100 iterations and it works fine for me.
plbossart commentedon Jul 29, 2020
@ClarexZhou do you actually run with the latest sof-test code? I don't have the same traces at all, I don't see the [COMMAND] parts, so wondering if we are aligned on the tools?
ClarexZhou commentedon Jul 30, 2020
Thanks. Error was reproduced because of not switching sof-test branch. Error not seen after using latest Master sof-test. Close.