Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1849196
Summary: | [ARK] kernel bug list_del corruption on s390x from stress-ng mknod and stress-ng symlink | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Jeff Bastian <jbastian> | ||||||||||||||
Component: | kernel | Assignee: | Kernel Maintainer List <kernel-maint> | ||||||||||||||
Status: | NEW --- | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||||||||||||
Severity: | medium | Docs Contact: | |||||||||||||||
Priority: | medium | ||||||||||||||||
Version: | rawhide | CC: | acaringi, airlied, bskeggs, bugproxy, dan, dzickus, hdegoede, ichavero, itamar, jarodwilson, jeremy, jglisse, john.j5live, jonathan, josef, kernel-maint, lgoncalv, linville, masami256, mchehab, mjg59, prudo, rasibley, steved, tstaudt | ||||||||||||||
Target Milestone: | --- | ||||||||||||||||
Target Release: | --- | ||||||||||||||||
Hardware: | s390x | ||||||||||||||||
OS: | Linux | ||||||||||||||||
Whiteboard: | |||||||||||||||||
Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||||||||||
Doc Text: | Story Points: | --- | |||||||||||||||
Clone Of: | Environment: | ||||||||||||||||
Last Closed: | Type: | Bug | |||||||||||||||
Regression: | --- | Mount Type: | --- | ||||||||||||||
Documentation: | --- | CRM: | |||||||||||||||
Verified Versions: | Category: | --- | |||||||||||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||||||||||
Embargoed: | |||||||||||||||||
Bug Depends On: | |||||||||||||||||
Bug Blocks: | 467765 | ||||||||||||||||
Attachments: |
|
Description
Jeff Bastian
2020-06-19 19:40:49 UTC
Created attachment 1698160 [details]
5.7.2-88cd4de.cki-console.log.xz
serial console log from kernel 5.7.2-88cd4de.cki
Created attachment 1698163 [details]
5.7.4-4418f34.cki-console.log.xz
serial console log from kernel 5.7.4-4418f34.cki
Created attachment 1698164 [details]
5.7.4-9b26e20.cki-console.log.xz
serial console log from kernel 5.7.4-9b26e20.cki
Created attachment 1698165 [details]
5.7.3-bbd1511.cki-console.log.xz
serial console log from kernel 5.7.3-bbd1511.cki
Created attachment 1698166 [details]
5.7.4-inf.cki-console.log.xz
serial console log from kernel 5.7.4-inf.cki
Created attachment 1698167 [details]
5.8.0-rc1-c1f840d.cki-console.log.xz
serial console log from kernel 5.8.0-rc1-c1f840d.cki
The full trace from kernel 5.8.0-rc1-c1f840d.cki [ 1256.534428] list_del corruption. next->prev should be 000003e000be7c98, but was 00000001a7a6d1b0 [ 1256.534463] ------------[ cut here ]------------ [ 1256.534466] kernel BUG at lib/list_debug.c:54! [ 1256.534535] monitor event: 0040 ilc:2 [#1] SMP [ 1256.534540] Modules linked in: loop binfmt_misc psnap llc salsa20_generic camellia_generic cast6_generic cast_common serpent_generic twofish_generic twofish_common ofb lrw tgr192 wp512 rmd320 rmd256 rmd160 rmd128 md4 lcs ctcm fsm zfcp scsi_transport_fc dasd_fba_mod rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfslockd grace fscache sunrpc qeth_l2 qeth qdio ccwgroup vfio_ccw vfio_mdev mdev vfio_iommu_type1 vfio drm drm_panel_orientation_quirks backlight i2c_core ip_tables xfs libcrc32c crc32_vx_s390 ghash_s390 prng aes_s390 des_s390 libdes sha512_s390 sha256_s390 sha1_s390 sha_common dasd_eckd_mod dasd_mod pkey zcrypt [ 1256.534806] CPU: 2 PID: 582352 Comm: stress-ng-mknod Kdump: loaded Not tainted 5.8.0-rc1-c1f840d.cki #1 [ 1256.534810] Hardware name: IBM 2964 N96 400 (z/VM 6.4.0) [ 1256.534820] Krnl PSW : 0404e00180000000 000000009d9b173c (__list_del_entry_valid+0x8c/0xb8) [ 1256.534831] R:0 T:1 IO:0 EX:0 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI:0 EA:3 [ 1256.534833] Krnl GPRS: 0000000000000064 000000009e5eace8 0000000000000054 0000001f5042a08 [ 1256.534834] 00000001f5051800 0000000000000000 0000000190040180 00000001f1c86570 [ 1256.534836] 00000001f1c86620 070000009ddc1582 000003e000be7c80 00000001a7a6d1a8 [ 1256.534837] 00000001f11ac000 00000001e44f1400 000000009d9b1738 000003e000be7b30 [ 1256.534851] Krnl Code: 000000009d9b172c: e33010000004 lg %r3,0(%r1) [ 1256.534851] 000000009d9b1732: c0e5ffdda063 brasl %r14,000000009d5657f8 [ 1256.534851] #000000009d9b1738: af000000 mc 0,0 [ 1256.534851] >000000009d9b173c: b9040032 lgr %r3,%r2 [ 1256.534851] 000000009d9b1740: c020003251ab larl %r2,000000009dffba96 [ 1256.534851] 000000009d9b1746: c0e5ffdda059 brasl %r14,000000009d5657f8 [ 1256.534851] 000000009d9b174c: af000000 mc 0,0 [ 1256.534851] 000000009d9b1750: b9040032 lgr %r3,%r2 [ 1256.534866] Call Trace: [ 1256.534868] [<000000009d9b173c>] __list_del_entry_valid+0x8c/0xb8 [ 1256.534871] ([<000000009d9b1738>] __list_del_entry_valid+0x88/0xb8) [ 1256.534876] [<000000009d54d800>] remove_wait_queue+0x48/0xa0 [ 1256.535024] [<000003ff8019f3d0>] xfs_log_commit_cil+0x900/0xa50 [xfs] [ 1256.535056] [<000003ff80197704>] __xfs_trans_commit+0x9c/0x3a8 [xfs] [ 1256.535089] [<000003ff80189a9c>] xfs_remove+0x274/0x328 [xfs] [ 1256.535121] [<000003ff80183962>] xfs_vn_unlink+0x5a/0xa8 [xfs] [ 1256.535126] [<000000009d768874>] vfs_unlink+0x134/0x250 [ 1256.535128] [<000000009d76d00a>] do_unlinkat+0x1ba/0x318 [ 1256.535133] [<000000009ddc692c>] system_call+0xe0/0x2b0 [ 1256.535134] Last Breaking-Event-Address: [ 1256.810858] [<000000009ddc7b40>] __s390_indirect_jump_r14+0x0/0xc [ 1256.810925] ---[ end trace d4f63cd47d1c630e ]--- The full trace from kernel 5.7.2-88cd4de.cki [ 1754.761295] list_del corruption. prev->next should be 000003e000e27a68, but was 00000001d6daa1f0 [ 1754.880585] ------------[ cut here ]------------ [ 1754.880588] kernel BUG at lib/list_debug.c:51! [ 1754.880656] monitor event: 0040 ilc:2 [#1] SMP [ 1754.880662] Modules linked in: unix_diag binfmt_misc psnap llc salsa20_generic camellia_generic cast6_generic cast_common serpent_generic twofish_generic twofish_common ofb lrw tgr192 wp512 rmd320 rmd256 rmd160 rmd128 md4 loop tun af_kecrypto_user scsi_transport_iscsi xt_multiport overlay xt_CONNSECMARK xt_SECMARKnft_counter xt_state xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat nf_tables nfnetlink ah6 ah4 sctp lcs ctcm fsm zfcp scsi_transport_fc dasd_fba_mod rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache sunrpc qeth_l2 qeth qdio ccwgroup vfio_ccw vfio_mdev mdev vfio_iommu_type1 vfio drm drm_panel_orientation_quirks backlight i2c_core ip_tables xfs libcrc32c crc32_vx_s390 ghash_s390 prng aes_s390 des_s390 libdes sha512_s390 sha256_s390 sha1_s390 sha_common dasd_eckd_mod dasd_mod pkey zcrypt [ 1754.880738] CPU: 3 PID: 592107 Comm: stress-ng-symli Kdump: loaded Not tainted 5.7.2-88cd4de.cki #1 [ 1754.880740] Hardware name: IBM 2964 N96 400 (z/VM 6.4.0) [ 1754.880742] Krnl PSW : 0404e00180000000 0000000021651180 (__list_del_entry_valid+0xa0/0xb8) [ 1754.880754] R:0 T:1 IO:0 EX:0 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI:0 EA:3 [ 1754.880756] Krnl GPRS: 0000000000000064 000000002226fbe8 0000000000000054 00000001f509fa10 [ 1754.880757] 00000001f50ae408 0000000000000000 0000000184ca0d30 00000001f104aae0 [ 1754.880759] 00000001f104ab90 0700000021a58c52 000003e000e27a50 00000001d6daa1e8 [ 1754.880760] 0000000089acc000 00000001e4125c00 000000002165117c 000003e000e27900 [ 1754.880768] Krnl Code: 0000000021651170: c0200031e8a3 larl %r2,0000000021c8e2b6 [ 1754.880768] 0000000021651176: c0e5ffddf2d9 brasl %r14,000000002120f728 [ 1754.880768] #000000002165117c: af000000 mc 0,0 [ 1754.880768] >0000000021651180: b9040032 lgr %r3,%r2 [ 1754.880768] 0000000021651184: c0200031e87d larl %r2,0000000021c8e27e [ 1754.880768] 000000002165118a: c0e5ffddf2cf brasl %r14,000000002120f728 [ 1754.880768] 0000000021651190: af000000 mc 0,0 [ 1754.880768] 0000000021651194: 0707 bcr 0,%r7 [ 1754.880783] Call Trace: [ 1754.880786] [<0000000021651180>] __list_del_entry_valid+0xa0/0xb8 [ 1754.880789] ([<000000002165117c>] __list_del_entry_valid+0x9c/0xb8) [ 1754.880793] [<00000000211f77d8>] remove_wait_queue+0x48/0xa0 [ 1754.884725] [<000003ff801e2240>] xfs_log_commit_cil+0x900/0xa50 [xfs] [ 1754.884763] [<000003ff801da55c>] __xfs_trans_commit+0x9c/0x3a8 [xfs] [ 1754.884795] [<000003ff8015dfd8>] xfs_attr_try_sf_addname+0x68/0xc8 [xfs] [ 1754.884827] [<000003ff8015f09a>] xfs_attr_set_args+0x9a/0x128 [xfs] [ 1754.885084] [<000003ff8015f37e>] xfs_attr_set+0x1be/0x2f8 [xfs] [ 1754.885117] [<000003ff801c61e8>] xfs_initxattrs+0x98/0xb8 [xfs] [ 1754.885122] [<000000002157d542>] security_inode_init_security+0x152/0x160 [ 1754.885154] [<000003ff801c6144>] xfs_init_security+0x2c/0x38 [xfs] [ 1754.885187] [<000003ff801c7bc8>] xfs_vn_symlink+0xb0/0x1d0 [xfs] [ 1754.885190] [<000000002140d7ae>] vfs_symlink+0xfe/0x1c8 [ 1754.885192] [<000000002141020a>] do_symlinkat+0xa2/0xf8 [ 1754.885196] [<0000000021a5dfd0>] system_call+0xdc/0x2c8 [ 1754.885197] Last Breaking-Event-Address: [ 1754.885199] [<0000000021a5f560>] __s390_indirect_jump_r14+0x0/0xc [ 1754.885249] ---[ end trace 9e0dbe149edf1c8a ]--- Hanns-Joachim, can you mirror this for IBM BZ? ------- Comment From geraldsc.com 2020-06-22 06:41 EDT------- This look like a common code / xfs issue, and probably should be reported to xfs maintainer. No s390 code involved here. I also cannot reproduce this on my ext4 system, can you verify that this only shows with xfs? Does it also show on other architectures? The same tests pass fine on x86_64, ppc64le, and aarch64. It only fails on s390x, and fails fairly regularly. I'll try ext4 and see what happens. I ran xfs tests 4-times on an ext4 file system and could not reproduce the problem. |