Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1509162
Summary: | Failing HTM tbegin for z Series guests despite claiming support. | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Dan Horák <dan> | ||||||||
Component: | kernel | Assignee: | Kernel Maintainer List <kernel-maint> | ||||||||
Status: | CLOSED CURRENTRELEASE | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||||||
Severity: | unspecified | Docs Contact: | |||||||||
Priority: | unspecified | ||||||||||
Version: | 27 | CC: | admiller, airlied, ajax, aoliva, arjun.is, brueckner, bskeggs, bugproxy, codonell, dan, dj, ewk, extras-qa, fweimer, gmarr, hannsj_uhl, hdegoede, herrold, ichavero, itamar, jakub, jarodwilson, jeremy, jglisse, john.j5live, jonathan, josef, kernel-maint, law, linville, mboddu, mchehab, mfabian, mjg59, pfrankli, siddhesh, steved | ||||||||
Target Milestone: | --- | Keywords: | Patch, Tracking | ||||||||
Target Release: | --- | ||||||||||
Hardware: | s390x | ||||||||||
OS: | Linux | ||||||||||
Whiteboard: | |||||||||||
Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||||
Doc Text: | Story Points: | --- | |||||||||
Clone Of: | 1499260 | Environment: | |||||||||
Last Closed: | 2018-03-23 16:49:37 UTC | Type: | Bug | ||||||||
Regression: | --- | Mount Type: | --- | ||||||||
Documentation: | --- | CRM: | |||||||||
Verified Versions: | Category: | --- | |||||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||||
Embargoed: | |||||||||||
Bug Depends On: | 1499260 | ||||||||||
Bug Blocks: | 467765 | ||||||||||
Attachments: |
|
Description
Dan Horák
2017-11-03 08:53:39 UTC
Cloning the original bug to allow further investigation. Created attachment 1347225 [details]
full boot log
Boot log from Rawhide installation started from zipl from a previous installation.
When the same glibc and kernel as in the installer are on installed, there is no such crash as when the installer is run. Installing kernel after glibc ensures that the new glibc is included in the boot initramfs.
Switching to kernel, because - the interruption code 0013 means a special-operation exception and PoP says for TBEGIN: "2. A special-operation exception is recognized and the operation is suppressed if the transactional-execution control, bit 8 of control register 0, is zero. - the content or CR0 can be confirmed with 00: d xg0 00: CRG 0 = 0000000014846A12 after catching the exception with a trace (TR prog 13) 00: -> 0000000001000582' TBEGIN E5600000FF0E 0000000000000000 00: *** 0000000001000582' PROG 0013 -> 00000000008EF144" 00: SPECIAL OPERATION The problem can be reproduced with something like #include <stdio.h> int tbegin() { int ret; __asm__ __volatile__ ( " tbegin 0, 0xFF0E\n\t" " jnz 0f\n\t" " lhi %0, 0\n\t" " j 1f\n\t" "0: ipm %0\n\t" " srl %0, 28\n\t" "1:" :"=&d" (ret):: ); return ret; } void tend() { __asm__ ( " tend\n\t" ::: ); } int main(void) { int t; t = tbegin(); printf("tbegin=%d\n", t); tend(); return 0; } linked staticly (or dynamicly) and used as /init CR0 content from another guest in the same z/VM instance [sharkcz@devel11 ~]$ sudo vmcp d xg0 [sudo] password for sharkcz: CRG 0 = 0080000014846A12 I suspect the kernel doesn't explicitly enable TE (under some conditions) when it runs on machine with TE available in the facilities bits (like our zEC12 with zVM 6.4.0). PPC kernel added recently a command line option to disable transactional memory - see http://patchwork.ozlabs.org/patch/824763/ It might be useful for s390x to follow the same idea. ------- Comment From h.carstens.com 2017-11-08 16:38 EDT------- (In reply to comment #7) > PPC kernel added recently a command line option to disable transactional > memory - see http://patchwork.ozlabs.org/patch/824763/ > It might be useful for s390x to follow the same idea. Yes, indeed. This bug has proven that would be useful. Created attachment 1349586 [details]
test patch against v4.13
------- Comment on attachment From h.carstens.com 2017-11-08 16:41 EDT-------
I had just a short look into the code and suspect the missing initial enabling of the transactional execution facility early at startup to be the culprit.
Later on the corresponding control register bit will be set at every task switch.
Could you please try this, and let me know if it fixes the problem?
The patch is against vanilla v4.13 but should easily apply to your kernel sources.
yes, the problem is away, both with minimal test case from comment #3 and also with Rawhide installer images recreated with an updated kernel. Shouldn't something like this be applied as well? diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c index 164a1e16b53e..ad6e4a4c65dc 100644 --- a/arch/s390/kernel/setup.c +++ b/arch/s390/kernel/setup.c @@ -764,7 +764,7 @@ static int __init setup_hwcaps(void) /* * Transactional execution support HWCAP_S390_TE is bit 10. */ - if (test_facility(50) && test_facility(73)) + if (test_facility(50) && test_facility(73) && __ctl_get_bit(0, 55)) elf_hwcap |= HWCAP_S390_TE; /* And how about KVM or qemu? Or arch/s390/kernel/ptrace.c (it checks for MACHINE_HAS_TE)? Would you have an explanation why we saw the problem only in a installation, but did not in already installed guests? ------- Comment From h.carstens.com 2017-11-09 07:08 EDT------- (In reply to comment #10) > yes, the problem is away, both with minimal test case from comment #3 and > also with Rawhide installer images recreated with an updated kernel. Thanks for testing. After looking a bit deeper into the code, it looks incomplete however. I will mark the current patch obsolete and a new one. There are more bugs with respect to transactional exection. > Shouldn't something like this be applied as well? > * Transactional execution support HWCAP_S390_TE is bit 10. > */ > - if (test_facility(50) && test_facility(73)) > + if (test_facility(50) && test_facility(73) && __ctl_get_bit(0, 55)) > elf_hwcap |= HWCAP_S390_TE; As a cleanup this should be converted to something like if (MACHINE_HAS_TE) But that doesn't fix any bug. > And how about KVM or qemu? Or arch/s390/kernel/ptrace.c (it checks for > MACHINE_HAS_TE)? KVM + QEMU are fine. This a "simple" kernel bug. > Would you have an explanation why we saw the problem only in a installation, but did not > but did not in already installed guests? Yes, as soon as the system runs, control register 0 is updated on every task switch and the facility gets enabled (if the following process is not a kernel thread). So you will see this behavior only on startup. Unless you use ptrace requests which disable it again... which is yet another bug. See new patch. ------- Comment From h.carstens.com 2017-11-09 07:11 EDT------- Could you give the new patch also a try, please? Created attachment 1349861 [details]
For more bugs within transactional execution
------- Comment on attachment From h.carstens.com 2017-11-09 07:10 EDT-------
There are several bugs with control register handling with respect to
transactional execution:
- on task switch update_per_regs() is only called if the next task has
an mm (is not a kernel thread). This however is incorrect. This
breaks e.g. for user mode helper handling, where the kernel creates
a kernel thread and then execve's a user space program. Control
register contents related to transactional execution won't be
updated on execve. If the previous task ran with transactional
execution disabled then the new task will also run with
transactional execution disabled, which is incorrect. Therefore call
update_per_regs() unconditionally within switch_to().
- on startup the transactional execution factility is not enabled for
the idle thread. This is not really a bug, but an inconsistency to
other facilities. Therefore enable the facility if it is available.
- on fork the new thread's per_flags field is not cleared. This means
that a child process inherits the PER_FLAG_NO_TE flag. This flag can
be set with a ptrace request to disable transactional execution for
the current process. It should not be inherited by new child
processes in order to be consistent with the handling of all other
PER related debugging option. Therefore clear the per_flags field in
copy_thread_tls().
> ------- Comment From h.carstens.com 2017-11-09 07:11 EDT-------
> Could you give the new patch also a try, please?
yes, all looks still good with the new patch
And from the "Cc: <stable.org> # v3.7+" note in the patch, I assume we need the fix in RHEL-7 kernel as well, right? Hanns, will you manage it from the IBM side for the whole enterprise part, or shall I initiate the process by cloning this bug to RHEL? ------- Comment From h.carstens.com 2017-11-10 05:32 EDT------- Yes, a backport to RHEL-7 is also needed. (In reply to Dan Horák from comment #12) > And from the "Cc: <stable.org> # v3.7+" note in the patch, I > assume we need the fix in RHEL-7 kernel as well, right? > > Hanns, will you manage it from the IBM side for the whole enterprise part, > or shall I initiate the process by cloning this bug to RHEL? . ... Dan, I will check at our side how to proceed for RHEL ... (In reply to IBM Bug Proxy from comment #10) > Created attachment 1349861 [details] > For more bugs within transactional execution > > . fyi ... the upstream git commit in the s390 tree is https://git.kernel.org/pub/scm/linux/kernel/git/s390/linux.git/commit/?h=features&id=a1c5befc1c24eb9c1ee83f711e0f21ee79cbb556 ("s390: fix transactional execution control register handling") . and merged to mainline as https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a1c5befc1c24eb9c1ee83f711e0f21ee79cbb556 We apologize for the inconvenience. There is a large number of bugs to go through and several of them have gone stale. As kernel maintainers, we try to keep up with bugzilla but due the rate at which the upstream kernel project moves, bugs may be fixed without any indication to us. Due to this, we are doing a mass bug update across all of the Fedora 27 kernel bugs. Fedora 27 has now been rebased to 4.15.3-300.f27. Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel. If you experience different issues, please open a new bug report for those. *********** MASS BUG UPDATE ************** This bug is being closed with INSUFFICIENT_DATA as there has not been a response in 4 weeks. If you did actually update, we apologize for the inconvenience (there are a lot of bugs). If you are still experiencing this issue, please reopen and attach the relevant data from the latest kernel you are running and any data that might have been requested previously. |