Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1510649

Summary: Touchpad not detected on an Asus GL702ZC laptop (possible i2c driver issue)
Product: [Fedora] Fedora Reporter: Hein-Pieter van Braam <hp>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: airlied, bskeggs, btissoir, ewk, hdegoede, ichavero, itamar, jarodwilson, jeremy, jglisse, john.j5live, jonathan, josef, kernel-maint, linville, l.wagner, mchehab, mjg59, steved, turgut
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: kernel-4.16.10-300.fc28 kernel-4.16.11-200.fc27 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-05-23 15:41:32 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
lspci output
none
dmesg output
none
dmidecode output
none
lscpu output
none
i2c debugging output leading up to the problem
none
i2c rmmod/modprobe failure
none
kernel .config not working
none
kernel .config working
none
differences between fedora 4.16.8-300.fc28.x86_64 and braams kernels
none
Mr. Braam's kernel config
none
dmesg of the test kernel
none
output of alsa-info.sh none

Description Hein-Pieter van Braam 2017-11-07 22:11:16 UTC
Description of problem:


Version-Release number of selected component (if applicable):

Fedora 26
Fedora 27
Fedora Rawide

How reproducible:

Boot into Fedora on a GL702ZC laptop

Actual results:

Touchpad not detected

Expected results:

Touchpad detected

Additional info:

evemu-record does not show the touchpad at all. There is a kernel backtrace in dmesg. I've attached the output

Comment 1 Hein-Pieter van Braam 2017-11-07 22:12:07 UTC
Created attachment 1349138 [details]
lspci output

Comment 2 Hein-Pieter van Braam 2017-11-07 22:12:44 UTC
Created attachment 1349139 [details]
dmesg output

Comment 3 Hein-Pieter van Braam 2017-11-07 22:13:33 UTC
Created attachment 1349141 [details]
dmidecode output

Comment 4 Hein-Pieter van Braam 2017-11-07 22:13:59 UTC
Created attachment 1349142 [details]
lscpu output

Comment 5 Hein-Pieter van Braam 2017-11-08 01:25:21 UTC
I have a little bit more information: I built linux 4.13.11 from https://github.com/M-Bab/linux-kernel-amdgpu/tree/mbab_4.13 and enabled i2c debugging. I see what appears to be normal output and the touchpad works but occasionally i2c output from i2c_designware AMDI0010:03 stops completely and the touchpad stops working. If I rmmod and modprobe i2c_hid the touchpad starts working again for a while.

I see no errors leading up to the touchpad stopping working.

Comment 6 Hein-Pieter van Braam 2017-11-08 08:34:47 UTC
Created attachment 1349305 [details]
i2c debugging output leading up to the problem

Comment 7 Hein-Pieter van Braam 2017-11-08 08:36:50 UTC
Created attachment 1349318 [details]
i2c rmmod/modprobe failure

Comment 8 Hein-Pieter van Braam 2017-11-08 08:38:16 UTC
I've added dmesg output for the i2c debug output as well as an instance in which rmmod/modprobe of the i2c_hid module did not result in a working touchpad again.

When the touchpad/i2c/whichever component this is gets in this state only a hard reboot seems to fix it.

Comment 9 Benjamin Tissoires 2017-11-08 13:36:30 UTC
Thanks for the report. Hans had a similar issue with an other cheap i2c-hid chip. See https://patchwork.kernel.org/patch/10046575/. I wonder if you apply the same quirk, your device may just work.

Comment 10 Hein-Pieter van Braam 2017-11-08 23:27:12 UTC
I just built 4.14.0-rc8 and the touchpad is now also detected and works. I haven't had a random stop yet. I'll update the ticket if it hasn't happened for a day or two. Or sooner if it does happen. I still have i2c debugging enabled.

Comment 11 Hein-Pieter van Braam 2017-11-10 02:10:02 UTC
So I have some results. The problem didn't reappear with the config I cobbled together from various sources. I decided to do a config more similar to the default Fedora .config and the problem reappeared. I haven't yet figured out what settings cause the problem to appear. 

For now I'll attach both the working and non-working .config, perhaps someone can do a diff and have an a-ha moment. I'll go through the ones that seem relevant to me and try it though.

Comment 12 Hein-Pieter van Braam 2017-11-10 02:11:23 UTC
Created attachment 1350251 [details]
kernel .config not working

Comment 13 Hein-Pieter van Braam 2017-11-10 02:12:00 UTC
Created attachment 1350252 [details]
kernel .config working

Comment 14 Hein-Pieter van Braam 2017-11-20 16:07:38 UTC
I've heard from another user of this same laptop that 4.14.0 with the default Ubuntu configs don't suffer from this problem. I'm still trying to work out what the exact issue is. There are a lot of differences :)

Comment 15 Hein-Pieter van Braam 2017-12-05 22:21:12 UTC
I spent a little more time on this issue now. I've found some things:

* Disabling CONFIG_X86_AMD_PLATFORM_DEVICE in my working .config results in a dmesg message that is the same as with the default Fedora .config.
* Enabling CONFIG_X86_AMD_PLATFORM_DEVICE in the Fedora .config by itself makes the dmesg error about i2c_dw_clk_rate+0x1b/0x20 listed above go away but the touchpad is still not moving.

There is no mention of the ELAN driver in dmesg, however:
$ ls -l /sys/bus/i2c/devices/i2c-0/
total 0
--w-------. 1 root root 4096 Dec  5 23:18 delete_device
lrwxrwxrwx. 1 root root    0 Dec  5 23:18 device -> ../../AMDI0010:03
lrwxrwxrwx. 1 root root    0 Dec  5 23:18 firmware_node -> ../../../LNXSYSTM:00/LNXSYBUS:00/AMDI0010:03
drwxr-xr-x. 3 root root    0 Dec  6  2017 i2c-ELAN1200:00
-r--r--r--. 1 root root 4096 Dec  5 23:18 name
--w-------. 1 root root 4096 Dec  5 23:18 new_device
drwxr-xr-x. 2 root root    0 Dec  5 23:18 power
lrwxrwxrwx. 1 root root    0 Dec  6  2017 subsystem -> ../../../../bus/i2c
-rw-r--r--. 1 root root 4096 Dec  5 23:18 uevent

So there is something being detected now. There are many more configuration differences to check though.

Comment 16 Lukas Wagner 2017-12-21 19:46:41 UTC
Anyone made any progress?

Comment 17 Hein-Pieter van Braam 2017-12-22 10:54:51 UTC
Sorry, I haven't been spending huge amounts of time on this. I haven't yet located the exact configuration change between the .config I use and the one used by Fedora which causes the touchpad to work besides CONFIG_X86_AMD_PLATFORM_DEVICE being required.

Comment 18 Lukas Wagner 2017-12-22 15:34:07 UTC
Yesterday I downloaded both configs and put them besides each other using meld. In meld you can use the search tool. Searching for "I2C" revealed some major differences. I will do a diff search for "ELAN", too. Maybe I can get a difftool to print out a patch.

Comment 19 Lukas Wagner 2017-12-25 22:31:15 UTC
I'm pretty sure it's not a config issue. I think they have fiddled with the sources over at canonical. My notebook has been pretty active for a few days now, I have tried a few dozens of configs based on configs merged from vanilla and the working one you uploaded. Unfortunately for me the working one does not work. I do not get any graphics support. Not even plain tty, so it's just uninitialized stuff on screen. Anyway no luck here and I'm positive it's not a config related issue, since I copied over anything I can think of affecting bus systems and hid drivers related to I2C and ELAN input. Modprobing doesn't change anything either.
If I find the time I will go to a LUG and put up a challenge :D

Comment 20 Lukas Wagner 2017-12-29 15:36:04 UTC
I finally got it to work.
Today I stumbled upon this git repo: https://notabug.org/hp/linux-gl702zc
Whoever this guy is: I owe you a beer or two. It's as easy as compile, install reboot. It comes complete with working config. Very well done. I posted this in case someone stumbled upon this and for the kernel devs to check out a running version. Just to be clear: this is not an ubuntu based kernel. This kernel pretty much behaves just like a RHEL kernel. Including selinux.
The only downside was that I had to put in the original Realtek wifi adpter, since the kernel in the repo had no drivers for my newly bought intel chip on board. This would be a job for the future once I'm done celebrating hat my notebook is a self contained unit at last :D

Comment 21 Hein-Pieter van Braam 2017-12-31 13:42:49 UTC
Yeah, that's mine :) No patches other than the k10temp stuff. Still have no idea what is causing this. I'm beginning to suspect some of the more arcane base address or alignment values.

Comment 22 Hein-Pieter van Braam 2018-04-10 21:02:45 UTC
This bug is still present in the Fedora 28 stock kernel. I still don't know what combination of settings makes it work and what doesn't. I'm almost certainly missing something rather obvious. I'll try to spend some more time on this soon. I haven't had much incentive since my custom kernels seem to work well, but that's not really helpful for the wider Fedora community :)

Comment 23 Lukas Wagner 2018-04-11 09:59:35 UTC
Depending on where you're located we could meet up in May and work on that together for a weekend or so. This month I'm occupied with writing my thesis.

Comment 24 Turgut Kalfaoglu 2018-04-11 19:27:28 UTC
Just a quick note to say that I, too have the same laptop, HAD the same issues until I compiled the kernel provided by Mr. Braam at https://notabug.org/hp/linux-gl702zc and everything works great now.. Wireless and touchpad being the major issues. Likewise for brightness control; that works too.

Comment 25 Turgut Kalfaoglu 2018-05-18 06:42:30 UTC
Happy anniversary for this bug!
Can it be fixed before the laptop goes out of fashion?

PS: I'm a linux user and a C programmer. Can I assist in fixing it?
Please let me know how to proceed..

Comment 26 Hans de Goede 2018-05-18 07:51:55 UTC
(In reply to Turgut Kalfaoglu from comment #25)
> Happy anniversary for this bug!
> Can it be fixed before the laptop goes out of fashion?
> 
> PS: I'm a linux user and a C programmer. Can I assist in fixing it?
> Please let me know how to proceed..

You can help by comparing the .config file which defines all the CONFIG_KERNEL_OPTION_FOO= values for building the kernel between the Fedora kernels and Mr. Braam's kernels and then if you see something which is "suspect" which is enabled in Fedora and not in Mr. Braam's kernels, or the other way around change that in the Fedora .config, build a new kernel with that config and test, until you've found the culprit Kconfig option.

Comment 27 Turgut Kalfaoglu 2018-05-18 11:34:21 UTC
Thank you!  There are lots of changes unfortunately..
The best thing I found was the "beyond compare" to compare these two.
I'm attaching the result and Mr. Braam's config as well..

Comment 28 Turgut Kalfaoglu 2018-05-18 11:35:32 UTC
Created attachment 1438488 [details]
differences between fedora 4.16.8-300.fc28.x86_64 and braams kernels

Comment 29 Turgut Kalfaoglu 2018-05-18 11:36:06 UTC
Created attachment 1438489 [details]
Mr. Braam's kernel config

Comment 30 Hans de Goede 2018-05-18 14:54:51 UTC
Ok, I think I've found the issue, Fedora kernels are not selecting CONFIG_X86_AMD_PLATFORM_DEVICE which is necessary for the i2c-controllers on recent AMD platforms to get enumerated.

I've started a test kernel build with this enabled here:
https://koji.fedoraproject.org/koji/taskinfo?taskID=27041524

(note this will take a couple of hours to finish).

Once finish, please follow these generic instructions for testing kernels directly from koji:
https://fedorapeople.org/~jwrdegoede/kernel-test-instructions.txt

And let me know if the test kernel fixes things.

Comment 31 Hein-Pieter van Braam 2018-05-18 15:21:36 UTC
I noted that in https://bugzilla.redhat.com/show_bug.cgi?id=1510649#c15. Only enabling the AMD platform device is not by itself sufficient to make the touchpad function.

Comment 32 Turgut Kalfaoglu 2018-05-18 15:32:12 UTC
Wonderful, thank you very much, you are a genious :)

Comment 33 Hans de Goede 2018-05-18 15:40:39 UTC
(In reply to Hein-Pieter van Braam from comment #31)
> I noted that in https://bugzilla.redhat.com/show_bug.cgi?id=1510649#c15.
> Only enabling the AMD platform device is not by itself sufficient to make
> the touchpad function.

Ah I missed that, still we have a bit newer base kernel now, so might be worth a shot, also dmesg output with the test kernel I build would be appreciated / useful.

Comment 34 Turgut Kalfaoglu 2018-05-18 18:56:32 UTC
The touchpad still cannot be enabled unfortunately..
I'm enclosing a dmesg..

Comment 35 Turgut Kalfaoglu 2018-05-18 18:57:07 UTC
Created attachment 1438768 [details]
dmesg of the test kernel

Comment 36 Hans de Goede 2018-05-18 19:22:14 UTC
(In reply to Turgut Kalfaoglu from comment #34)
> The touchpad still cannot be enabled unfortunately..

Bummer.

Comment 37 Hans de Goede 2018-05-19 08:35:59 UTC
Ok, so I had some inspiration about this but this morning. I realized that besides the i2c bus the touchpad also needs an interrupt and that is connected to a GPIO, which means we may need a pinctrl and/or gpio driver for this.

Which led to me finding that CONFIG_GPIO_AMDPT and CONFIG_PINCTRL_AMD are not set in the Fedora test kernels, so here is a new test build with these enabled:
https://koji.fedoraproject.org/koji/taskinfo?taskID=27060582

As usual this will take a couple of hours to finish and testing instructions are here: https://fedorapeople.org/~jwrdegoede/kernel-test-instructions.txt

Regardless of if this fixes things, can you please run alsa-info.sh let it save to a file and attach the file here? I know this is not a sound related bug, but that is the easiest way to get a log of which ACPI devices are present on your systems and what their status is.

Also if things still do not work, please run "ls -l /sys/bus/i2c/devices" and copy and paste the output here.

Comment 38 Turgut Kalfaoglu 2018-05-19 15:36:13 UTC
Created attachment 1438927 [details]
output of alsa-info.sh

Comment 39 Turgut Kalfaoglu 2018-05-19 15:44:19 UTC
Many thanks.. I'm checking it now.. I haven't installed the latest kernel yet..

[root@tklaptop ~]# ls -l /sys/bus/i2c/devices
total 0
lrwxrwxrwx 1 root root 0 May 19  2018 i2c-0 -> ../../../devices/platform/AMDI0010:03/i2c-0
lrwxrwxrwx 1 root root 0 May 19 18:27 i2c-1 -> ../../../devices/pci0000:00/0000:00:03.1/0000:0c:00.0/i2c-1
lrwxrwxrwx 1 root root 0 May 19 18:27 i2c-10 -> ../../../devices/pci0000:00/0000:00:03.1/0000:0c:00.0/drm/card0/card0-DP-1/i2c-10
lrwxrwxrwx 1 root root 0 May 19 18:27 i2c-11 -> ../../../devices/pci0000:00/0000:00:03.1/0000:0c:00.0/drm/card0/card0-DP-2/i2c-11
lrwxrwxrwx 1 root root 0 May 19 18:27 i2c-12 -> ../../../devices/pci0000:00/0000:00:14.0/i2c-12
lrwxrwxrwx 1 root root 0 May 19 18:27 i2c-13 -> ../../../devices/pci0000:00/0000:00:14.0/i2c-13
lrwxrwxrwx 1 root root 0 May 19 18:27 i2c-14 -> ../../../devices/pci0000:00/0000:00:14.0/i2c-14
lrwxrwxrwx 1 root root 0 May 19 18:27 i2c-15 -> ../../../devices/pci0000:00/0000:00:14.0/i2c-15
lrwxrwxrwx 1 root root 0 May 19 18:27 i2c-2 -> ../../../devices/pci0000:00/0000:00:03.1/0000:0c:00.0/i2c-2
lrwxrwxrwx 1 root root 0 May 19 18:27 i2c-3 -> ../../../devices/pci0000:00/0000:00:03.1/0000:0c:00.0/i2c-3
lrwxrwxrwx 1 root root 0 May 19 18:27 i2c-4 -> ../../../devices/pci0000:00/0000:00:03.1/0000:0c:00.0/i2c-4
lrwxrwxrwx 1 root root 0 May 19 18:27 i2c-5 -> ../../../devices/pci0000:00/0000:00:03.1/0000:0c:00.0/i2c-5
lrwxrwxrwx 1 root root 0 May 19 18:27 i2c-6 -> ../../../devices/pci0000:00/0000:00:03.1/0000:0c:00.0/i2c-6
lrwxrwxrwx 1 root root 0 May 19 18:27 i2c-7 -> ../../../devices/pci0000:00/0000:00:03.1/0000:0c:00.0/i2c-7
lrwxrwxrwx 1 root root 0 May 19 18:27 i2c-8 -> ../../../devices/pci0000:00/0000:00:03.1/0000:0c:00.0/i2c-8
lrwxrwxrwx 1 root root 0 May 19 18:27 i2c-9 -> ../../../devices/pci0000:00/0000:00:03.1/0000:0c:00.0/drm/card0/card0-eDP-1/i2c-9
lrwxrwxrwx 1 root root 0 May 19  2018 i2c-ELAN1200:00 -> ../../../devices/platform/AMDI0010:03/i2c-0/i2c-ELAN1200:00
[root@tklaptop ~]#

Comment 40 Turgut Kalfaoglu 2018-05-19 16:07:15 UTC
THANK YOU!  
It is solved; and the touchpad now works :)

Do we need to do something special so that these changes are reflected to  the mainstream Fedora kernel?   I'd love these changes to stay if possible..

Comment 41 Turgut Kalfaoglu 2018-05-19 16:11:37 UTC
PS: Right click results in a left click; but that's probably a gnome issue..

Comment 42 Hans de Goede 2018-05-19 16:13:29 UTC
(In reply to Turgut Kalfaoglu from comment #40)
> THANK YOU!  
> It is solved; and the touchpad now works :)
> 
> Do we need to do something special so that these changes are reflected to 
> the mainstream Fedora kernel?   I'd love these changes to stay if possible..

I will push these changes to the Fedora kernel, to be picked up by the next official Fedora kernel build, so there is nothing which you need to do.

Comment 43 Hans de Goede 2018-05-19 16:15:26 UTC
(In reply to Turgut Kalfaoglu from comment #41)
> PS: Right click results in a left click; but that's probably a gnome issue..

Right, GNOME3 now defaults to clickfinger, which means that clicking anywhere with 2 fingers down results in a right click, to get a right click area at the bottom right of the touchpad as you probable expect, install gnome-tweaks, start it, go to "Keyboard & Mouse" and change the "Mouse Click Emulation" from "Fingers" to "Area".

Comment 44 Turgut Kalfaoglu 2018-05-19 18:05:31 UTC
Thank you very much for all your help!

Comment 45 Fedora Update System 2018-05-21 19:27:23 UTC
kernel-4.16.10-300.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-8ce90c8b24

Comment 46 Fedora Update System 2018-05-21 19:29:46 UTC
kernel-4.16.10-200.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2018-42898720ed

Comment 47 Fedora Update System 2018-05-22 15:03:22 UTC
kernel-4.16.10-200.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-42898720ed

Comment 48 Fedora Update System 2018-05-22 19:38:19 UTC
kernel-4.16.10-300.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-8ce90c8b24

Comment 49 Fedora Update System 2018-05-23 00:38:20 UTC
kernel-4.16.11-200.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2018-93c2e74446

Comment 50 Hans de Goede 2018-05-23 08:44:50 UTC
*** Bug 1573133 has been marked as a duplicate of this bug. ***

Comment 51 Fedora Update System 2018-05-23 15:41:32 UTC
kernel-4.16.10-300.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.

Comment 52 Fedora Update System 2018-05-23 16:20:47 UTC
kernel-4.16.11-200.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-93c2e74446

Comment 53 Fedora Update System 2018-05-25 15:45:08 UTC
kernel-4.16.11-200.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.

Comment 54 Hein-Pieter van Braam 2018-05-31 00:15:06 UTC
4.16.12-300.fc28.x86_64 fixed the issue for me. Awesome!

Comment 55 Turgut Kalfaoglu 2019-02-25 04:28:39 UTC
The issue is back.. GL702ZC touchpad no longer works..

$ uname -a
Linux ....  4.20.10-200.fc29.x86_64 #1 SMP Fri Feb 15 18:33:14 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

]$ more /etc/sysconfig/grub 
GRUB_CMDLINE_LINUX="pti=off"

Comment 56 Hans de Goede 2019-02-25 13:07:08 UTC
I just checked, the 2 kernel config options which were missing before are still present in the current Fedora kernels, so that is not the problem.