Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1358845 - nghttpx crashes on armv7hl while running HTTP/2 curl tests
Summary: nghttpx crashes on armv7hl while running HTTP/2 curl tests
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: nghttp2
Version: 25
Hardware: armv7hl
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Kamil Dudka
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 1360319
Blocks: ARMTracker
TreeView+ depends on / blocked
 
Reported: 2016-07-21 15:19 UTC by Kamil Dudka
Modified: 2016-07-29 13:56 UTC (History)
2 users (show)

Fixed In Version: nghttp2-1.13.0-2.fc25
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-07-29 13:56:05 UTC
Type: Bug
Embargoed:

Attachments (Terms of Use)
[PATCH] nghttpx: avoid using std::function to fix crash on armv7hl (3.53 KB, patch)
2016-07-26 11:11 UTC, Kamil Dudka 		no flags Details | Diff
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1358749 0 unspecified CLOSED curl-7.50.0 is available 2022-05-16 11:32:56 UTC

Internal Links: 1358749

Description Kamil Dudka 2016-07-21 15:19:20 UTC 
Version-Release number of selected component (if applicable):
nghttp2-1.13.0-1.fc25.armv7hl


Actual results:
[INFO] shrpx_config.cc:2881 Resolving backend address
[INFO] shrpx_config.cc:2891 Host-path pattern: group 0: '/'
[INFO] shrpx_config.cc:2894 group 0 -> 127.0.0.1:8990, proto=http/1.1
[INFO] shrpx_config.cc:2910 Catch-all pattern is group 0
[INFO] shrpx_config.cc:3002 Address resolution for 127.0.0.1 succeeded: 127.0.0.1
[INFO] shrpx_config.cc:2958 Resolved backend address: 127.0.0.1:8990 -> 127.0.0.1:8990
[NOTICE] Listening on 0.0.0.0:9015
[NOTICE] Listening on [::]:9015
[NOTICE] Worker process [29578] spawned
[INFO] shrpx_worker.cc:265 number of http/1.1 backend: 1, number of h2 backend: 0
[INFO] shrpx_worker_process.cc:528 Entering event loop
[INFO] shrpx_connection_handler.cc:365 [LISTEN:0xbefab428] Accepted connection. fd=11
[INFO] shrpx_client_handler.cc:70 [CLIENT_HANDLER:0x80e91750] Time out
[INFO] shrpx_client_handler.cc:449 [CLIENT_HANDLER:0x80e91750] Deleting
[INFO] shrpx_client_handler.cc:472 [CLIENT_HANDLER:0x80e91750] Deleted
[INFO] shrpx_connection_handler.cc:365 [LISTEN:0xbefab428] Accepted connection. fd=11
[NOTICE] Worker process: [29578] exited abnormally with status 8b; exit status 0; signal Segmentation fault(11)
[NOTICE] Shutdown momentarily


Additional info:
http://koji.fedoraproject.org/koji/taskinfo?taskID=14968582
http://koji.fedoraproject.org/koji/taskinfo?taskID=14968582
Comment 1 Kamil Dudka 2016-07-25 15:30:30 UTC 
Reproduced on an f24 box:

# rpm -qa libev\* \*nghttp\* | sort -V
libev-4.20-2.fc24.armv7hl
libev-debuginfo-4.20-2.fc24.armv7hl
libev-devel-4.20-2.fc24.armv7hl
libnghttp2-1.13.0-1.fc24.armv7hl
libnghttp2-devel-1.13.0-1.fc24.armv7hl
nghttp2-1.13.0-1.fc24.armv7hl
nghttp2-debuginfo-1.13.0-1.fc24.armv7hl

(gdb) bt
#0  0x7f62ebc4 in std::__invoke_impl<int, int (shrpx::ClientHandler::* const&)(), shrpx::ClientHandler&> (__t=..., __f=<optimized out>) at /usr/include/c++/6.1.1/functional:226
#1  std::__invoke<int (shrpx::ClientHandler::* const&)(), shrpx::ClientHandler&> (__fn=<optimized out>) at /usr/include/c++/6.1.1/functional:260
#2  std::_Mem_fn_base<int (shrpx::ClientHandler::*)(), true>::operator()<shrpx::ClientHandler&> (this=<optimized out>) at /usr/include/c++/6.1.1/functional:613
#3  std::_Function_handler<int (shrpx::ClientHandler&), int (shrpx::ClientHandler::*)()>::_M_invoke(std::_Any_data const&, shrpx::ClientHandler&) (__functor=..., __args#0=...) at /usr/include/c++/6.1.1/functional:1788
#4  0x7f6295b0 in std::function<int (shrpx::ClientHandler&)>::operator()(shrpx::ClientHandler&) const (__args#0=..., this=0x7fadce8c) at /usr/include/c++/6.1.1/functional:2136
#5  shrpx::ClientHandler::do_read (this=0x7fadcc60) at shrpx_client_handler.cc:615
#6  shrpx::(anonymous namespace)::readcb (loop=<optimized out>, w=<optimized out>, revents=<optimized out>) at shrpx_client_handler.cc:94
#7  0xb6e9893c in ev_invoke_pending (loop=0xb6eb0358 <default_loop_struct>) at ev.c:3155
#8  0xb6e9bc18 in ev_run (loop=0xb6eb0358 <default_loop_struct>, loop@entry=0x0, flags=-1226195944, flags@entry=0) at ev.c:3555
#9  0x7f618f30 in shrpx::worker_process_event_loop (wpconf=0x0) at shrpx_worker_process.cc:537
#10 0x7f5bf524 in shrpx::(anonymous namespace)::fork_worker_process (ssv=0xbeaf1c8c) at shrpx.cc:884
#11 shrpx::(anonymous namespace)::event_loop () at shrpx.cc:966
#12 0x7f5b9678 in shrpx::main (argc=6, argv=0xbeaf45b4) at shrpx.cc:3113
#13 0x7f5c1b58 in std::function<int (int, char**)>::operator()(int, char**) const (__args#1=<optimized out>, __args#0=<optimized out>, this=<optimized out>) at /usr/include/c++/6.1.1/functional:2136
#14 nghttp2::run_app(std::function<int (int, char**)>, int, char**) (app=..., argc=<optimized out>, argv=<optimized out>) at template.h:542
#15 0x7f5b7c90 in main (argc=<optimized out>, argv=<optimized out>) at shrpx.cc:3126

If I rebuild/reinstall libev from SRPM locally (the same NVR), nghttpx stops crashing but HTTP/2 curl's tests fail anyway after a while of hanging.
Comment 2 Kamil Dudka 2016-07-25 15:44:04 UTC 
> If I rebuild/reinstall libev from SRPM locally (the same NVR), nghttpx stops
> crashing but HTTP/2 curl's tests fail anyway after a while of hanging.

Actually, only the first test hangs whereas the subsequent tests still crash the nghttpx worker.  It is reproducible also with libev-4.22-1.fc25 rebuilt for f24.

Here is another backtrace caught with the updated libev:

(gdb) bt
#0  0x00000000 in ?? ()
#1  std::function<int (shrpx::ClientHandler&)>::operator()(shrpx::ClientHandler&) const (__args#0=..., this=0x81188e8c) at /usr/include/c++/6.1.1/functional:2136
#2  shrpx::ClientHandler::do_read (this=0x81188c60) at shrpx_client_handler.cc:615
#3  shrpx::(anonymous namespace)::readcb (loop=<optimized out>, w=<optimized out>, revents=<optimized out>) at shrpx_client_handler.cc:94
#4  ev_invoke_pending (loop=0xb6f51358 <default_loop_struct>) at ev.c:3288
#5  ev_run (loop=0xb6f51358 <default_loop_struct>, loop@entry=0x0, flags=-1225536504, flags@entry=0) at ev.c:3688
#6  shrpx::worker_process_event_loop (wpconf=0x0) at shrpx_worker_process.cc:537
#7  shrpx::(anonymous namespace)::fork_worker_process (ssv=0xbeb99c8c) at shrpx.cc:884
#8  shrpx::(anonymous namespace)::event_loop () at shrpx.cc:966
#9  shrpx::main (argc=6, argv=0xbeb9c5b4) at shrpx.cc:3113
#10 std::function<int (int, char**)>::operator()(int, char**) const (__args#1=<optimized out>, __args#0=<optimized out>, this=<optimized out>) at /usr/include/c++/6.1.1/functional:2136
#11 nghttp2::run_app(std::function<int (int, char**)>, int, char**) (app=..., argc=<optimized out>, argv=<optimized out>) at template.h:542
#12 main (argc=<optimized out>, argv=<optimized out>) at shrpx.cc:3126
Comment 3 Kamil Dudka 2016-07-25 17:38:25 UTC 
Rebuilding nghttp2 with -O1 in CXXFLAGS prevents nghttpx from crashing.
Comment 4 Kamil Dudka 2016-07-25 22:28:32 UTC 
-- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -156,8 +156,12 @@ endif # HAVE_MRUBY
 noinst_LIBRARIES = libnghttpx.a
 libnghttpx_a_SOURCES = ${NGHTTPX_SRCS}
 libnghttpx_a_CPPFLAGS = ${AM_CPPFLAGS}

+# needed for shrpx_client_handler.cc and shrpx_http_downstream_connection.cc
+# on armv7hl -- see https://bugzilla.redhat.com/1358845 for details
+libnghttpx_a_CXXFLAGS = -fno-strict-aliasing
+
 nghttpx_SOURCES = shrpx.cc shrpx.h
 nghttpx_CPPFLAGS = ${libnghttpx_a_CPPFLAGS}
 nghttpx_LDADD = libnghttpx.a ${LDADD}
Comment 5 Jan Kurik 2016-07-26 04:30:14 UTC 
This bug appears to have been reported against 'rawhide' during the Fedora 25 development cycle.
Changing version to '25'.
Comment 6 Kamil Dudka 2016-07-26 11:11:01 UTC 
Created attachment 1184185 [details]
[PATCH] nghttpx: avoid using std::function to fix crash on armv7hl

There seems to be some problem with nghttp's use of std::function.  The attached patch makes the crashes go away.  I am not familiar with std::function myself, so it is not clear to me whether it is a bug of nghttp or a gcc bug.
Comment 7 Kamil Dudka 2016-07-26 12:30:14 UTC 
Reported as gcc bug #1360319 with minimal example: attachment #1184246 [details]
Comment 8 Kamil Dudka 2016-07-26 15:07:48 UTC 
Reported to nghttp2 upstream: https://github.com/nghttp2/nghttp2/issues/635
Comment 9 Kamil Dudka 2016-07-29 13:56:05 UTC 
curl now builds on armv7hl without any workaround in place:

http://koji.fedoraproject.org/koji/buildinfo?buildID=785581
Note You need to log in before you can comment on or make changes to this bug.