Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1281714

Summary: glibc: reused_arena can pick an arena on the free list, leading to an assertion failure
Product: [Fedora] Fedora Reporter: Vít Ondruch <vondruch>
Component: glibcAssignee: Florian Weimer <fweimer>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: arjun.is, codonell, fweimer, jakub, law, mfabian, pfrankli, siddhesh
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glibc-2.22.90-25.fc24 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-12-16 13:01:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
ABRT's core dump none

Description Vít Ondruch 2015-11-13 09:11:39 UTC
Description of problem:
Ruby test suite randomly core dumps since glibc-2.22.90-16.fc24 with message:

test_thread.rb           .............................................E....

                         
#1006 test_thread.rb:390:in `<top (required)>': core dumped  
FAIL 1/1010 tests failed
uncommon.mk:549: recipe for target 'yes-btest-ruby' failed
make: *** [yes-btest-ruby] Error 1


This is the source code of the offending test:

https://github.com/ruby/ruby/blob/v2_2_3/bootstraptest/test_thread.rb

You can observe the issues in Koschei [1].


Version-Release number of selected component (if applicable):
$ rpm -q glibc
glibc-2.22.90-16.fc24.x86_64
$ rpm -q ruby
ruby-2.2.3-44.fc24.x86_64


How reproducible:
Randomly


Steps to Reproduce:
1. make btest TESTOPTS="--sets=thread"
2.
3.

Actual results:


Expected results:


Additional info:


[1] https://apps.fedoraproject.org/koschei/package/ruby

Comment 1 Vít Ondruch 2015-11-13 09:12:40 UTC
The precise link to source code should be:

https://github.com/ruby/ruby/blob/v2_2_3/bootstraptest/test_thread.rb#L390

Comment 2 Florian Weimer 2015-11-13 09:22:34 UTC
(In reply to Vít Ondruch from comment #1)
> The precise link to source code should be:
> 
> https://github.com/ruby/ruby/blob/v2_2_3/bootstraptest/test_thread.rb#L390

Thanks.  Would you please provide instructions how to run this test in isolation?

Comment 3 Vít Ondruch 2015-11-13 09:58:57 UTC
(In reply to Florian Weimer from comment #2)
> Thanks.  Would you please provide instructions how to run this test in
> isolation?



> Steps to Reproduce:
> 1. make btest TESTOPTS="--sets=thread"


This is as isolated as it is possible. If you really want to run just the tiniest part of the code just with the specific part of the code, I am afraid you have to modify the file.


And just one additional remark, I have not been successful in reproducing this issue locally so far (I am running the test locally 500 times right now). Not sure what might be the difference in Koji environment, since there is apparently higher chance to hit this issue.

Comment 4 Vít Ondruch 2015-11-13 13:22:17 UTC
Created attachment 1093634 [details]
ABRT's core dump

Ok, in less then 1000 iteration, I was able to reproduce the issue locally. Please see the attached core dump caught by ABRT.

Comment 5 Florian Weimer 2015-11-13 16:30:24 UTC
This assert fires:

821		  /* Arenas on the free list are not attached to any thread.  */
822		  assert (result->attached_threads == 0);

I don't know yet why this supposed invariant is broken.

Comment 6 Florian Weimer 2015-11-13 20:37:29 UTC
A possible explanation: reused_arena picks an arena on the free list which was added to it after the initial check in arena_get2.  There is no locking which prevents that, and arenas on the free list are not removed from the circular arena list around main_arena.

This is definitely a bug.  I don't know if it is this bug, though.

Comment 7 Florian Weimer 2015-11-16 09:16:06 UTC
I have a very reliable test case, just needs some polishing.

Comment 8 Vít Ondruch 2015-12-08 09:16:51 UTC
This is still unresolved as of glibc-2.22.90-24.fc24. This is recent build failure in Koschei:

https://apps.fedoraproject.org/koschei/build/1153943

Comment 9 Florian Weimer 2015-12-08 09:31:57 UTC
(In reply to Vít Ondruch from comment #8)
> This is still unresolved as of glibc-2.22.90-24.fc24. This is recent build
> failure in Koschei:
> 
> https://apps.fedoraproject.org/koschei/build/1153943

Yes, this is expected.  The upstream patch still hasn't been reviewed.  I expect some movement there this week.  If not, I will back out the patch in rawhide which causes this regression.

Comment 10 Vít Ondruch 2015-12-08 09:46:26 UTC
thx for the update.

Comment 11 Florian Weimer 2015-12-16 13:01:27 UTC
Finally fixed in rawhide (release branches were not affected).  Sorry for the delay.