Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1270480

Summary: *fatal* sort broken: yet again!
Product: [Fedora] Fedora Reporter: Ali Akcaagac <aliakc>
Component: coreutilsAssignee: Ondrej Vasik <ovasik>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 22CC: admiller, aliakc, kdudka, kzak, ooprala, ovasik, pbrady, p, twaugh
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-10-14 13:22:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
old.txt
none
new.txt
none
correct old.txt
none
correct new.txt none

Description Ali Akcaagac 2015-10-10 09:22:39 UTC
Created attachment 1081562 [details]
old.txt

It came to my attention, that coreutils 8.23-11 has a broken sort utility while coreutirls 8.23-9 has a working sort utility.

This is inacceptable. Please test your patches before applying them. Sorry for sounding a bit harsh here but *sort* is a core element and tons of scripts use sort to sort lists or pipe sort output that get's processed later on. The current sort totally misbehaves, gives wrong output and if you use the outout of the new sort to diff e.g. lists that has been done with old sort, then the result is - to say it in good words - fatal!

Please find 2 attachments:

old.txt -> coreutils 8.23-9
new.txt -> coreutils 8.23-11

diff -ruN old.txt new.txt > diff.txt

... or simply load in an editor and you will see.

You can also find in "new.txt" that

a
b
c
d
e
                      : e
f
g
h

...

... but the long spaced line with " : e" is not the correct sorting after e because it has a bunch of spaces and a colon infront of it.

Comment 1 Ali Akcaagac 2015-10-10 09:23:31 UTC
Created attachment 1081563 [details]
new.txt

Commands issued:

old coreutirls
sudo dnf search ios | LC_ALL=C sort > old.txt

new coreutirls
sudo dnf search ios | LC_ALL=C sort > new.txt

Comment 2 Ali Akcaagac 2015-10-10 09:48:53 UTC
Created attachment 1081580 [details]
correct old.txt

Comment 3 Ali Akcaagac 2015-10-10 09:49:24 UTC
Created attachment 1081581 [details]
correct new.txt

Comment 4 Pádraig Brady 2015-10-10 10:52:31 UTC
I can't see what is going on here sorry.

From the initial description you want the lines with initial spaces sorted first, which is what you should get with LC_ALL=C.
However in old.txt the lines with initial spaces do not sort first.
I thought old.txt is what you expected, so I'm confused.

Anyway the difference between the two sorting orders are due to
different LC_ALL settings for each sort run. Can you double check
that the env vars are set appropriately for each run.

Also I downloaded both 8.23-9.fc22 and 8.23-11.fc22
and didn't notice any difference between them for the attached inputs.

Comment 5 Pádraig Brady 2015-10-10 11:18:38 UTC
For completeness, note the setlocale() calls in sort(1) are not checked,
so if one system had de_DE installed while another didn't and fell
back to the "good" C locale, it might explain things?

Comment 6 Kamil Dudka 2015-10-12 09:55:28 UTC
Please re-test.

Comment 7 Ali Akcaagac 2015-10-14 13:22:10 UTC
Yes you are right! I have to apologize here for not looking closer to the real problem. The problem was not coreutils or wrong locales being used. It was because a previous installation of glibc (GNU C Library) that has shown up a few days before coreutils update caused a segmentation fault during update. I figured this out once I repeated and reviewed the steps for the past days of dnf updates. The locale file stored in /usr/lib/locale/locale-archive (not the template) was null bytes in size. I had to re-create that one. Now the ouput is as expected again - corresponding my language used. Therefore closing as *NOTABUG*

Comment 8 Pádraig Brady 2015-10-14 15:09:00 UTC
Note the setlocale() calls in sort(1) don't warn upon failure, silently falling back to the "C" locale. This is due to too many warnings in various cases of incorrectly setup locales. Now it could be argued that locales are central to sort's operation, and so it shold at least warn. As an interim measure at least, we've added a warning upon setlocale() failure when --debug is specified, in the not yet released version of coreutils.