Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1270480 - *fatal* sort broken: yet again!
Summary: *fatal* sort broken: yet again!
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: coreutils
Version: 22
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Ondrej Vasik
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-10-10 09:22 UTC by Ali Akcaagac
Modified: 2015-10-14 15:09 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-10-14 13:22:10 UTC
Type: Bug
Embargoed:

Attachments (Terms of Use)
old.txt (9.66 KB, text/plain)
2015-10-10 09:22 UTC, Ali Akcaagac 		no flags Details
new.txt (9.66 KB, text/plain)
2015-10-10 09:23 UTC, Ali Akcaagac 		no flags Details
correct old.txt (9.66 KB, text/plain)
2015-10-10 09:48 UTC, Ali Akcaagac 		no flags Details
correct new.txt (9.66 KB, text/plain)
2015-10-10 09:49 UTC, Ali Akcaagac 		no flags Details
Show Obsolete (2) View All

Description Ali Akcaagac 2015-10-10 09:22:39 UTC 
Created attachment 1081562 [details]
old.txt

It came to my attention, that coreutils 8.23-11 has a broken sort utility while coreutirls 8.23-9 has a working sort utility.

This is inacceptable. Please test your patches before applying them. Sorry for sounding a bit harsh here but *sort* is a core element and tons of scripts use sort to sort lists or pipe sort output that get's processed later on. The current sort totally misbehaves, gives wrong output and if you use the outout of the new sort to diff e.g. lists that has been done with old sort, then the result is - to say it in good words - fatal!

Please find 2 attachments:

old.txt -> coreutils 8.23-9
new.txt -> coreutils 8.23-11

diff -ruN old.txt new.txt > diff.txt

... or simply load in an editor and you will see.

You can also find in "new.txt" that

a
b
c
d
e
                      : e
f
g
h

...

... but the long spaced line with " : e" is not the correct sorting after e because it has a bunch of spaces and a colon infront of it.
Comment 1 Ali Akcaagac 2015-10-10 09:23:31 UTC 
Created attachment 1081563 [details]
new.txt

Commands issued:

old coreutirls
sudo dnf search ios | LC_ALL=C sort > old.txt

new coreutirls
sudo dnf search ios | LC_ALL=C sort > new.txt
Comment 2 Ali Akcaagac 2015-10-10 09:48:53 UTC 
Created attachment 1081580 [details]
correct old.txt
Comment 3 Ali Akcaagac 2015-10-10 09:49:24 UTC 
Created attachment 1081581 [details]
correct new.txt
Comment 4 Pádraig Brady 2015-10-10 10:52:31 UTC 
I can't see what is going on here sorry.

From the initial description you want the lines with initial spaces sorted first, which is what you should get with LC_ALL=C.
However in old.txt the lines with initial spaces do not sort first.
I thought old.txt is what you expected, so I'm confused.

Anyway the difference between the two sorting orders are due to
different LC_ALL settings for each sort run. Can you double check
that the env vars are set appropriately for each run.

Also I downloaded both 8.23-9.fc22 and 8.23-11.fc22
and didn't notice any difference between them for the attached inputs.
Comment 5 Pádraig Brady 2015-10-10 11:18:38 UTC 
For completeness, note the setlocale() calls in sort(1) are not checked,
so if one system had de_DE installed while another didn't and fell
back to the "good" C locale, it might explain things?
Comment 6 Kamil Dudka 2015-10-12 09:55:28 UTC 
Please re-test.
Comment 7 Ali Akcaagac 2015-10-14 13:22:10 UTC 
Yes you are right! I have to apologize here for not looking closer to the real problem. The problem was not coreutils or wrong locales being used. It was because a previous installation of glibc (GNU C Library) that has shown up a few days before coreutils update caused a segmentation fault during update. I figured this out once I repeated and reviewed the steps for the past days of dnf updates. The locale file stored in /usr/lib/locale/locale-archive (not the template) was null bytes in size. I had to re-create that one. Now the ouput is as expected again - corresponding my language used. Therefore closing as *NOTABUG*
Comment 8 Pádraig Brady 2015-10-14 15:09:00 UTC 
Note the setlocale() calls in sort(1) don't warn upon failure, silently falling back to the "C" locale. This is due to too many warnings in various cases of incorrectly setup locales. Now it could be argued that locales are central to sort's operation, and so it shold at least warn. As an interim measure at least, we've added a warning upon setlocale() failure when --debug is specified, in the not yet released version of coreutils.
Note You need to log in before you can comment on or make changes to this bug.