Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1397428

Summary: Redirection to file results in traceback
Product: [Fedora] Fedora Reporter: Jaroslav Škarvada <jskarvad>
Component: pythonAssignee: Charalampos Stratakis <cstratak>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 24CC: bkabrda, cstratak, dmalcolm, ivazqueznet, jonathansteffan, mhroncok, ncoghlan, pviktori, rkuska, tomspur, torsava
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-22 15:59:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jaroslav Škarvada 2016-11-22 13:55:54 UTC 
Description of problem:
The same code when redirected from STDOUT to file can result in traceback, which is strange.

Version-Release number of selected component (if applicable):
python-2.7.12-6.fc24.x86_64

How reproducible:
Always

Steps to Reproduce:
1.cat > test.py <<:EOF
#!/bin/python

url = 'https://github.com/EttusResearch/uhd/releases'
import requests
text = requests.get(url).text
print(text)
:EOF

2. python test.py
3. python test.py > log.txt

Actual results:
2. content
3. Traceback (most recent call last):
  File "./test0.py", line 10, in <module>
    print text
UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 1016: ordinal not in range(128)

Expected results:
2. content
3. content

Additional info:
Redirection to file shouldn't result in different behavior, nor in traceback.
Comment 1 Miro Hrončok 2016-11-22 15:55:10 UTC 
Smaller reproducer:

1.cat > test.py <<:EOF
print(u'\xb7')
:EOF

2. python test.py
3. python test.py | head

Actual results:
2. ·
3. Traceback (most recent call last):
  File "test.py", line 1, in <module>
    print(u'\xb7')
UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 0: ordinal not in range(128)

Expected results:
2. ·
3. ·

But I'm afraid this is a "feature" of Python 2.

See http://stackoverflow.com/questions/19696652/piping-output-causes-python-program-to-fail
Comment 2 Dave Malcolm 2016-11-22 15:59:29 UTC 
The issue is that in CPython 2 the encoding of stdout changes from utf-8 to ascii if stdout is not a tty.

I agree that it's counter-intuitive.

FWIW, I proposed changing this in Fedora 13:

  https://fedoraproject.org/wiki/Features/PythonEncodingUsesSystemLocale

but I was asked to not change this by the upstream Python development community, to ensure consistent behavior with every other CPython 2 implementation.

See that link for more information, and some workarounds.

Closing this out as WONTFIX.