Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1269145 - Syncing some repos results in UnicodeEncodeError: 'ascii' codec can't encode character
Summary: Syncing some repos results in UnicodeEncodeError: 'ascii' codec can't encode ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Repositories
Version: 6.1.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: Unspecified
Assignee: Katello Bug Bin
QA Contact: Kedar Bidarkar
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-10-06 12:58 UTC by Justin Sherrill
Modified: 2021-04-06 18:01 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-07-27 11:31:39 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Pulp Redmine 1264 0 High CLOSED - CURRENTRELEASE UnicodeEncodeError while synchronizing Fedora 21 and 22 updates Never
Pulp Redmine 1301 0 High CLOSED - NOTABUG Error producing repo metadata Never

Description Justin Sherrill 2015-10-06 12:58:38 UTC
Description of problem:

Cloned from https://pulp.plan.io/issues/1264

Syncing some repos (such as F21 and F22 updates repo) results in an error and traceback.


Version-Release number of selected component (if applicable):


How reproducible:
Always according to user

Steps to Reproduce:
1. create a repo pointed to http://dl.fedoraproject.org/pub/fedora/linux/updates/22/x86_64/
2. Sync the repo

Actual results:
Error with traceback below


Expected results:
Should sync successfully. 

Additional info:


Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 148, in run
        self.update_content(metadata_files)
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 324, in update_content
        rpms_to_download, drpms_to_download = self._decide_what_to_download(metadata_files)
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 342, in decide_what_to_download
        rpms_to_download, rpms_count, rpms_total_size = self._decide_rpms_to_download(metadata_files)
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 374, in _decide_rpms_to_download
        wanted = self._identify_wanted_versions(package_info_generator)
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 687, in _identify_wanted_versions
        for model in package_info_generator:
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/repomd/packages.py", line 63, in package_list_generator
        package_info = process_func(element)
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/repomd/primary.py", line 180, in process_package_element
        model.raw_xml = utils.element_to_raw_xml(package_element, [rpm_namespace], COMMON_SPEC_URL)
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/utils.py", line 55, in element_to_raw_xml
        tree.write(io)
File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 663, in write
        self._write(file, self._root, encoding, {})
File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 707, in _write
        self._write(file, n, encoding, namespaces)
File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 705, in _write
        file.write(_escape_cdata(node.text, encoding))
File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 807, in _escape_cdata
        return _encode_entity(text)
File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 792, in _encode_entity
        return _encode(pattern.sub(escape_entities, text), "ascii")
File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 751, in _encode 
        return s.encode(encoding)
UnicodeEncodeError: 'ascii' codec can't encode character u'\U000ff004' in position 145: ordinal not in range(128)

Comment 2 pulp-infra@redhat.com 2015-10-06 13:30:21 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 3 pulp-infra@redhat.com 2015-10-06 13:30:22 UTC
The Pulp upstream bug priority is at High. Updating the external tracker on this bug.

Comment 4 pulp-infra@redhat.com 2015-10-06 14:00:23 UTC
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 5 Michael Hrivnak 2015-10-09 14:29:40 UTC
We discovered an additional problem that is related, and I am adding the tracker for that new redmine issue.

Comment 6 pulp-infra@redhat.com 2015-10-09 15:00:16 UTC
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.

Comment 7 pulp-infra@redhat.com 2015-10-09 15:00:18 UTC
The Pulp upstream bug priority is at High. Updating the external tracker on this bug.

Comment 8 pulp-infra@redhat.com 2015-10-20 15:00:16 UTC
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.

Comment 9 pulp-infra@redhat.com 2015-10-22 14:30:22 UTC
The Pulp upstream bug status is at CLOSED - NOTABUG. Updating the external tracker on this bug.

Comment 10 pulp-infra@redhat.com 2015-10-22 15:00:33 UTC
The Pulp upstream bug status is at ON_QA. Updating the external tracker on this bug.

Comment 11 pulp-infra@redhat.com 2015-11-09 21:00:31 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 13 Bryan Kearney 2016-02-15 18:30:23 UTC
Pulp 2.7 appears to fix this. Moving this to POST.

Comment 15 Kedar Bidarkar 2016-04-13 11:59:01 UTC
Syncing of the fedora 22 url was successful without any error as mentioned in the bug description.

2016-04-13 07:04:58 [app] [I] Processing by Katello::Api::V2::RepositoriesController#sync_complete as */*
2016-04-13 07:04:58 [app] [I]   Parameters: {"call_report"=>"[FILTERED]", "event_type"=>"repo.sync.finish", "payload"=>{"importer_id"=>"yum_importer", "exception"=>nil, "repo_id"=>"Default_Organization-fedora_prd-fedora", "traceback"=>nil, "started"=>"2016-04-13T09:45:25Z", "_ns"=>"repo_sync_results", "completed"=>"2016-04-13T11:04:58Z", "importer_type_id"=>"yum_importer", "error_message"=>nil, "summary"=>{"content"=>{"state"=>"FINISHED"}, "comps"=>{"state"=>"FINISHED"}, "purge_duplicates"=>{"state"=>"FINISHED"}, "distribution"=>{"state"=>"FINISHED"}, "errata"=>{"state"=>"FINISHED"}, "metadata"=>{"state"=>"FINISHED"}}, "added_count"=>40075, "result"=>"success", "updated_count"=>0, "details"=>{"content"=>{"size_total"=>31885621008, "items_left"=>0, "items_total"=>36325, "state"=>"FINISHED", "size_left"=>0, "details"=>{"rpm_total"=>18919, "rpm_done"=>18919, "drpm_total"=>17406, "drpm_done"=>17406}, "error_details"=>nil}, "comps"=>{"state"=>"FINISHED"}, "purge_duplicates"=>{"state"=>"FINISHED"}, "distribution"=>{"items_total"=>0, "state"=>"FINISHED", "error_details"=>nil, "items_left"=>0}, "errata"=>{"state"=>"FINISHED"}, "metadata"=>{"state"=>"FINISHED"}}, "id"=>"570e27da952da6d3130ec07b", "removed_count"=>0}, "token"=>"ZhGSbRpxPy7Q6QvHbmbFY6iBysgySKGm", "api_version"=>"v2", "repository"=>{}}
2016-04-13 07:04:59 [app] [I] Sync_complete called for fedora, running after_sync.
2016-04-13 07:04:59 [app] [I] Completed 200 OK in 97ms (Views: 1.0ms | ActiveRecord: 4.5ms)
2016-04-13 07:04:59 [app] [I] Started GET "/katello/sync_management/sync_status?repoids%5B%5D=36&_=1460540702900" for 10.10.50.140 at 2016-04-13 07:04:59 -0400
2016-04-13 07:04:59 [app] [I] Processing by Katello::SyncManagementController#sync_status as JSON
2016-04-13 07:04:59 [app] [I]   Parameters: {"repoids"=>["36"], "_"=>"1460540702900"}
2016-04-13 07:04:59 [app] [I] Completed 200 OK in 148ms (Views: 0.7ms | ActiveRecord: 14.8ms)


VERIFIED With sat62-snap7.1

Comment 16 Bryan Kearney 2016-07-27 11:31:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501


Note You need to log in before you can comment on or make changes to this bug.