Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1605672 - python-fedimg: FTBFS in Fedora rawhide
Summary: python-fedimg: FTBFS in Fedora rawhide
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: python-fedimg
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Fedora Infrastructure SIG
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: F29FTBFS
TreeView+ depends on / blocked
 
Reported: 2018-07-20 16:30 UTC by Mohan Boddu
Modified: 2018-07-25 16:22 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-07-25 16:22:12 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
build.log (deleted)
2018-07-20 16:30 UTC, Mohan Boddu
no flags Details
root.log (deleted)
2018-07-20 16:30 UTC, Mohan Boddu
no flags Details
state.log (deleted)
2018-07-20 16:30 UTC, Mohan Boddu
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github fedora-infra fedimg issues 149 0 None None None Never

Description Mohan Boddu 2018-07-20 16:30:03 UTC
python-fedimg failed to build from source in Fedora rawhide

https://koji.fedoraproject.org/koji/taskinfo?taskID=28221692


For details on the mass rebuild see:

https://fedoraproject.org/wiki/Fedora_29_Mass_Rebuild
Please fix python-fedimg at your earliest convenience and set the bug's status to
ASSIGNED when you start fixing it. If the bug remains in NEW state for 8 weeks,
python-fedimg will be orphaned. Before branching of Fedora 30,
python-fedimg will be retired, if it still fails to build.

For more details on the FTBFS policy, please visit:
https://fedoraproject.org/wiki/Fails_to_build_from_source

Comment 1 Mohan Boddu 2018-07-20 16:30:15 UTC
Created attachment 1466176 [details]
build.log

Comment 2 Mohan Boddu 2018-07-20 16:30:20 UTC
Created attachment 1466177 [details]
root.log

file root.log too big, will only attach last 32768 bytes

Comment 3 Mohan Boddu 2018-07-20 16:30:23 UTC
Created attachment 1466178 [details]
state.log

Comment 4 Kevin Fenzi 2018-07-20 21:55:35 UTC
I did some fixups here, but tests still fail.

See: 

https://koji.fedoraproject.org/koji/taskinfo?taskID=28478934

Which has: 

diff --git a/python-fedimg.spec b/python-fedimg.spec
index 55149d9..37028e7 100644
--- a/python-fedimg.spec
+++ b/python-fedimg.spec
@@ -2,7 +2,7 @@

 Name:               python-fedimg
 Version:            1.3.0
-Release:            2%{?dist}
+Release:            3%{?dist}
 Summary:            Automatically upload Fedora Cloud images to cloud providers                                      

 Group:              Development/Libraries
@@ -16,6 +16,9 @@ BuildRequires:      python2-devel
 BuildRequires:      python2-setuptools
 BuildRequires:      python2-nose
 BuildRequires:      python2-mock
+BuildRequires:      python2-vcrpy
+BuildRequires:      python2-service-identity
+BuildRequires:      python2-qpid

 BuildRequires:      fedmsg
 BuildRequires:      python2-libcloud
@@ -48,7 +51,7 @@ Requires:           python2-toml
 rm -rf %{modname}.egg-info

 %build
-%{__python} setup.py build
+%{__python2} setup.py build

 %install

@@ -57,10 +60,10 @@ rm -rf %{modname}.egg-info
 %{__cp} -p fedimg-conf.toml.example %{buildroot}%{_sysconfdir}/%{modname}/fedimg-conf.toml                           
 %{__cp} -p fedmsg.d/fedimg.py %{buildroot}%{_sysconfdir}/fedmsg.d/.                                                  

-%{__python} setup.py install -O1 --skip-build --root=%{buildroot}                                                    
+%{__python2} setup.py install -O1 --skip-build --root=%{buildroot}                                                   

 %check
-%{__python} setup.py test
+%{__python2} setup.py test

 %files -n python2-fedimg
 %doc docs/ README.md LICENSE
@@ -71,6 +74,9 @@ rm -rf %{modname}.egg-info
 %{python_sitelib}/%{modname}-%{version}*

 %changelog
+* Fri Jul 20 2018 Kevin Fenzi <kevin> - 1.3.0-3
+- Fix FTBFS bug #1605672
+
 * Sat Jul 14 2018 Fedora Release Engineering <releng> - 1.3.0-2                                    
 - Rebuilt for https://fedoraproject.org/wiki/Fedora_29_Mass_Rebuild                                                  

Seems like it's looking for the toml file in an installed directory, but the tests are being run from the rpm and would be using the stuff under the buildroot?

+ /usr/bin/python2 setup.py test
running test
running egg_info
writing requirements to fedimg.egg-info/requires.txt
writing fedimg.egg-info/PKG-INFO
writing top-level names to fedimg.egg-info/top_level.txt
writing dependency_links to fedimg.egg-info/dependency_links.txt
writing entry points to fedimg.egg-info/entry_points.txt
reading manifest file 'fedimg.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'CHANGELOG.md'
writing manifest file 'fedimg.egg-info/SOURCES.txt'
running build_ext
Failure: IOError ([Errno 2] No such file or directory: '/etc/fedimg/fedimg-conf.toml') ... ERROR
Failure: IOError ([Errno 2] No such file or directory: '/etc/fedimg/fedimg-conf.toml') ... ERROR
test_external_run_command (test_utils.TestFedimgUtils) ... ok
test_get_file_arch_aarch64 (test_utils.TestFedimgUtils) ... ok
test_get_file_arch_x86_64 (test_utils.TestFedimgUtils) ... ok
test_get_file_name_image (test_utils.TestFedimgUtils) ... ok
test_get_image_name_from_ami_name (test_utils.TestFedimgUtils) ... ok
test_get_image_name_from_ami_name_for_fedmsg (test_utils.TestFedimgUtils) ... ok
test_get_image_name_from_image (test_utils.TestFedimgUtils) ... ok
test_get_image_name_from_image_without_vol_type (test_utils.TestFedimgUtils) ... ok
test_get_item_from_regex (test_utils.TestFedimgUtils) ... ok
test_get_rawxz_urls (test_utils.TestFedimgUtils) ... ok
test_get_rawxz_urls_empty (test_utils.TestFedimgUtils) ... ok
test_get_source_from_image (test_utils.TestFedimgUtils) ... FAIL
test_get_source_from_image_retcode_1 (test_utils.TestFedimgUtils) ... ok
test_get_value_from_dict (test_utils.TestFedimgUtils) ... ok
test_get_virt_type_from_image_hvm (test_utils.TestFedimgUtils) ... ok
test_get_virt_types_hvm_pv (test_utils.TestFedimgUtils) ... ok
test_get_virt_types_only_hvm (test_utils.TestFedimgUtils) ... ok
test_get_volume_type_from_image (test_utils.TestFedimgUtils) ... ok
test_region_to_driver (test_utils.TestFedimgUtils) ... ok
test_ssh_connection_works_false (test_utils.TestFedimgUtils) ... ok
test_ssh_connection_works_true (test_utils.TestFedimgUtils) ... ok
======================================================================
ERROR: Failure: IOError ([Errno 2] No such file or directory: '/etc/fedimg/fedimg-conf.toml')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/builddir/build/BUILD/fedimg-1.3.0/tests/test_consumers.py", line 29, in <module>
    import fedimg.consumers
  File "/builddir/build/BUILD/fedimg-1.3.0/fedimg/consumers.py", line 36, in <module>
    import fedimg.uploader
  File "/builddir/build/BUILD/fedimg-1.3.0/fedimg/uploader.py", line 29, in <module>
    from fedimg.config import ACTIVE_SERVICES
  File "/builddir/build/BUILD/fedimg-1.3.0/fedimg/config.py", line 25, in <module>
    with open("/etc/fedimg/fedimg-conf.toml") as conffile:
IOError: [Errno 2] No such file or directory: '/etc/fedimg/fedimg-conf.toml'
======================================================================
ERROR: Failure: IOError ([Errno 2] No such file or directory: '/etc/fedimg/fedimg-conf.toml')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/builddir/build/BUILD/fedimg-1.3.0/tests/test_uploader.py", line 26, in <module>
    import fedimg.uploader
  File "/builddir/build/BUILD/fedimg-1.3.0/fedimg/uploader.py", line 29, in <module>
    from fedimg.config import ACTIVE_SERVICES
  File "/builddir/build/BUILD/fedimg-1.3.0/fedimg/config.py", line 25, in <module>
    with open("/etc/fedimg/fedimg-conf.toml") as conffile:
IOError: [Errno 2] No such file or directory: '/etc/fedimg/fedimg-conf.toml'
======================================================================
FAIL: test_get_source_from_image (test_utils.TestFedimgUtils)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "/builddir/build/BUILD/fedimg-1.3.0/tests/test_utils.py", line 240, in test_get_source_from_image
    mock.ANY
  File "/usr/lib/python2.7/site-packages/mock/mock.py", line 947, in assert_called_once_with
    raise AssertionError(msg)
AssertionError: Expected 'external_run_command' to be called once. Called 2 times.
-------------------- >> begin captured logging << --------------------
fedimg.utils: INFO: [PREP] Preparing temporary directory for download: '/tmp/tmpMiEEQC'
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 23 tests in 0.422s
FAILED (failures=1, errors=2)
Test failed: <unittest.runner.TextTestResult run=23 errors=2 failures=1>
error: Test failed: <unittest.runner.TextTestResult run=23 errors=2 failures=1>

Comment 5 Zbigniew Jędrzejewski-Szmek 2018-07-21 21:51:47 UTC
The tests fail because they hardcode reading /etc/fedimg/fedimg-conf.toml, which means that not only the package BuildRequires itself, but the installed version must have a matching config.

Comment 6 Sayan Chowdhury 2018-07-24 21:48:06 UTC
I did some changes which should fix this. 

The diff includes three patches:
- Fix the setup version
- Fix for the configuration path. If the configuration is not found it will look for it in the project root directory
- Fix the tests, update the broken vcr cassettes.

* Here is the diff of the spec file 

diff --git a/python-fedimg.spec b/python-fedimg.spec
index 55149d9..55e82f9 100644
--- a/python-fedimg.spec
+++ b/python-fedimg.spec
@@ -1,27 +1,37 @@
 %global modname fedimg
 
 Name:               python-fedimg
-Version:            1.3.0
-Release:            2%{?dist}
+Version:            2.2.0
+Release:            1%{?dist}
 Summary:            Automatically upload Fedora Cloud images to cloud providers
 
 Group:              Development/Libraries
 License:            AGPLv3+
 URL:                http://pypi.python.org/pypi/fedimg
 Source0:            http://pypi.python.org/packages/source/f/%{modname}/%{modname}-%{version}.tar.gz
+Patch0001:          0001-setup-Fix-the-version-to-2.2.0.patch
+Patch0002:          0002-config-Fix-the-config-to-pick-local-if-needed.patch
+Patch0003:          0003-tests-Fix-the-tests-for-the-broken-links.patch
 
 BuildArch:          noarch
 
+BuildRequires:      git
 BuildRequires:      python2-devel
 BuildRequires:      python2-setuptools
-BuildRequires:      python2-nose
 BuildRequires:      python2-mock
 
 BuildRequires:      fedmsg
+BuildRequires:      euca2ools
 BuildRequires:      python2-libcloud
 BuildRequires:      python2-paramiko
 BuildRequires:      python2-fedfind
 BuildRequires:      python2-toml
+BuildRequires:      python2-pytest-runner
+BuildRequires:      python-vcrpy
+BuildRequires:      python2-service-identity
+BuildRequires:      python2-qpid
+BuildRequires:      python2-pytest-cov
+
 
 
 %global _description\
@@ -37,18 +47,26 @@ Requires:           python2-libcloud
 Requires:           python2-paramiko
 Requires:           python2-fedfind
 Requires:           python2-toml
+Requires:           python2-pytest-runner
+Requires:           python-vcrpy
 %{?python_provide:%python_provide python2-fedimg}
 
 %description -n python2-fedimg %_description
 
 %prep
 %setup -q -n %{modname}-%{version}
+git init
+git config user.email "sysadmin-fedimg-owner"
+git config user.name "Fedimg Maintainers"
+git add .
+git commit -a -q -m "%{version} baseline."
+[ -n "%{patches}" ] && git am %{patches}
 
 # Remove bundled egg-info in case it exists
 rm -rf %{modname}.egg-info
 
 %build
-%{__python} setup.py build
+%{__python2} setup.py build
 
 %install
 
@@ -57,20 +75,23 @@ rm -rf %{modname}.egg-info
 %{__cp} -p fedimg-conf.toml.example %{buildroot}%{_sysconfdir}/%{modname}/fedimg-conf.toml
 %{__cp} -p fedmsg.d/fedimg.py %{buildroot}%{_sysconfdir}/fedmsg.d/.
 
-%{__python} setup.py install -O1 --skip-build --root=%{buildroot}
+%{__python2} setup.py install -O1 --skip-build --root=%{buildroot}
 
 %check
-%{__python} setup.py test
+%{__python2} setup.py test
 
 %files -n python2-fedimg
 %doc docs/ README.md LICENSE
 %config(noreplace) %{_sysconfdir}/%{modname}/fedimg-conf.toml
 %config(noreplace) %{_sysconfdir}/fedmsg.d/fedimg.py*
 
-%{python_sitelib}/%{modname}/
-%{python_sitelib}/%{modname}-%{version}*
+%{python2_sitelib}/%{modname}/
+%{python2_sitelib}/%{modname}-%{version}*
 
 %changelog
+* Tue Jul 24 2018 Sayan Chowdhury <sayanchowdhury> 2.2.0-1
+- Fix FTBFS bug #1605672
+
 * Sat Jul 14 2018 Fedora Release Engineering <releng> - 1.3.0-2
 - Rebuilt for https://fedoraproject.org/wiki/Fedora_29_Mass_Rebuild
 
* Link to scratch build on Koji: https://koji.fedoraproject.org/koji/taskinfo?taskID=28576875

Comment 7 Zbigniew Jędrzejewski-Szmek 2018-07-25 09:10:15 UTC
FWIW, the spec file changes look fine to me.

Comment 8 Sayan Chowdhury 2018-07-25 14:10:27 UTC
Pushed & Built: https://koji.fedoraproject.org/koji/taskinfo?taskID=28596356. Should I go ahead and close the issue?

Comment 9 Pierre-YvesChibon 2018-07-25 14:38:52 UTC
Yes :)

Comment 10 Pierre-YvesChibon 2018-07-25 14:39:51 UTC
And likely: https://bugzilla.redhat.com/show_bug.cgi?id=1540515 as well :)
(both can be closed: Rawhide I think)

Comment 11 Igor Raits 2018-07-25 16:22:12 UTC
There has been at least one successfull build after mass rebuild.

python-fedimg-2.3.0-1.fc29: https://koji.fedoraproject.org/koji/buildinfo?buildID=1130646


Note You need to log in before you can comment on or make changes to this bug.