Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1605672

Summary: python-fedimg: FTBFS in Fedora rawhide
Product: [Fedora] Fedora Reporter: Mohan Boddu <mboddu>
Component: python-fedimgAssignee: Fedora Infrastructure SIG <infra-sig>
Status: CLOSED NEXTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: infra-sig, kevin, pingou, rbean, sayan.chowdhury2012, zbyszek
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-07-25 16:22:12 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1602938    
Attachments:
Description Flags
build.log
none
root.log
none
state.log none

Description Mohan Boddu 2018-07-20 16:30:03 UTC 
python-fedimg failed to build from source in Fedora rawhide

https://koji.fedoraproject.org/koji/taskinfo?taskID=28221692


For details on the mass rebuild see:

https://fedoraproject.org/wiki/Fedora_29_Mass_Rebuild
Please fix python-fedimg at your earliest convenience and set the bug's status to
ASSIGNED when you start fixing it. If the bug remains in NEW state for 8 weeks,
python-fedimg will be orphaned. Before branching of Fedora 30,
python-fedimg will be retired, if it still fails to build.

For more details on the FTBFS policy, please visit:
https://fedoraproject.org/wiki/Fails_to_build_from_source
Comment 1 Mohan Boddu 2018-07-20 16:30:15 UTC 
Created attachment 1466176 [details]
build.log
Comment 2 Mohan Boddu 2018-07-20 16:30:20 UTC 
Created attachment 1466177 [details]
root.log

file root.log too big, will only attach last 32768 bytes
Comment 3 Mohan Boddu 2018-07-20 16:30:23 UTC 
Created attachment 1466178 [details]
state.log
Comment 4 Kevin Fenzi 2018-07-20 21:55:35 UTC 
I did some fixups here, but tests still fail.

See: 

https://koji.fedoraproject.org/koji/taskinfo?taskID=28478934

Which has: 

diff --git a/python-fedimg.spec b/python-fedimg.spec
index 55149d9..37028e7 100644
--- a/python-fedimg.spec
+++ b/python-fedimg.spec
@@ -2,7 +2,7 @@

 Name:               python-fedimg
 Version:            1.3.0
-Release:            2%{?dist}
+Release:            3%{?dist}
 Summary:            Automatically upload Fedora Cloud images to cloud providers                                      

 Group:              Development/Libraries
@@ -16,6 +16,9 @@ BuildRequires:      python2-devel
 BuildRequires:      python2-setuptools
 BuildRequires:      python2-nose
 BuildRequires:      python2-mock
+BuildRequires:      python2-vcrpy
+BuildRequires:      python2-service-identity
+BuildRequires:      python2-qpid

 BuildRequires:      fedmsg
 BuildRequires:      python2-libcloud
@@ -48,7 +51,7 @@ Requires:           python2-toml
 rm -rf %{modname}.egg-info

 %build
-%{__python} setup.py build
+%{__python2} setup.py build

 %install

@@ -57,10 +60,10 @@ rm -rf %{modname}.egg-info
 %{__cp} -p fedimg-conf.toml.example %{buildroot}%{_sysconfdir}/%{modname}/fedimg-conf.toml                           
 %{__cp} -p fedmsg.d/fedimg.py %{buildroot}%{_sysconfdir}/fedmsg.d/.                                                  

-%{__python} setup.py install -O1 --skip-build --root=%{buildroot}                                                    
+%{__python2} setup.py install -O1 --skip-build --root=%{buildroot}                                                   

 %check
-%{__python} setup.py test
+%{__python2} setup.py test

 %files -n python2-fedimg
 %doc docs/ README.md LICENSE
@@ -71,6 +74,9 @@ rm -rf %{modname}.egg-info
 %{python_sitelib}/%{modname}-%{version}*

 %changelog
+* Fri Jul 20 2018 Kevin Fenzi <kevin> - 1.3.0-3
+- Fix FTBFS bug #1605672
+
 * Sat Jul 14 2018 Fedora Release Engineering <releng> - 1.3.0-2                                    
 - Rebuilt for https://fedoraproject.org/wiki/Fedora_29_Mass_Rebuild                                                  

Seems like it's looking for the toml file in an installed directory, but the tests are being run from the rpm and would be using the stuff under the buildroot?

+ /usr/bin/python2 setup.py test
running test
running egg_info
writing requirements to fedimg.egg-info/requires.txt
writing fedimg.egg-info/PKG-INFO
writing top-level names to fedimg.egg-info/top_level.txt
writing dependency_links to fedimg.egg-info/dependency_links.txt
writing entry points to fedimg.egg-info/entry_points.txt
reading manifest file 'fedimg.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'CHANGELOG.md'
writing manifest file 'fedimg.egg-info/SOURCES.txt'
running build_ext
Failure: IOError ([Errno 2] No such file or directory: '/etc/fedimg/fedimg-conf.toml') ... ERROR
Failure: IOError ([Errno 2] No such file or directory: '/etc/fedimg/fedimg-conf.toml') ... ERROR
test_external_run_command (test_utils.TestFedimgUtils) ... ok
test_get_file_arch_aarch64 (test_utils.TestFedimgUtils) ... ok
test_get_file_arch_x86_64 (test_utils.TestFedimgUtils) ... ok
test_get_file_name_image (test_utils.TestFedimgUtils) ... ok
test_get_image_name_from_ami_name (test_utils.TestFedimgUtils) ... ok
test_get_image_name_from_ami_name_for_fedmsg (test_utils.TestFedimgUtils) ... ok
test_get_image_name_from_image (test_utils.TestFedimgUtils) ... ok
test_get_image_name_from_image_without_vol_type (test_utils.TestFedimgUtils) ... ok
test_get_item_from_regex (test_utils.TestFedimgUtils) ... ok
test_get_rawxz_urls (test_utils.TestFedimgUtils) ... ok
test_get_rawxz_urls_empty (test_utils.TestFedimgUtils) ... ok
test_get_source_from_image (test_utils.TestFedimgUtils) ... FAIL
test_get_source_from_image_retcode_1 (test_utils.TestFedimgUtils) ... ok
test_get_value_from_dict (test_utils.TestFedimgUtils) ... ok
test_get_virt_type_from_image_hvm (test_utils.TestFedimgUtils) ... ok
test_get_virt_types_hvm_pv (test_utils.TestFedimgUtils) ... ok
test_get_virt_types_only_hvm (test_utils.TestFedimgUtils) ... ok
test_get_volume_type_from_image (test_utils.TestFedimgUtils) ... ok
test_region_to_driver (test_utils.TestFedimgUtils) ... ok
test_ssh_connection_works_false (test_utils.TestFedimgUtils) ... ok
test_ssh_connection_works_true (test_utils.TestFedimgUtils) ... ok
======================================================================
ERROR: Failure: IOError ([Errno 2] No such file or directory: '/etc/fedimg/fedimg-conf.toml')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/builddir/build/BUILD/fedimg-1.3.0/tests/test_consumers.py", line 29, in <module>
    import fedimg.consumers
  File "/builddir/build/BUILD/fedimg-1.3.0/fedimg/consumers.py", line 36, in <module>
    import fedimg.uploader
  File "/builddir/build/BUILD/fedimg-1.3.0/fedimg/uploader.py", line 29, in <module>
    from fedimg.config import ACTIVE_SERVICES
  File "/builddir/build/BUILD/fedimg-1.3.0/fedimg/config.py", line 25, in <module>
    with open("/etc/fedimg/fedimg-conf.toml") as conffile:
IOError: [Errno 2] No such file or directory: '/etc/fedimg/fedimg-conf.toml'
======================================================================
ERROR: Failure: IOError ([Errno 2] No such file or directory: '/etc/fedimg/fedimg-conf.toml')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/builddir/build/BUILD/fedimg-1.3.0/tests/test_uploader.py", line 26, in <module>
    import fedimg.uploader
  File "/builddir/build/BUILD/fedimg-1.3.0/fedimg/uploader.py", line 29, in <module>
    from fedimg.config import ACTIVE_SERVICES
  File "/builddir/build/BUILD/fedimg-1.3.0/fedimg/config.py", line 25, in <module>
    with open("/etc/fedimg/fedimg-conf.toml") as conffile:
IOError: [Errno 2] No such file or directory: '/etc/fedimg/fedimg-conf.toml'
======================================================================
FAIL: test_get_source_from_image (test_utils.TestFedimgUtils)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "/builddir/build/BUILD/fedimg-1.3.0/tests/test_utils.py", line 240, in test_get_source_from_image
    mock.ANY
  File "/usr/lib/python2.7/site-packages/mock/mock.py", line 947, in assert_called_once_with
    raise AssertionError(msg)
AssertionError: Expected 'external_run_command' to be called once. Called 2 times.
-------------------- >> begin captured logging << --------------------
fedimg.utils: INFO: [PREP] Preparing temporary directory for download: '/tmp/tmpMiEEQC'
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 23 tests in 0.422s
FAILED (failures=1, errors=2)
Test failed: <unittest.runner.TextTestResult run=23 errors=2 failures=1>
error: Test failed: <unittest.runner.TextTestResult run=23 errors=2 failures=1>
Comment 5 Zbigniew Jędrzejewski-Szmek 2018-07-21 21:51:47 UTC 
The tests fail because they hardcode reading /etc/fedimg/fedimg-conf.toml, which means that not only the package BuildRequires itself, but the installed version must have a matching config.
Comment 6 Sayan Chowdhury 2018-07-24 21:48:06 UTC 
I did some changes which should fix this. 

The diff includes three patches:
- Fix the setup version
- Fix for the configuration path. If the configuration is not found it will look for it in the project root directory
- Fix the tests, update the broken vcr cassettes.

* Here is the diff of the spec file 

diff --git a/python-fedimg.spec b/python-fedimg.spec
index 55149d9..55e82f9 100644
--- a/python-fedimg.spec
+++ b/python-fedimg.spec
@@ -1,27 +1,37 @@
 %global modname fedimg
 
 Name:               python-fedimg
-Version:            1.3.0
-Release:            2%{?dist}
+Version:            2.2.0
+Release:            1%{?dist}
 Summary:            Automatically upload Fedora Cloud images to cloud providers
 
 Group:              Development/Libraries
 License:            AGPLv3+
 URL:                http://pypi.python.org/pypi/fedimg
 Source0:            http://pypi.python.org/packages/source/f/%{modname}/%{modname}-%{version}.tar.gz
+Patch0001:          0001-setup-Fix-the-version-to-2.2.0.patch
+Patch0002:          0002-config-Fix-the-config-to-pick-local-if-needed.patch
+Patch0003:          0003-tests-Fix-the-tests-for-the-broken-links.patch
 
 BuildArch:          noarch
 
+BuildRequires:      git
 BuildRequires:      python2-devel
 BuildRequires:      python2-setuptools
-BuildRequires:      python2-nose
 BuildRequires:      python2-mock
 
 BuildRequires:      fedmsg
+BuildRequires:      euca2ools
 BuildRequires:      python2-libcloud
 BuildRequires:      python2-paramiko
 BuildRequires:      python2-fedfind
 BuildRequires:      python2-toml
+BuildRequires:      python2-pytest-runner
+BuildRequires:      python-vcrpy
+BuildRequires:      python2-service-identity
+BuildRequires:      python2-qpid
+BuildRequires:      python2-pytest-cov
+
 
 
 %global _description\
@@ -37,18 +47,26 @@ Requires:           python2-libcloud
 Requires:           python2-paramiko
 Requires:           python2-fedfind
 Requires:           python2-toml
+Requires:           python2-pytest-runner
+Requires:           python-vcrpy
 %{?python_provide:%python_provide python2-fedimg}
 
 %description -n python2-fedimg %_description
 
 %prep
 %setup -q -n %{modname}-%{version}
+git init
+git config user.email "sysadmin-fedimg-owner"
+git config user.name "Fedimg Maintainers"
+git add .
+git commit -a -q -m "%{version} baseline."
+[ -n "%{patches}" ] && git am %{patches}
 
 # Remove bundled egg-info in case it exists
 rm -rf %{modname}.egg-info
 
 %build
-%{__python} setup.py build
+%{__python2} setup.py build
 
 %install
 
@@ -57,20 +75,23 @@ rm -rf %{modname}.egg-info
 %{__cp} -p fedimg-conf.toml.example %{buildroot}%{_sysconfdir}/%{modname}/fedimg-conf.toml
 %{__cp} -p fedmsg.d/fedimg.py %{buildroot}%{_sysconfdir}/fedmsg.d/.
 
-%{__python} setup.py install -O1 --skip-build --root=%{buildroot}
+%{__python2} setup.py install -O1 --skip-build --root=%{buildroot}
 
 %check
-%{__python} setup.py test
+%{__python2} setup.py test
 
 %files -n python2-fedimg
 %doc docs/ README.md LICENSE
 %config(noreplace) %{_sysconfdir}/%{modname}/fedimg-conf.toml
 %config(noreplace) %{_sysconfdir}/fedmsg.d/fedimg.py*
 
-%{python_sitelib}/%{modname}/
-%{python_sitelib}/%{modname}-%{version}*
+%{python2_sitelib}/%{modname}/
+%{python2_sitelib}/%{modname}-%{version}*
 
 %changelog
+* Tue Jul 24 2018 Sayan Chowdhury <sayanchowdhury> 2.2.0-1
+- Fix FTBFS bug #1605672
+
 * Sat Jul 14 2018 Fedora Release Engineering <releng> - 1.3.0-2
 - Rebuilt for https://fedoraproject.org/wiki/Fedora_29_Mass_Rebuild
 
* Link to scratch build on Koji: https://koji.fedoraproject.org/koji/taskinfo?taskID=28576875
Comment 7 Zbigniew Jędrzejewski-Szmek 2018-07-25 09:10:15 UTC 
FWIW, the spec file changes look fine to me.
Comment 8 Sayan Chowdhury 2018-07-25 14:10:27 UTC 
Pushed & Built: https://koji.fedoraproject.org/koji/taskinfo?taskID=28596356. Should I go ahead and close the issue?
Comment 9 Pierre-YvesChibon 2018-07-25 14:38:52 UTC 
Yes :)
Comment 10 Pierre-YvesChibon 2018-07-25 14:39:51 UTC 
And likely: https://bugzilla.redhat.com/show_bug.cgi?id=1540515 as well :)
(both can be closed: Rawhide I think)
Comment 11 Igor Raits 2018-07-25 16:22:12 UTC 
There has been at least one successfull build after mass rebuild.

python-fedimg-2.3.0-1.fc29: https://koji.fedoraproject.org/koji/buildinfo?buildID=1130646