1653777 – python3-molecule missing dependencies

Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1653777 - python3-molecule missing dependencies

Summary: python3-molecule missing dependencies

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Fedora
Classification:	Fedora
Component:	python-molecule
Sub Component:
Version:	31
Hardware:	x86_64
OS:	Unspecified
Priority:	unspecified
Severity:	unspecified
Target Milestone:	---
Assignee:	Brett Lentz
QA Contact:	Fedora Extras Quality Assurance
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2018-11-27 15:50 UTC by Nicolas Hicher
Modified:	2020-05-04 05:49 UTC (History)
CC List:	7 users (show)
Fixed In Version:	python-molecule-3.0.2-6.fc32 python-molecule-3.0.2-6.fc31
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2020-05-04 04:55:59 UTC
Type:	Bug
Embargoed:
Dependent Products:

Attachments	(Terms of Use)

Description Nicolas Hicher 2018-11-27 15:50:37 UTC

Description of problem:

After adding provisioner/ansible directory https://bugzilla.redhat.com/show_bug.cgi?id=1653467 I can run molecule test but destroy playbook failed

Version-Release number of selected component (if applicable):
python3-molecule-2.19-2.fc29.noarch

How reproducible:
Always

Steps to Reproduce:
dnf install python3-molecule yamllint python3-ansible-lint docker git -y
git clone https://github.com/ansible/molecule.git
rsync -av molecule/molecule/provisioner/ansible /usr/lib/python3.7/site-packages/molecule/provisioner/
molecule init role -r dummy
cd dummy
molecule --debug test


Actual results:
        "item": {
            "ansible_job_id": "890012419113.23428",
            "changed": true,
            "failed": false,
            "finished": 0,
            "item": {
                "image": "centos:7",
                "name": "instance"
            },
            "results_file": "/root/.ansible_async/890012419113.23428",
            "started": 1
        },
        "msg": "Failed to import docker or docker-py - No module named 'docker'. Try `pip install docker` or `pip install docker-py` (Python 2.6)",
        "stderr": "/root/.ansible/tmp/ansible-tmp-1543333514.6987603-200358869414890/AnsiballZ_docker_container.py:17: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses\n  import imp\n",
        "stderr_lines": [
            "/root/.ansible/tmp/ansible-tmp-1543333514.6987603-200358869414890/AnsiballZ_docker_container.py:17: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses",
            "  import imp"
        ]
    }
    
    PLAY RECAP *********************************************************************
    localhost                  : ok=1    changed=1    unreachable=0    failed=1   


Expected results:
Command should finish without error

Additional info:

Adding python3-docker fix this issue, but failed on another dependency:

--> Scenario: 'default'
--> Action: 'verify'
Traceback (most recent call last):
  File "/usr/bin/molecule", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python3.7/site-packages/click/core.py", line 721, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 1065, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.7/site-packages/click/core.py", line 894, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/lib/python3.7/site-packages/molecule/command/test.py", line 135, in test
    base.execute_subcommand(scenario.config, action)
  File "/usr/lib/python3.7/site-packages/molecule/command/base.py", line 100, in execute_subcommand
    return command(config).execute()
  File "/usr/lib/python3.7/site-packages/molecule/command/verify.py", line 72, in execute
    self._config.verifier.execute()
  File "/usr/lib/python3.7/site-packages/molecule/verifier/testinfra.py", line 183, in execute
    self.bake()
  File "/usr/lib/python3.7/site-packages/molecule/verifier/testinfra.py", line 162, in bake
    self._testinfra_command = sh.Command('py.test').bake(
  File "/usr/lib/python3.7/site-packages/sh.py", line 1202, in __init__
    raise CommandNotFound(path)
sh.CommandNotFound: py.test

Only python2-pytest provides py.test on fedora29, but installing the package raise another issue:

--> Action: 'verify'
--> Executing Testinfra tests found in /root/dummy/molecule/default/tests/...
usage: py.test [options] [file_or_dir] [file_or_dir] [...]
py.test: error: unrecognized arguments: --connection=ansible --ansible-inventory=/tmp/molecule/dummy/default/ansible_inventory.yml
  inifile: None
  rootdir: /root/dummy/molecule/default
An error occurred during the test sequence action: 'verify'. Cleaning up.
--> Scenario: 'default'
--> Action: 'destroy'

Comment 1 Nicolas Hicher 2018-11-27 16:16:25 UTC

It works if I install pip-3 pytest but with deprecation warnings:

[root@fedora-29 dummy]# pip-3 install   pytest                                                                                     
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip-3 install --user` instead.                
Collecting pytest                                                                                                                  
  Using cached https://files.pythonhosted.org/packages/81/27/d4302e4e00497448081120f65029696070806bc8e649b83f644de006d710/pytest-4.0
.1-py2.py3-none-any.whl                
Requirement already satisfied: pluggy>=0.7 in /usr/local/lib/python3.7/site-packages (from pytest) (0.8.0)                         
Requirement already satisfied: attrs>=17.4.0 in /usr/lib/python3.7/site-packages (from pytest) (17.4.0)                            
Requirement already satisfied: more-itertools>=4.0.0 in /usr/lib/python3.7/site-packages (from pytest) (4.1.0)                     
Requirement already satisfied: six>=1.10.0 in /usr/lib/python3.7/site-packages (from pytest) (1.11.0)
Requirement already satisfied: setuptools in /usr/lib/python3.7/site-packages (from pytest) (40.4.3)                                Requirement already satisfied: atomicwrites>=1.0 in /usr/lib/python3.7/site-packages (from pytest) (1.1.5)                         
Requirement already satisfied: py>=1.5.0 in /usr/lib/python3.7/site-packages (from pytest) (1.5.4)
molecule 2.19 requires python-gilt==1.2.1, which is not installed.
molecule 2.19 has requirement colorama==0.3.9, but you'll have colorama 0.4.0 which is incompatible.
molecule 2.19 has requirement pbr==4.1.0, but you'll have pbr 4.1.1 which is incompatible.                                          
molecule 2.19 has requirement psutil==5.4.6, but you'll have psutil 5.4.3 which is incompatible.                                   
molecule 2.19 has requirement PyYAML==3.13, but you'll have pyyaml 4.2b4 which is incompatible.
molecule 2.19 has requirement testinfra==1.16.0, but you'll have testinfra 1.17.0 which is incompatible.
molecule 2.19 has requirement tree-format==0.1.2, but you'll have tree-format 0.1.0 which is incompatible.
molecule 2.19 has requirement yamllint==1.11.1, but you'll have yamllint 1.12.1 which is incompatible.                              
Installing collected packages: pytest                                                                                              
Successfully installed pytest-4.0.1                                                                                                
[root@fedora-29 dummy]# pip-3 install  --user pytest                                
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip-3 install --user` instead.                
Requirement already satisfied: pytest in /usr/local/lib/python3.7/site-packages (4.0.1)                                             
Requirement already satisfied: six>=1.10.0 in /usr/lib/python3.7/site-packages (from pytest) (1.11.0)                              
Requirement already satisfied: setuptools in /usr/lib/python3.7/site-packages (from pytest) (40.4.3)
Requirement already satisfied: pluggy>=0.7 in /usr/local/lib/python3.7/site-packages (from pytest) (0.8.0)                          
Requirement already satisfied: attrs>=17.4.0 in /usr/lib/python3.7/site-packages (from pytest) (17.4.0)                            
Requirement already satisfied: py>=1.5.0 in /usr/lib/python3.7/site-packages (from pytest) (1.5.4)                                 
Requirement already satisfied: atomicwrites>=1.0 in /usr/lib/python3.7/site-packages (from pytest) (1.1.5)
Requirement already satisfied: more-itertools>=4.0.0 in /usr/lib/python3.7/site-packages (from pytest) (4.1.0)                     
[root@fedora-29 dummy]# molecule test  

--> Executing Testinfra tests found in /root/dummy/molecule/default/tests/...                                              [36/5708]
    ============================= test session starts ==============================                                               
    platform linux -- Python 3.7.1, pytest-4.0.1, py-1.5.4, pluggy-0.8.0                                                           
    rootdir: /root/dummy/molecule/default, inifile:                                                                                 
    plugins: testinfra-1.17.0, xdist-1.22.5, forked-0.2, cov-2.5.1                                                                 
collected 1 item                                                                                                                    
                                       
    tests/test_default.py .                                                  [100%]                                                
                                                                                                                                    
    =============================== warnings summary ===============================                                               
    /usr/lib/python3.7/site-packages/jinja2/utils.py:485                                             
      /usr/lib/python3.7/site-packages/jinja2/utils.py:485: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working                                                        
        from collections import MutableMapping                                                    
                                                                  
    /usr/lib64/python3.7/site-packages/markupsafe/__init__.py:13                                    
      /usr/lib64/python3.7/site-packages/markupsafe/__init__.py:13: DeprecationWarning: Using or importing the ABCs from 'collection
s' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working                                                
        from collections import Mapping                                                        
                                                                                                        
    /usr/lib64/python3.7/site-packages/yaml/constructor.py:126                                            
      /usr/lib64/python3.7/site-packages/yaml/constructor.py:126: DeprecationWarning: Using or importing the ABCs from 'collections'
 instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working                                                  
        if not isinstance(key, collections.Hashable):                                                                              
                                                                                    
    /usr/lib/python3.7/site-packages/ansible/utils/unsafe_proxy.py:56                                                              
      /usr/lib/python3.7/site-packages/ansible/utils/unsafe_proxy.py:56: DeprecationWarning: Using or importing the ABCs from 'colle
ctions' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working                                           
        from collections import Mapping, MutableSequence, Set                                       
      /usr/lib/python3.7/site-packages/ansible/utils/unsafe_proxy.py:56: DeprecationWarning: Using or importing the ABCs from 'colle
ctions' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working                                           
        from collections import Mapping, MutableSequence, Set
    /usr/lib/python3.7/site-packages/ansible/utils/plugin_docs.py:23                                                               
      /usr/lib/python3.7/site-packages/ansible/utils/plugin_docs.py:23: DeprecationWarning: Using or importing the ABCs from 'collec
tions' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working                                            
        from collections import MutableMapping, MutableSet, MutableSequence                                                        
                                                                                    
    /usr/lib/python3.7/site-packages/ansible/utils/listify.py:22
      /usr/lib/python3.7/site-packages/ansible/utils/listify.py:22: DeprecationWarning: Using or importing the ABCs from 'collection
s' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
        from collections import Iterable

    tests/test_default.py::test_hosts_file[ansible://instance]
      /usr/lib/python3.7/site-packages/socks.py:58: DeprecationWarning: Using or importing the ABCs from 'collections' instead of fr
om 'collections.abc' is deprecated, and in 3.8 it will stop working
        from collections import Callable
      /usr/lib64/python3.7/site-packages/ruamel/yaml/comments.py:14: DeprecationWarning: Using or importing the ABCs from 'collectio
ns' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
        from collections import MutableSet, Sized, Set

    -- Docs: https://docs.pytest.org/en/latest/warnings.html
    ===================== 1 passed, 9 warnings in 6.14 seconds =====================
Verifier completed successfully.
--> Scenario: 'default'
--> Action: 'destroy'

Comment 2 Brett Lentz 2018-11-28 14:40:46 UTC

The python-docker package isn't a strict dependency, as molecule supports multiple ways of operating. e.g. Running your molecule tests on AWS would require boto and would not require python-docker.

I can add some weak deps (Recommends) for the various drivers that molecule supports. That should help make the user experience a little nicer if you just want to take all of the defaults. But, it'll bloat out the install footprint and inevitably include support for many more options than just docker. e.g. Few people will require installing the libraries to run on AWS, GCE, Azure, OpenStack, LXD, Vagrant, and Docker.

The py.test issue is not a molecule issue. It's an issue for the testinfra package to resolve. Please open a bug against that package for this part of the issue.

Comment 3 Nicolas Hicher 2018-11-28 15:47:29 UTC

Hello Brett,

I agree that's python-docker is not a strict dependency, however, it's the default driver for molecule. But I agree, a weak deps should do the job.

I will create a bug report for testinfra, thanks for your review.

Comment 4 Nicolas Hicher 2018-11-28 18:16:53 UTC

I created a bug report on python-testinfra: https://bugzilla.redhat.com/show_bug.cgi?id=1654418

Comment 5 Nicolas Hicher 2018-11-28 18:42:58 UTC

I found what is the issue, python3-molecule depends on python3-testinfra, and python3-testinfra depends on python3-pytest.

python3-python contains only:

/usr/bin/py.test-3

And molecule/verifier/testinfra.py contains:

self._testinfra_command = sh.Command('py.test').bake(...

If I replace py.test by py.test-3 everything is ok.

I will close the testinfra bug, thanks.

Comment 6 Nicolas Hicher 2018-11-28 19:00:05 UTC

I tried to install python2-pytest, but molecule failed with:

    ============================= test session starts ==============================
    platform linux2 -- Python 2.7.15, pytest-3.6.4, py-1.5.4, pluggy-0.6.0
    rootdir: /root/dummy/molecule/default, inifile:
    plugins: testinfra-1.17.0, xdist-1.22.5, forked-0.2, cov-2.5.1
collected 0 items / 1 errors                                                   
    
    ==================================== ERRORS ====================================
    ____________________ ERROR collecting tests/test_default.py ____________________
    tests/test_default.py:3: in <module>
        import testinfra.utils.ansible_runner
    /usr/lib/python2.7/site-packages/_pytest/assertion/rewrite.py:226: in load_module
        py.builtin.exec_(co, mod.__dict__)
    /usr/lib/python2.7/site-packages/testinfra/utils/ansible_runner.py:28: in <module>
        "You must install ansible package to use the ansible backend")
    E   RuntimeError: You must install ansible package to use the ansible backend
    !!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
    =========================== 1 error in 0.14 seconds ============================

Comment 7 Fedora Update System 2019-08-31 16:04:42 UTC

FEDORA-2019-95a631828d has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-95a631828d

Comment 8 Ben Cotton 2019-10-31 18:53:37 UTC

This message is a reminder that Fedora 29 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '29'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 29 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 9 Drew Northup 2019-11-01 17:16:24 UTC

This bug also applies to Fedora 30 and most likely also Fedora 31. Please fix.

Comment 10 Ben Cotton 2019-11-27 18:15:39 UTC

Fedora 29 changed to end-of-life (EOL) status on 2019-11-26. Fedora 29 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 11 Fedora Update System 2020-02-24 14:06:40 UTC

FEDORA-2020-d83a251120 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-d83a251120

Comment 12 Michael DePaulo 2020-04-21 14:39:39 UTC

The missing dependency python-click-help-colors is in testing in bodhi right now, per this package request:
https://bugzilla.redhat.com/show_bug.cgi?id=1821158
https://bodhi.fedoraproject.org/updates/?packages=python-click-help-colors

Comment 13 Drew Northup 2020-04-21 15:27:56 UTC

@Michael DePaulo Did you mean to annotate a different ticket?

With respect to F31 [1] there are three package build errors left that don't seem to have anything to do with python-click-help-colors. The molecule packages that are for F32 & F33 look good in bodhi.

Am I missing something?


https://taskotron.fedoraproject.org/artifacts/all/b50040e2-570c-11ea-abcc-525400364adf/tests.yml/python-molecule-3.0.2-3.fc31.log

Comment 14 Fedora Update System 2020-04-25 15:14:11 UTC

FEDORA-2020-da7af6e03f has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2020-da7af6e03f

Comment 15 Fedora Update System 2020-04-25 15:14:12 UTC

FEDORA-2020-0c221024ae has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-0c221024ae

Comment 16 Fedora Update System 2020-04-25 15:14:13 UTC

FEDORA-2020-fd94e0d12d has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-fd94e0d12d

Comment 17 Fedora Update System 2020-04-26 04:15:08 UTC

FEDORA-2020-0c221024ae has been pushed to the Fedora 31 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-0c221024ae`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-0c221024ae

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 18 Fedora Update System 2020-04-26 04:29:09 UTC

FEDORA-2020-fd94e0d12d has been pushed to the Fedora 32 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-fd94e0d12d`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-fd94e0d12d

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 19 Fedora Update System 2020-04-26 04:51:21 UTC

FEDORA-2020-da7af6e03f has been pushed to the Fedora 30 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-da7af6e03f`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-da7af6e03f

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 20 Fedora Update System 2020-05-04 04:55:59 UTC

FEDORA-2020-fd94e0d12d has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 21 Fedora Update System 2020-05-04 05:49:09 UTC

FEDORA-2020-0c221024ae has been pushed to the Fedora 31 stable repository.
If problem still persists, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.