Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1708004 - python-sqlalchemy fails to build with Python 3.8
Summary: python-sqlalchemy fails to build with Python 3.8
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: python-sqlalchemy
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: Nils Philippsen
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON38
TreeView+ depends on / blocked
 
Reported: 2019-05-08 23:02 UTC by Miro Hrončok
Modified: 2019-05-17 21:08 UTC (History)
6 users (show)

Fixed In Version: python3-3.8.0~a4-2.fc31
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-05-17 09:27:30 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Full log from Copr (1.30 MB, text/plain)
2019-05-08 23:02 UTC, Miro Hrončok
no flags Details

Description Miro Hrončok 2019-05-08 23:02:29 UTC
Created attachment 1565837 [details]
Full log from Copr

python-sqlalchemy-1.3.3-1.fc31 fails to build with Python 3.8.0a4 due to a DeprecationWarning being treated as error:

+ /usr/bin/python3 -m pytest test
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 207, in wrap_session
INTERNALERROR>     config.hook.pytest_sessionstart(session=session)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 68, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 59, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/fixtures.py", line 61, in pytest_sessionstart
INTERNALERROR>     session._fixturemanager = FixtureManager(session)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/fixtures.py", line 1119, in __init__
INTERNALERROR>     session.config.pluginmanager.register(self, "funcmanage")
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 316, in register
INTERNALERROR>     ret = super(PytestPluginManager, self).register(plugin, name)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 95, in register
INTERNALERROR>     hookimpl = HookImpl(plugin, plugin_name, method, hookimpl_opts)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 344, in __init__
INTERNALERROR>     self.argnames, self.kwargnames = varnames(self.function)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 167, in varnames
INTERNALERROR>     spec = _getargspec(func)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 130, in _getargspec
INTERNALERROR>     return inspect.getfullargspec(func)
INTERNALERROR>   File "/usr/lib64/python3.8/inspect.py", line 1114, in getfullargspec
INTERNALERROR>     warnings.warn("Use inspect.signature() instead of inspect.getfullargspec()",
INTERNALERROR> DeprecationWarning: Use inspect.signature() instead of inspect.getfullargspec()

Full log attached.

Comment 1 Miro Hrončok 2019-05-09 11:38:48 UTC
See also https://github.com/pytest-dev/pluggy/issues/209

Comment 2 Michael Bayer 2019-05-09 13:56:02 UTC
as the link above implies that seems to be originating from py.test, but SQLAlchemy uses getfullargspec() also internally, not sure if the warning would just move.  getfullargspec() is already a Py3k-only method I had to move from getargspec(), I wish they'd stop changing this.

Comment 3 Miro Hrončok 2019-05-09 14:12:32 UTC
What bothers me is that a DeprecationWarning renders the entire thing failed :(

Comment 4 Michael Bayer 2019-05-09 17:53:17 UTC
so this should be tunable via the warnings filter and possibly a py.test option in setup.cfg. OTOH if pluggy is forcing this to behave this way then they'd have to fix it or fedora needs to patch it.

Comment 5 Miro Hrončok 2019-05-09 18:10:25 UTC
Let's wait a bit for pluggy upstream response I guess. It's in pytest-dev GitHub org, so if pytest is forcing it, they should know as well.

Comment 6 Miro Hrončok 2019-05-13 09:15:57 UTC
(In reply to Michael Bayer from comment #2)
> as the link above implies that seems to be originating from py.test, but
> SQLAlchemy uses getfullargspec() also internally, not sure if the warning
> would just move.  getfullargspec() is already a Py3k-only method I had to
> move from getargspec(), I wish they'd stop changing this.

Apparently, you are not alone. If I get this right, this will not be deprecated in 3.8:

https://bugs.python.org/issue36751
https://github.com/python/cpython/pull/13245

Comment 7 Miro Hrončok 2019-05-15 12:14:41 UTC
Pluggy 0.11.1 will include a fix for this.

Comment 8 Miro Hrončok 2019-05-16 16:38:26 UTC
Heh, but sqlalchemy has that as well:

Traceback (most recent call last):
  File "/usr/lib64/python3.8/runpy.py", line 192, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib64/python3.8/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.8/site-packages/pytest.py", line 89, in <module>
    raise SystemExit(pytest.main())
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 79, in main
    return config.hook.pytest_cmdline_main(config=config)
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 296, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 68, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 59, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 242, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 234, in wrap_session
    config.hook.pytest_sessionfinish(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 296, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 68, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 59, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 656, in pytest_sessionfinish
    outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/testing/plugin/pytestplugin.py", line 98, in pytest_sessionfinish
    plugin_base.final_process_cleanup()
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/testing/plugin/plugin_base.py", line 565, in final_process_cleanup
    engines.testing_reaper._stop_test_ctx_aggressive()
  File "<string>", line 2, in _stop_test_ctx_aggressive
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/testing/assertions.py", line 124, in decorate
    return fn(*args, **kw)
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/testing/engines.py", line 97, in _stop_test_ctx_aggressive
    rec.dispose()
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/engine/base.py", line 2014, in dispose
    self.pool = self.pool.recreate()
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/pool/impl.py", line 298, in recreate
    return self.__class__(
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/pool/impl.py", line 290, in __init__
    Pool.__init__(self, creator, **kw)
  File "<string>", line 2, in __init__
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/util/deprecations.py", line 130, in warned
    return fn(*args, **kwargs)
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/pool/base.py", line 202, in __init__
    self._creator = creator
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/pool/base.py", line 238, in _creator
    self._invoke_creator = self._should_wrap_creator(creator)
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/pool/base.py", line 247, in _should_wrap_creator
    argspec = util.get_callable_argspec(self._creator, no_self=True)
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/util/langhelpers.py", line 351, in get_callable_argspec
    return compat.inspect_getfullargspec(fn)
  File "/usr/lib64/python3.8/inspect.py", line 1114, in getfullargspec
    warnings.warn("Use inspect.signature() instead of inspect.getfullargspec()",
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/testing/assertions.py", line 152, in our_warn
    return real_warn(msg, *arg, **kw)
DeprecationWarning: Use inspect.signature() instead of inspect.getfullargspec()

Comment 9 Michael Bayer 2019-05-16 17:17:20 UTC
yes see comment #2 as well as https://github.com/sqlalchemy/sqlalchemy/issues/4674

Comment 10 Michael Bayer 2019-05-16 17:53:22 UTC
see also https://github.com/sqlalchemy/alembic/issues/563 as people that use SQLAlchemy usually use Alembic as well as https://github.com/sqlalchemy/mako/issues/295 which Alembic uses.

Comment 11 Miro Hrončok 2019-05-16 19:39:54 UTC
See https://src.fedoraproject.org/rpms/python-tornado/pull-request/5 - that is an example of a probably simplest course of action for a downstream maintainer.

Comment 12 Miro Hrončok 2019-05-16 19:42:34 UTC
lib/sqlalchemy/testing/warnings.py has:

def setup_filters():
    """Set global warning behavior for the test suite."""

    warnings.filterwarnings(
        "ignore", category=sa_exc.SAPendingDeprecationWarning
    )
    warnings.filterwarnings("error", category=sa_exc.SADeprecationWarning)
    warnings.filterwarnings("error", category=sa_exc.SAWarning)

    # some selected deprecations...
    warnings.filterwarnings("error", category=DeprecationWarning)
    warnings.filterwarnings(
        "ignore", category=DeprecationWarning, message=".*StopIteration"
    )
    warnings.filterwarnings(
        "ignore", category=DeprecationWarning, message=".*inspect.getargspec"
    )

I suppose we can sed s/getargspec/*/ and call it a day.

Comment 13 Miro Hrončok 2019-05-16 19:49:06 UTC
Testing a build with:

%prep
...
# https://bugzilla.redhat.com/show_bug.cgi?id=1708004
sed -i 's/getargspec//' lib/sqlalchemy/testing/warnings.py


It is enough to make the tests at least run.

Comment 14 Michael Bayer 2019-05-16 19:54:17 UTC
yes if you can do that on the package side here, that is your best course right now.

Comment 15 Miro Hrončok 2019-05-16 19:57:00 UTC
A test failure:

=================================== FAILURES ===================================
_ ComponentReflectionTest_sqlite+pysqlite_3_28_0.test_deprecated_get_primary_keys _
Traceback (most recent call last):
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/testing/suite/test_reflection.py", line 594, in test_deprecated_get_primary_keys
    users.name,
  File "/builddir/build/BUILD/SQLAlchemy-1.3.3/test/../lib/sqlalchemy/testing/assertions.py", line 309, in assert_raises_message
    assert False, "Callable did not raise an exception"
AssertionError: Callable did not raise an exception
=========================== short test summary info ============================


It speaks of deprecation, will check the test code.

Comment 16 Miro Hrončok 2019-05-16 20:10:19 UTC
It fails even with: sed -i 's/getargspec/get(full)?argspec/' lib/sqlalchemy/testing/warnings.py

Comment 17 Miro Hrončok 2019-05-16 20:47:05 UTC
For now, upstream has removed the DeprecationWarning. I will do the same and later upload an up to date build.log if it keeps failing.

Comment 18 Michael Bayer 2019-05-16 21:51:19 UTC
that test is looking for:

 sa_exc.SADeprecationWarning,
            r".*get_primary_keys\(\) method is deprecated",

Comment 19 Miro Hrončok 2019-05-17 09:27:30 UTC
> For now, upstream has removed the DeprecationWarning. I will do the same and later upload an up to date build.log if it keeps failing.

All good now. Thank you, Michael.

Comment 20 Michael Bayer 2019-05-17 21:08:28 UTC
what a drama.  I'm going to pursue changing getfullargspec() in any case since the one in Python 3 is very slow.


Note You need to log in before you can comment on or make changes to this bug.