Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 2113503 - mailman3: FTBFS in Fedora rawhide/f37
Summary: mailman3: FTBFS in Fedora rawhide/f37
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: mailman3
Version: 37
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Michel Lind
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 1852603
Blocks: F38FTBFS F39FTBFS F37FTBFS
TreeView+ depends on / blocked
 
Reported: 2022-08-01 23:54 UTC by Fedora Release Engineering
Modified: 2023-05-15 01:47 UTC (History)
7 users (show)

Fixed In Version: mailman3-3.3.8-1.fc38 mailman3-3.3.8-1.fc37
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-05-15 01:27:11 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
build.log (deleted)
2022-08-01 23:54 UTC, Fedora Release Engineering
no flags Details
root.log (deleted)
2022-08-01 23:54 UTC, Fedora Release Engineering
no flags Details
state.log (deleted)
2022-08-01 23:54 UTC, Fedora Release Engineering
no flags Details

Description Fedora Release Engineering 2022-08-01 23:54:26 UTC
mailman3 failed to build from source in Fedora rawhide/f37

https://koji.fedoraproject.org/koji/taskinfo?taskID=89826489


For details on the mass rebuild see:

https://fedoraproject.org/wiki/Fedora_37_Mass_Rebuild
Please fix mailman3 at your earliest convenience and set the bug's status to
ASSIGNED when you start fixing it. If the bug remains in NEW state for 8 weeks,
mailman3 will be orphaned. Before branching of Fedora 38,
mailman3 will be retired, if it still fails to build.

For more details on the FTBFS policy, please visit:
https://docs.fedoraproject.org/en-US/fesco/Fails_to_build_from_source_Fails_to_install/

Comment 1 Fedora Release Engineering 2022-08-01 23:54:28 UTC
Created attachment 1901934 [details]
build.log

Comment 2 Fedora Release Engineering 2022-08-01 23:54:30 UTC
Created attachment 1901935 [details]
root.log

file root.log too big, will only attach last 32768 bytes

Comment 3 Fedora Release Engineering 2022-08-01 23:54:31 UTC
Created attachment 1901936 [details]
state.log

Comment 4 Ben Cotton 2022-08-09 13:30:37 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 37 development cycle.
Changing version to 37.

Comment 5 romulasry 2023-01-28 04:00:57 UTC
3.3.8 is the latest:
Trying to get it to update, bug here: https://github.com/fedora-infra/anitya/issues/1535

Comment 6 Peter Bieringer 2023-03-23 07:48:48 UTC
It looks like mailman3 has too many python module dependencies which makes it very hard to solve future builds if one or the other dependency is not available as RPM or not up-to-date.

I'm currently working on a "mailman3-virtualenv" all-in-one RPM based on instructions from https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-install

Goal: getting reproducable RPM without requiring PIP dependency on installation, this is done by special download option in SPEC which results in some tar balls which will be then packaged in the SRPMS...one of it is containing all PIP-dependencies.

Target OS for testing is EL9 (as here the python dependency "hell" is even more difficult to fix), but this would also work on Fedora.

Hopefully finished in 2 weeks from now latest, will make sneak preview accessable via COPR then.

Comment 7 Peter Bieringer 2023-04-03 06:26:46 UTC
First major step done, working RPMs for EL8 and EL9 were created:

https://koji.fedoraproject.org/koji/taskinfo?taskID=99463464
https://koji.fedoraproject.org/koji/taskinfo?taskID=99464433

It creates new user/group and should be able to run in parallel on an EL8 system with mailman version 2 for local migration (which is untested so far by me, but planned).
Network ports were adjuste to avoid any conflicts with already existing definitions in SELinux

=> please test and comment, there are now 2 options for making this public available:
Preferred: request new package on Fedora and EPEL named "mailman3-virtualenv"
 -> would potentially need some "voters"

Fallback: create github repo "mailman3-virtualenv" and use this as feed for personal COPR for the time being



Details from RPM info:

user/group   : mailman3/mailman3
directory    : /usr/lib/mailman3/venv

it contains also
 mailman-web       : 0.0.6 using 'gunicorn'
 mailman-hyperkitty: 1.2.1

important forms are extended with CAPTCHA protection
 recaptcha hcaptcha friendlycaptcha

default configuration
 database: sqlite3
 indexer : whoosh
 CAPTCHA : NONE

preconfigured ports (required for SELinux)
LMTP         : 8024
REST-API     : 8027
Web Interface: 8026

Comment 8 Neal Gompa 2023-04-03 11:49:42 UTC
We don't do virtualenv bundle packages in Fedora.

Comment 9 Peter Bieringer 2023-04-04 06:43:58 UTC
(In reply to Neal Gompa from comment #8)
> We don't do virtualenv bundle packages in Fedora.

Can make sense, a short cross check shows that something between 10 and 20% of required Python modules are still missing in Fedora 37 still for 3.3.8 with CAPTCHA support enabled

But for Enterprise Linux the missing ones are approx. 40% (EL8 with EPEL enabled) and 50% (EL9 with EPEL enabled) -> if anyone want to run "mailman3" on EL 8/9 with a reproducable RPM and not relying to install via PIP directly on a production server then I see no alternative to packaging a virtualenv based one into a RPM.

Comment 10 Peter Bieringer 2023-04-06 15:12:53 UTC
while extending my local spec file to be able to create "virtualenv" or a standard edition I found several deadlocks:

mailman-3.3.8/setup.py:        'flufl.bounce>=4.0',
mailman-3.3.8/setup.py:        'flufl.i18n>=3.2',
mailman-3.3.8/setup.py:        'flufl.lock>=5.1',

Fedora 37:
python3-flufl-bounce.noarch 3.0-17.fc37   fedora *** req >= 4.0
python3-flufl-i18n.noarch   2.0.2-10.fc37 fedora *** req >= 3.2
python3-flufl-lock.noarch   6.0-1.fc37    fedora

=> upgrade on F37 not possible until major updates of dependent packages were done


On EL9 also a DEADLOCK found in the version dependency of
  readme-renderer -> cmarkgfm -> cffi

Is there any solution for solving this Python dependency hell for a particular application?

Bundling required but not in base or EPEL existing packages into "site-packages" (and define __provides_exclude_from + __requires_exclude) is a workaround...
...but not working for packages already existing on the system in a lower version.

Comment 11 Peter Bieringer 2023-04-06 17:04:44 UTC
Fedora 38 is not improving the situation:
python3-flufl-bounce.noarch  3.0-18.fc38   fedora
python3-flufl-i18n.noarch    2.0.2-11.fc38 fedora

looks like mailman version 3 package has to be marked as stale in F37 and discontinued in F38 and newer as long there is no solution found for packaging additional "local" packages within which are preferred against the ones in "site-packages"

Comment 13 Peter Bieringer 2023-04-23 15:44:42 UTC
It took some time but now it's done, mailman 3.3.8 is now available for Fedora 37/38 and EL8/9 on my copr repository using USER_SITE support in Python to package all dependencies which are not available via OS/EPEL repository.

https://copr.fedorainfracloud.org/coprs/pbiering/InternetServerExtensions/packages/

the spec file has some magic inside and is also able to package on-demand a virtualenv version, cron jobs instead of systemd timers and use same user as mailman major version 2

More details can be found here:
https://github.com/pbiering/mailman3-rpm

At least on a test system I was able to install mailman3 in parallel to mailman2 and converted one list (migration instructions are still candidate to be published as the ones on mailman3 website are not that accurate).

Regarding the future, if all are fine with that USER_SITE approach then one can think of making mailman3 also available on EPEL and adjust the one on Fedora.

Comment 14 romulasry 2023-05-01 23:13:32 UTC
python-flufl-bounce

Bugzilla Assignee:

Fedora:
    abompard 

python-flufl-i18n

Fedora:
    abompard 

Maybe you could contact him?

Comment 15 romulasry 2023-05-01 23:15:52 UTC
https://aurelien.bompard.org/

I will see if I can contact him.

Comment 16 romulasry 2023-05-01 23:17:56 UTC
Found his email here: https://github.com/abompard

Comment 17 romulasry 2023-05-01 23:20:35 UTC
I sent him an email notifying him about these two packages: https://bugzilla.redhat.com/show_bug.cgi?id=2113503#c12

Thanks for your work.

Comment 18 Peter Bieringer 2023-05-02 05:26:29 UTC
Hmm, these 2 modules are not the only ones which needs to be adjusted...

Currently need to bundle on Fedora:

# Mailman3 directly related
django_mailman3
mailman
mailman_hyperkitty
mailman_web
postorius

# Dependencies
aiosmtpd
appconf
asgiref
flufl
flufl.bounce
flufl.i18n
haystack
hyperkitty
importlib_resources
networkx
whoosh

# CAPTCHA support:
captcha
hcaptcha
friendly_captcha

# Injection of CAPTCHA support into web interfaces
allauth
django
django_extensions
django_q
picklefield
rest_framework
compressor

Comment 19 Aurelien Bompard 2023-05-02 08:50:04 UTC
Hey folks!
I'm happy to add as co-maintainer anyone that wants to work on the packages that mailman3 depends on (and that I "own").

(also FYI you can contact people with Fedora accounts at <username>@fedoraproject.org ;-) )

Comment 20 Fedora Update System 2023-05-13 22:43:17 UTC
FEDORA-2023-db3f3a88e0 has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2023-db3f3a88e0

Comment 21 Fedora Update System 2023-05-13 22:43:47 UTC
FEDORA-2023-151627e86c has been submitted as an update to Fedora 37. https://bodhi.fedoraproject.org/updates/FEDORA-2023-151627e86c

Comment 22 Fedora Update System 2023-05-14 02:49:12 UTC
FEDORA-2023-151627e86c has been pushed to the Fedora 37 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-151627e86c`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-151627e86c

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 23 Fedora Update System 2023-05-14 03:05:46 UTC
FEDORA-2023-db3f3a88e0 has been pushed to the Fedora 38 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-db3f3a88e0`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-db3f3a88e0

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 24 Fedora Update System 2023-05-15 01:27:11 UTC
FEDORA-2023-db3f3a88e0 has been pushed to the Fedora 38 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 25 Fedora Update System 2023-05-15 01:47:56 UTC
FEDORA-2023-151627e86c has been pushed to the Fedora 37 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.