Sadly, this version has grown a dep on tinycss... which is in Fedora, but only as python3-tinycss.

I suppose it's time to look at the end of the line since python2 is going away. ;(

I hope we can find a different way!

Latest upstream release: 3.39.0
Current version/release in rawhide: 3.36.0-2.fc30
URL: https://calibre-ebook.com/

Please consult the package updates policy before you issue an update to a stable branch: https://fedoraproject.org/wiki/Updates_Policy

More information about the service that created this bug can be found at: https://fedoraproject.org/wiki/Upstream_release_monitoring

Please keep in mind that with any upstream change, there may also be packaging changes that need to be made. Specifically, please remember that it is your responsibility to review the new version to ensure that the licensing is still correct and that no non-free or legally problematic items have been added upstream.

Based on the information from anitya:  https://release-monitoring.org/project/6141/

Latest upstream release: 3.39.1
Current version/release in rawhide: 3.36.0-2.fc30
URL: https://calibre-ebook.com/

Please consult the package updates policy before you issue an update to a stable branch: https://fedoraproject.org/wiki/Updates_Policy

More information about the service that created this bug can be found at: https://fedoraproject.org/wiki/Upstream_release_monitoring

Please keep in mind that with any upstream change, there may also be packaging changes that need to be made. Specifically, please remember that it is your responsibility to review the new version to ensure that the licensing is still correct and that no non-free or legally problematic items have been added upstream.

Based on the information from anitya:  https://release-monitoring.org/project/6141/

(In reply to Zbigniew Jędrzejewski-Szmek from comment #2)
> I hope we can find a different way!

Well, I am open to ideas, but:

* tinycss is in fedora, but as python3-tinycss. 
* We are not supposed to add python2 subpackages at this point. 
* python2 is going away after f31 anyhow
* calibre upstream has 0 plans to port to python3.

Latest upstream release: 3.40.0
Current version/release in rawhide: 3.36.0-4.fc31
URL: https://calibre-ebook.com/

Please consult the package updates policy before you issue an update to a stable branch: https://fedoraproject.org/wiki/Updates_Policy

More information about the service that created this bug can be found at: https://fedoraproject.org/wiki/Upstream_release_monitoring

Please keep in mind that with any upstream change, there may also be packaging changes that need to be made. Specifically, please remember that it is your responsibility to review the new version to ensure that the licensing is still correct and that no non-free or legally problematic items have been added upstream.

Based on the information from anitya:  https://release-monitoring.org/project/6141/

Latest upstream release: 3.40.1
Current version/release in rawhide: 3.36.0-4.fc31
URL: https://calibre-ebook.com/

Please consult the package updates policy before you issue an update to a stable branch: https://fedoraproject.org/wiki/Updates_Policy

More information about the service that created this bug can be found at: https://fedoraproject.org/wiki/Upstream_release_monitoring

Please keep in mind that with any upstream change, there may also be packaging changes that need to be made. Specifically, please remember that it is your responsibility to review the new version to ensure that the licensing is still correct and that no non-free or legally problematic items have been added upstream.

Based on the information from anitya:  https://release-monitoring.org/project/6141/

Latest upstream release: 3.41.0
Current version/release in rawhide: 3.36.0-4.fc31
URL: https://calibre-ebook.com/

Please consult the package updates policy before you issue an update to a stable branch: https://fedoraproject.org/wiki/Updates_Policy

More information about the service that created this bug can be found at: https://fedoraproject.org/wiki/Upstream_release_monitoring

Please keep in mind that with any upstream change, there may also be packaging changes that need to be made. Specifically, please remember that it is your responsibility to review the new version to ensure that the licensing is still correct and that no non-free or legally problematic items have been added upstream.

Based on the information from anitya:  https://release-monitoring.org/project/6141/

Latest upstream release: 3.41.1
Current version/release in rawhide: 3.36.0-4.fc31
URL: https://calibre-ebook.com/

Please consult the package updates policy before you issue an update to a stable branch: https://fedoraproject.org/wiki/Updates_Policy

More information about the service that created this bug can be found at: https://fedoraproject.org/wiki/Upstream_release_monitoring

Please keep in mind that with any upstream change, there may also be packaging changes that need to be made. Specifically, please remember that it is your responsibility to review the new version to ensure that the licensing is still correct and that no non-free or legally problematic items have been added upstream.

Based on the information from anitya:  https://release-monitoring.org/project/6141/

Latest upstream release: 3.41.2
Current version/release in rawhide: 3.36.0-4.fc31
URL: https://calibre-ebook.com/

Please consult the package updates policy before you issue an update to a stable branch: https://fedoraproject.org/wiki/Updates_Policy

More information about the service that created this bug can be found at: https://fedoraproject.org/wiki/Upstream_release_monitoring

Please keep in mind that with any upstream change, there may also be packaging changes that need to be made. Specifically, please remember that it is your responsibility to review the new version to ensure that the licensing is still correct and that no non-free or legally problematic items have been added upstream.

Based on the information from anitya:  https://release-monitoring.org/project/6141/

Latest upstream release: 3.41.3
Current version/release in rawhide: 3.36.0-4.fc31
URL: https://calibre-ebook.com/

Please consult the package updates policy before you issue an update to a stable branch: https://fedoraproject.org/wiki/Updates_Policy

More information about the service that created this bug can be found at: https://fedoraproject.org/wiki/Upstream_release_monitoring

Please keep in mind that with any upstream change, there may also be packaging changes that need to be made. Specifically, please remember that it is your responsibility to review the new version to ensure that the licensing is still correct and that no non-free or legally problematic items have been added upstream.

Based on the information from anitya:  https://release-monitoring.org/project/6141/

Latest upstream release: 3.42.0
Current version/release in rawhide: 3.36.0-4.fc31
URL: https://calibre-ebook.com/

Please consult the package updates policy before you issue an update to a stable branch: https://fedoraproject.org/wiki/Updates_Policy

More information about the service that created this bug can be found at: https://fedoraproject.org/wiki/Upstream_release_monitoring

Please keep in mind that with any upstream change, there may also be packaging changes that need to be made. Specifically, please remember that it is your responsibility to review the new version to ensure that the licensing is still correct and that no non-free or legally problematic items have been added upstream.

Based on the information from anitya:  https://release-monitoring.org/project/6141/

(In reply to Kevin Fenzi from comment #5)
> * calibre upstream has 0 plans to port to python3.

This has changed (see [1] for some discussion on the topic). Calibre has experimental support for Python 3 since version 3.41. It's far from being ready for everyday use, but I'm pretty confident that it will be just in time for Fedora 31.

I looked through the list of Calibre's dependencies [2], and it seems like the following packages are currently not available in Fedora:
* python3-css-parser
  -> https://github.com/ebook-utils/css-parser
* python3-mechanize (although Fedora has python2-mechanize)
  -> https://github.com/python-mechanize/mechanize
* python3-soupsieve
  -> https://github.com/facelessuser/soupsieve

I would suggest to add these to the Fedora repositories to be prepared for building new releases of Calibre with Python 3 instead of Python 2.

[1] https://github.com/kovidgoyal/calibre/pull/870
[2] https://github.com/kovidgoyal/calibre/blob/master/bypy/sources.json

(In reply to Marcus from comment #13)
> (In reply to Kevin Fenzi from comment #5)
> > * calibre upstream has 0 plans to port to python3.
> 
> This has changed (see [1] for some discussion on the topic). Calibre has
> experimental support for Python 3 since version 3.41. It's far from being
> ready for everyday use, but I'm pretty confident that it will be just in
> time for Fedora 31.

yeah, I am aware... it changed since I wrote that. :) 
> 
> I looked through the list of Calibre's dependencies [2], and it seems like
> the following packages are currently not available in Fedora:
> * python3-css-parser
>   -> https://github.com/ebook-utils/css-parser
> * python3-mechanize (although Fedora has python2-mechanize)
>   -> https://github.com/python-mechanize/mechanize
> * python3-soupsieve
>   -> https://github.com/facelessuser/soupsieve

Yeah... will have to get those setup. ;( 
> 
> I would suggest to add these to the Fedora repositories to be prepared for
> building new releases of Calibre with Python 3 instead of Python 2.

Yep. I will do so as time permits... if others would like to before then, go for it!

I tried building and packaging version 3.42.0 with Python 3 today. It seems to work, but some of the tests are failing (most likely because of missing or outdated dependencies). Unfortunately I was unable to build directly from the git tree, as Calibre's bootstrapping mechanism still requires Python 2. A lot of bug fixes have been pushed since the last release, some of which should also help to significantly simplify the SPEC file.

I've set up a Copr repository in case someone wants to give it a try. Just make sure to backup your settings and libraries before installing the Python 3 version, it is very buggy.
* https://copr.fedorainfracloud.org/coprs/aimylios/calibre-python3/

The sources of my packaging scripts can be found here:
* https://github.com/aimylios/fedora-calibre-packaging/tree/python3
* https://github.com/aimylios/fedora-python-mechanize-packaging/tree/python3
* https://github.com/aimylios/fedora-calibre-dependencies-packaging

Thanks Marcus! Awesome work... I guess we should see how things look after the next release?

Would you like to submit a PR then, and/or help maintain calibre in fedora?

(In reply to Kevin Fenzi from comment #16)
> I guess we should see how things look after the next release?

Yep. Version 3.43 might be a good candidate for a first rawhide build (in case all prerequisites are met until then).

> Would you like to submit a PR then, and/or help maintain calibre in fedora?

I plan to support the transition to Python 3 as good as I can, but I'm not sure if I'll be motivated enough and have the time to help maintaining Calibre in the long term.
The most work will be to package all of the new dependencies for Fedora. I can start with a PR for python-mechanize, if you think this would help.

In the meantime I made some progress to package Calibre's git head revision, which includes a lot of bug fixes for Python 3. I've updated my scripts and the binaries are again available on copr. The biggest issue I'm currently facing is that the bootstrapping step still relies on Python 2. I've worked this around by doing the bootstrapping locally and then putting the precompiled result into a tarball that is then consumed by the packaging script. Unfortunately this means that the RPM also depends on python2-libs.
There is a lot more that needs to be done (resolve dependency issues, get rid of workarounds for xdg-utils, unbundle mathjax, etc.), but most of the functionality I tested seems to work pretty well already.

Latest upstream release: 3.43.0
Current version/release in rawhide: 3.36.0-4.fc31
URL: https://calibre-ebook.com/

Please consult the package updates policy before you issue an update to a stable branch: https://fedoraproject.org/wiki/Updates_Policy


More information about the service that created this bug can be found at: https://fedoraproject.org/wiki/Upstream_release_monitoring


Please keep in mind that with any upstream change, there may also be packaging changes that need to be made. Specifically, please remember that it is your responsibility to review the new version to ensure that the licensing is still correct and that no non-free or legally problematic items have been added upstream.


Based on the information from anitya: https://release-monitoring.org/project/6141/

> The biggest issue I'm currently facing is that the bootstrapping step still relies on Python 2.

That is not a problem actually, if there aren't too many python2 packages required
for the bootstrapping. Some version of python2 is going to be packages for the
foreseeable future. As long as it is only used during build, depending on it
is not an issue.

> https://github.com/aimylios/fedora-calibre-dependencies-packaging

Marcus, do you plan to submit those as review requests? If not, I would
do that.

I have finished my work on the Calibre packaging scripts. My proposal for the update to version 3.43.0 can be found on GitHub [1]. All build-time and run-time dependencies are now Python 3 only. I plan to submit a PR as soon as all the required packages are available in Rawhide.
I've had to rewrite a lot of things and removed some obsolete workarounds in the process. Please let me know what you think. I might have missed something or accidentally violated the packaging guidelines.

(In reply to Zbigniew Jędrzejewski-Szmek from comment #20)
> > https://github.com/aimylios/fedora-calibre-dependencies-packaging
> 
> Marcus, do you plan to submit those as review requests? If not, I would
> do that.

My PR for python3-mechanize was merged yesterday by Kevin. But I have no plans to submit review requests for the new dependencies. If you want to take care of that, please go ahead. The highest priority would be to package css-parser and soupsieve. Then beautifulsoup4 should be updated to 4.7.x (which also depends on soupsieve).

The dukpy package is not required, and unrardll can not be included because of licensing issues (Calibre "depends" on both of these for a very long time now, and so far their absence didn't cause any issues for Fedora users). I'm not sure what to do about sgmllib(3k) [3]. There's a comment in test_build.py stating "sgmllib is needed for feedparser parsing malformed feeds on python3". It is not directly used by Calibre, so I would say that this should be handled in the python3-feedparser package (which currently doesn't care, because it's an optional dependency [4]).

[1] https://github.com/aimylios/fedora-calibre-packaging/tree/3.43.0
[2] https://github.com/kovidgoyal/calibre/blob/master/src/calibre/test_build.py#L299
[3] https://pypi.org/project/sgmllib3k/
[4] https://github.com/kurtmckee/feedparser/commit/7449030004c4c4b0a4b6ce43eeed5148fc7c95e4

Latest upstream release: 3.44.0
Current version/release in rawhide: 3.36.0-4.fc31
URL: https://calibre-ebook.com/

Please consult the package updates policy before you issue an update to a stable branch: https://fedoraproject.org/wiki/Updates_Policy


More information about the service that created this bug can be found at: https://fedoraproject.org/wiki/Upstream_release_monitoring


Please keep in mind that with any upstream change, there may also be packaging changes that need to be made. Specifically, please remember that it is your responsibility to review the new version to ensure that the licensing is still correct and that no non-free or legally problematic items have been added upstream.


Based on the information from anitya: https://release-monitoring.org/project/6141/

I submitted a PR for unrardll upstream, already merged: https://github.com/kovidgoyal/unrardll/pull/3.

Based on your packaging template, I made a full package for python-unrardll.
Indeed licensing of some dependencies does not allow it in Fedora for now, but I'm
including the link here in case it's useful for someone:
https://in.waw.pl/~zbyszek/fedora/python-unrardll-0.1.3-1.fc31.src.rpm
https://in.waw.pl/~zbyszek/fedora/python-unrardll.spec

Review for python-soupsieve: https://bugzilla.redhat.com/show_bug.cgi?id=1719008.

PR to update beautifoulsoup:
https://src.fedoraproject.org/rpms/python-beautifulsoup4/pull-request/3.

I don't want to be a cattle prod but kernels >5.1.10 cause kindles to disconnect unexpectedly in calibre versions <=3.44.0.

It looks like it is unlikely to be fixed upstream since apparently the Kindle is a unique case, but Kovid came up with a workaround in the latest calibre dev branch.

See calibre discussion here: https://bugs.launchpad.net/calibre/+bug/1834641

And kernel discussion here: https://bugzilla.kernel.org/show_bug.cgi?id=203973

We still need python-soupsieve review to complete, then I think we can update.

python2-apsw was retired in rawhide, which complicates things. I built calibre, python3-apsw, python3-beautifulsoup4,
python3-css-parser, python3-soupsieve using python3.8, and it's seems to run. There's some warnings, but books can be opened ;)
So my plan is to switch to python3 in rawhide when all the deps are built.

Right now it's blocked on soupsieve approval.

This sounds very promising, thank you! I am using Calibre with Python 3 for almost two months now, and all the issues I found have been fixed. Although there is still more porting work to do, all of the basic functionality is there and most users will probably not even notice the difference (unless they rely on some of the more exotic features or have third party plugins installed).

Just for reference, my packaging script for Calibre version 3.46.0 (released today) with Python 3 can be found here:
https://github.com/aimylios/fedora-calibre-packaging/tree/3.46.0

(In reply to bryanhoop from comment #24)
> I don't want to be a cattle prod but kernels >5.1.10 cause kindles to
> disconnect unexpectedly in calibre versions <=3.44.0.

So since as discussed in this bug we are stuck with 3.36.0 for now, I've backported the calibre fix/workaround for the disconnect issue to 3.36.0. I've filed a separate bug, with the patch attached, for tracking the disconnect issue. See bug 1731734.

Latest upstream release: 3.45.0
Current version/release in rawhide: 3.36.0-7.fc31
URL: https://calibre-ebook.com/

Please consult the package updates policy before you issue an update to a stable branch: https://fedoraproject.org/wiki/Updates_Policy


More information about the service that created this bug can be found at: https://fedoraproject.org/wiki/Upstream_release_monitoring


Please keep in mind that with any upstream change, there may also be packaging changes that need to be made. Specifically, please remember that it is your responsibility to review the new version to ensure that the licensing is still correct and that no non-free or legally problematic items have been added upstream.


Based on the information from anitya: https://release-monitoring.org/project/6141/

Latest upstream release: 3.45.1
Current version/release in rawhide: 3.36.0-7.fc31
URL: https://calibre-ebook.com/

Please consult the package updates policy before you issue an update to a stable branch: https://fedoraproject.org/wiki/Updates_Policy


More information about the service that created this bug can be found at: https://fedoraproject.org/wiki/Upstream_release_monitoring


Please keep in mind that with any upstream change, there may also be packaging changes that need to be made. Specifically, please remember that it is your responsibility to review the new version to ensure that the licensing is still correct and that no non-free or legally problematic items have been added upstream.


Based on the information from anitya: https://release-monitoring.org/project/6141/

Latest upstream release: 3.45.2
Current version/release in rawhide: 3.36.0-7.fc31
URL: https://calibre-ebook.com/

Please consult the package updates policy before you issue an update to a stable branch: https://fedoraproject.org/wiki/Updates_Policy


More information about the service that created this bug can be found at: https://fedoraproject.org/wiki/Upstream_release_monitoring


Please keep in mind that with any upstream change, there may also be packaging changes that need to be made. Specifically, please remember that it is your responsibility to review the new version to ensure that the licensing is still correct and that no non-free or legally problematic items have been added upstream.


Based on the information from anitya: https://release-monitoring.org/project/6141/

Latest upstream release: 3.46.0
Current version/release in rawhide: 3.36.0-7.fc31
URL: https://calibre-ebook.com/

Please consult the package updates policy before you issue an update to a stable branch: https://fedoraproject.org/wiki/Updates_Policy


More information about the service that created this bug can be found at: https://fedoraproject.org/wiki/Upstream_release_monitoring


Please keep in mind that with any upstream change, there may also be packaging changes that need to be made. Specifically, please remember that it is your responsibility to review the new version to ensure that the licensing is still correct and that no non-free or legally problematic items have been added upstream.


Based on the information from anitya: https://release-monitoring.org/project/6141/

Built in rawhide and F31.
calibre should now run under python3.
There are some test failures, possibly not related to the python version switch, see #1743645.

Interesting that you've decided to build exclusively as python3. :p I'm one of the people who have been doing a lot of the python3 porting work, and I'm happy to see it getting used but I'm quite positive there are lingering issues. In fact, I fixed one since you built the package: https://github.com/kovidgoyal/calibre/commit/074e3ff829518db1cb8e089d2ff213944ef3c241
And I know there are format conversions that simply haven't been ported yet.

Of course you should choose whether to use python2 or python3 based on what you feel is best for your users, but in the meantime calibre is not quite ready to publicly beta a python3 build, and it *will* guaranteedly break many (most?) third-party plugins. So that is a consideration.

...

Meanwhile, I think you did not notice that mathjax is now handled quite a bit differently since version 3.36.0:
https://github.com/kovidgoyal/calibre/commit/82617d3a74954d4d9cc7d779fa563ad3d212867a

While, before, there were two copies of mathjax (only one of which Fedora debundled), now there is only one: https://github.com/kovidgoyal/calibre/commit/f78a7dad581aa2c29800598e371a42889d6e0eb7

The new version requires a manifest file for the mathjax files in use, containing an etag embedded in index-generated.html, so changing the rm -r && ln -s calls will not work as-is, and mathjax support in calibre-server will be broken, but...

As of my commit https://github.com/kovidgoyal/calibre/commit/82617d3a74954d4d9cc7d779fa563ad3d212867a, you can bootstrap calibre with the system mathjax, with the caveat that you must then rebuild the rapydscript resources (requires python-dukpy installed, and optionally rapydscript-ng to greatly speed up the time spent transpiling rapydscript *.pyj to javascript).

...

Likewise, on newer versions of calibre most xdg-utils hacks can now be removed due to https://github.com/kovidgoyal/calibre/commit/2a63948440fe2d60a5573b829f27000d5c0389e2, and calibre-uninstall is no longer created. For context, see https://github.com/kovidgoyal/calibre/pull/988

...

Feel free to use my Arch Linux build recipe for inspiration: https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/calibre#n59
Especially take note of my use of --staging-root=%{buildroot}%{_prefix} --prefix=%{_prefix}

And for rapydscript-ng, see https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/rapydscript-ng

P.S. It's probably better in general to use rm -r instead of rm -rf, because if the thing you are trying to delete is no longer present upstream (mathjax files) then your whole build will fail and therefore warn you that something has changed and you'd better figure out what the new version is doing differently.

P.P.S. calibre-mount-helper is stupendously old and no longer needed at all.

Hi, thanks for the all the hints. Cross-distro collaboration at its best ;)

> Interesting that you've decided to build exclusively as python3.

My plan was to provide a python3 preview for a while in parallel with the python2 version, but
then a python2 dep got retired and it just seemed easier to accelerate the process.
This is for Fedora 31+ only, so there's still ~2 months before it gets widespread use.

As for the other stuff: ack, looks like there's lot of stuff to fix.

> It's probably better in general to use rm -r instead of rm -rf

Agreed. It's unfortunate that rm conflates the force mode with suppressing errors.
People often want the first without the second. (Though in this particular case,
neither is needed, so -r is appropriate.)

Cross-distro collaboration is great, and it's nice to see other distro packagers contributing upstream to calibre. :) It's not actually something I've seen a lot of, aside for my various efforts to polish up the build and distro integration bits (which I want people to use!) and a handful of qt5 compat fixes from my Arch comaintainer.

I've been providing a parallel-installable calibre-python3-git package myself via the Arch Linux AUR, and yeah, it's an adventure. I hope to be able to beta the python3 version in Arch Linux [community] by 1/1/2020 though!