Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1221165
Summary: | [RFC] What is the future of par2cmdline? | ||
---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Tadej Janež <tadej.j> |
Component: | par2cmdline | Assignee: | Erik van Pienbroek <erik-fedora> |
Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
Severity: | unspecified | Docs Contact: | |
Priority: | unspecified | ||
Version: | 23 | CC: | erik-fedora, mesat, reikred |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | par2cmdline-0.6.14-1.fc23 | Doc Type: | Bug Fix |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2015-11-30 02:20:47 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Tadej Janež
2015-05-13 12:05:04 UTC
FWIW, Debian uses the BlackIkeEagle (Parchive) fork: https://tracker.debian.org/pkg/par2cmdline I agree that the situation with par2 is pretty messy these days, but I don't know what the future is of par2cmdline (as I'm only the downstream Fedora packager).. I was hoping that the BlackIkeEagle fork would have threading support by now, but apparently this takes longer than expected. In order to decide what path to go more testing is needed of the various possible solutions. I'm tempted to start off with the 0.6.12 release (https://github.com/Parchive/par2cmdline/releases/tag/v0.6.12) and backport the OpenMP pieces onto this: https://github.com/Parchive/par2cmdline/compare/3038e09fcee00f4dd628bcb040ebd43741ebadfb...jkansanen:master If this works reliable we can put it up for testing in Fedora to give it more attention. By doing this we can help upstream with testing these changes and increasing the chance that these changes will be accepted upstream more quickly. I've prepared a test build containing OpenMP support @ http://koji.fedoraproject.org/koji/taskinfo?taskID=9755424 Could you test if this works for you? I've done some initial testing myself and things look good on it. It is using all my available CPU cores during repair. This package was also able to repair some files where the old par2-tbb failed. I've also done some time tests (to verify and repair some broken files) with the various versions and here are the results: par2-tbb: real 2m13.139s user 14m36.899s sys 0m6.576s par2cmdline 0.6.12: real 5m29.718s user 5m27.731s sys 0m1.223s par2cmdline 0.6.12 with OpenMP support: real 3m34.538s user 23m47.680s sys 0m2.383s So with OpenMP changes the performance improves nicely but it's not on par yet with par2-tbb. But on the other hand the new par2cmdline is able to repair files which par2-tbb wasn't able to so that's a plus. (In reply to Erik van Pienbroek from comment #2) > I agree that the situation with par2 is pretty messy these days, but I don't > know what the future is of par2cmdline (as I'm only the downstream Fedora > packager). Sure, I understand. > In order to decide what path to go more testing is needed of the various > possible solutions. I'm tempted to start off with the 0.6.12 release > (https://github.com/Parchive/par2cmdline/releases/tag/v0.6.12) and backport > the OpenMP pieces onto this: > https://github.com/Parchive/par2cmdline/compare/ > 3038e09fcee00f4dd628bcb040ebd43741ebadfb...jkansanen:master Seems like a good plan. > If this works reliable we can put it up for testing in Fedora to give it > more attention. By doing this we can help upstream with testing these > changes and increasing the chance that these changes will be accepted > upstream more quickly. Agreed. (In reply to Erik van Pienbroek from comment #3) > I've prepared a test build containing OpenMP support @ > http://koji.fedoraproject.org/koji/taskinfo?taskID=9755424 > Could you test if this works for you? Thanks! I'll test this and report back. > I've done some initial testing myself and things look good on it. It is > using all my available CPU cores during repair. This package was also able > to repair some files where the old par2-tbb failed. > > I've also done some time tests (to verify and repair some broken files) with > the various versions and here are the results: > par2-tbb: > real 2m13.139s > user 14m36.899s > sys 0m6.576s > > par2cmdline 0.6.12: > real 5m29.718s > user 5m27.731s > sys 0m1.223s > > par2cmdline 0.6.12 with OpenMP support: > real 3m34.538s > user 23m47.680s > sys 0m2.383s > > So with OpenMP changes the performance improves nicely but it's not on par > yet with par2-tbb. But on the other hand the new par2cmdline is able to > repair files which par2-tbb wasn't able to so that's a plus. Great. I would go for ability to repair first and performance later. BTW, why are 'user' times so different for the different runs above? (In reply to Tadej Janež from comment #4) > BTW, why are 'user' times so different for the different runs above? My guess would be that the par2-tbb and openmp versions both use multiple CPU cores but I'm not sure about that I think it will be nice to see a consolidation of forks to fix and improve all projects. This will start to clean up the mess. I am following the parchive-devel mail list on this. This bug appears to have been reported against 'rawhide' during the Fedora 23 development cycle. Changing version to '23'. (As we did not run this process for some time, it could affect also pre-Fedora 23 development cycle bugs. We are very sorry. It will help us with cleanup during Fedora 23 End Of Life. Thank you.) More information and reason for this action is here: https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora23 par2cmdline-0.6.14-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-d36d127ea7 par2cmdline-0.6.14-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with $ su -c 'dnf --enablerepo=updates-testing update par2cmdline' You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-d36d127ea7 I had just downloaded the Github version to repair an archive that the F22 version of par2 wouldn't fix. Yes it is slower not being multithreaded but it is better since it actually fixes the archive instead of leaving a major mess. Thank you for moving ahead. nntpgrab-0.7.2-12.fc23 par2cmdline-0.6.14-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-d36d127ea7 nntpgrab-0.7.2-12.fc23, par2cmdline-0.6.14-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with $ su -c 'dnf --enablerepo=updates-testing update nntpgrab par2cmdline' You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-d36d127ea7 nntpgrab-0.7.2-12.fc23, par2cmdline-0.6.14-1.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report. Okay, so I now have par2cmdline-0.6.14-1.fc23, which per the above I have deduced (hopefully correctly) should be safe to use with "bup fsck -g". But there is no new version of bup in fc23 that allows par2 to be run, and, AFAICT, no override flag. What should I do? Download the latest bup src and compile it myself? Or is there an fc 24 version where par2 has been enabled, that might dnf install in fc23? Hi, Reik! (In reply to Reik Red from comment #14) > Okay, so I now have par2cmdline-0.6.14-1.fc23, which per the above I have > deduced (hopefully correctly) should be safe to use with "bup fsck -g". But > there is no new version of bup in fc23 that allows par2 to be run, and, > AFAICT, no override flag. > > What should I do? Download the latest bup src and compile it myself? Or is > there an fc 24 version where par2 has been enabled, that might dnf install > in fc23? I'm the maintainer of bup in Fedora. I haven't yet updated bup to version 0.28 (tracking bug is #1323522). After the update, I'll test par2 support and if successful, reenable par2cmdline support in bup afterwards. (In reply to Tadej Janež from comment #15) I decided to recompile bup and install to .usr/local to get a solution that works right away. It was pretty simple given that bup is mostly python anyway. Seems to work. People who are in a a pinch getting "bup fsck -g" to work in Fedora 23 or 24 may want to try this. % /bin/bup --version unknown-d612d9a % /usr/local/bin/bup --version 0.28.1-23-g461b5b4 How to compile, assuming you already have the necessary pytho stuff, otherwise refer to https://github.com/bup/bup mkdir ~/src cd ~/src git clone https://github.com/bup/bup cd bup ./configure |& tee log.configure make | & tee log.make make install -n # install goes to /usr/local/ sudo make install | & tee log.make.install make test |& tee log.make.test make test has 2 errors related to xattr.set, but that appears to be just some selinux weirdness. (In reply to Tadej Janež from comment #15) > > I'm the maintainer of bup in Fedora. I haven't yet updated bup to version > 0.28 (tracking bug is #1323522). After the update, I'll test par2 support > and if successful, reenable par2cmdline support in bup afterwards. For the record, bup 0.28.1 with par2 support is available on Fedora 24+: https://bodhi.fedoraproject.org/updates/FEDORA-2016-1bf266af1e https://bodhi.fedoraproject.org/updates/FEDORA-2016-addd1684e1 |