Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1816696

Summary: mirrorlists stopped working
Product: [Fedora] Fedora Reporter: Ralf Corsepius <rc040203>
Component: mockAssignee: Pavel Raiskup <praiskup>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 31CC: copr-team, jdisnard, jkeating, mebrown, msuchy, philip.wyett, praiskup, williams
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: mock-2.2-1.fc32 mock-2.2-1.fc31 mock-2.2-1.fc30 mock-2.3-1.fc32 mock-2.3-1.fc31 mock-2.3-1.el7 mock-2.3-1.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-24 03:28:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
brief log
none
verbose log none

Description Ralf Corsepius 2020-03-24 14:45:21 UTC
Description of problem:

In /etc/mock/*.cfgs, the mirrorslist option seems to have stopped working 

Version-Release number of selected component (if applicable):
mock-2.1-1.fc31.noarch
dnf-4.2.18-1.fc31.noarch

How reproducible:
Always


Actual results:

# mock -r fedora-31-x86_64-packman --init
...
Errors during downloading metadata for repository 'packman':
  - Curl error (37): Couldn't read a file:// file for file:///usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist [Couldn't open file /usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist]
Error: Failed to download metadata for repo 'packman': Cannot prepare internal mirrorlist: Curl error (37): Couldn't read a file:// file for file:///usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist [Couldn't open file /usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist]


The corresponding /etc/mock/fedora-31-x86_64-packman.cfg  contains this:
..
[packman]
name=packman
mirrorlist=file:///usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist
...

/usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist exists and is readable.

Expected results:
Function.

Additional info:
This is a regression, severely reducing mock's usablity.

Comment 1 Pavel Raiskup 2020-03-24 14:53:49 UTC
Thanks for the report.

Please work-around this for now by disabling bootstrap, `--no-bootstrap-chroot` or
`config_opts['use_bootstrap'] = False`.

Comment 2 Ralf Corsepius 2020-03-24 16:23:02 UTC
Thanks for the workaround. Seems to work for me.

Comment 4 Fedora Update System 2020-04-02 07:48:12 UTC
FEDORA-2020-fba9845e22 has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-fba9845e22

Comment 5 Fedora Update System 2020-04-02 07:49:22 UTC
FEDORA-2020-6b7c342fb4 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-6b7c342fb4

Comment 6 Fedora Update System 2020-04-02 07:49:49 UTC
FEDORA-2020-85df0014c1 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2020-85df0014c1

Comment 7 Fedora Update System 2020-04-03 19:56:11 UTC
FEDORA-2020-fba9845e22 has been pushed to the Fedora 32 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-fba9845e22`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-fba9845e22

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 8 Fedora Update System 2020-04-03 20:35:11 UTC
FEDORA-2020-85df0014c1 has been pushed to the Fedora 30 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-85df0014c1`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-85df0014c1

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 9 Fedora Update System 2020-04-03 20:49:40 UTC
FEDORA-2020-6b7c342fb4 has been pushed to the Fedora 31 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-6b7c342fb4`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-6b7c342fb4

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 10 Fedora Update System 2020-04-07 05:05:09 UTC
FEDORA-2020-fba9845e22 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 11 Fedora Update System 2020-04-07 13:16:05 UTC
FEDORA-2020-fba9845e22 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 12 Fedora Update System 2020-04-20 22:14:52 UTC
FEDORA-2020-6b7c342fb4 has been pushed to the Fedora 31 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 13 Fedora Update System 2020-04-20 22:32:04 UTC
FEDORA-2020-85df0014c1 has been pushed to the Fedora 30 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 14 Ralf Corsepius 2020-04-23 04:11:04 UTC
Reopening: The symptoms have changed, but this bug persists:

# mock -r fedora-31-x86_64-packman --init
...
FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/mock/fedora-31-x86_64-bootstrap/root/usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist'

Apparently, mock mixes up host and chroot'ed environment and bogusly tries to read the mirrorlist file from inside the chroot.

Comment 15 Pavel Raiskup 2020-04-23 06:55:32 UTC
> Apparently, mock mixes up host and chroot'ed environment and bogusly tries to
> read the mirrorlist file from inside the chroot.

No, the mirror list shoudl be copied into the _bootstrap_ chroot, because mock
executes dnf/yum from withing the _bootstrap_ (see carefully the path).

IOW, when you drop the '...bootstrap/root' part, is this the right location?

    /usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist

If yes, can you help us to diagnose why it was not available in the bootstrap
chroot?  I probably need full traceback ... also full log with --trace --verbose
could be attached (compressed) here.

Comment 16 Ralf Corsepius 2020-04-23 09:52:14 UTC
(In reply to Pavel Raiskup from comment #15)
> > Apparently, mock mixes up host and chroot'ed environment and bogusly tries to
> > read the mirrorlist file from inside the chroot.
> 
> No, the mirror list shoudl be copied into the _bootstrap_ chroot, because
> mock
> executes dnf/yum from withing the _bootstrap_ (see carefully the path).
> 
> IOW, when you drop the '...bootstrap/root' part, is this the right location?
> 
>     /usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist
Yes, this is the location, this file is installed on the host.

> If yes, can you help us to diagnose why it was not available in the bootstrap
> chroot?
I'll try.

>  I probably need full traceback ... also full log with --trace
> --verbose
> could be attached (compressed) here.
I'll do so, soonish.

Comment 17 Ralf Corsepius 2020-04-23 09:54:29 UTC
Created attachment 1681050 [details]
brief log

Log created this way:
# mock -r fedora-31-x86_64-packman --scrub=all
# mock -r fedora-31-x86_64-packman --init > mock1.log 2>&1

Comment 18 Ralf Corsepius 2020-04-23 09:57:02 UTC
Created attachment 1681051 [details]
verbose log

Log created this way:
# mock -r fedora-31-x86_64-packman --scrub=all
# mock -r fedora-31-x86_64-packman --trace --verbose --init > mock2.log 2>&1

Comment 19 Pavel Raiskup 2020-04-23 11:10:52 UTC
Thanks, can you please try the following patch?

```
diff --git a/mock/py/mockbuild/mounts.py b/mock/py/mockbuild/mounts.py
index 8290aa02..f5515cbc 100644
--- a/mock/py/mockbuild/mounts.py
+++ b/mock/py/mockbuild/mounts.py
@@ -99,6 +99,8 @@ class BindMountPoint(MountPoint):
             if os.path.isdir(self.srcpath):
                 util.mkdirIfAbsent(self.bindpath)
             elif not os.path.exists(self.bindpath):
+                normbindpath = os.path.normpath(self.bindpath)
+                util.mkdirIfAbsent(os.path.dirname(normbindpath))
                 util.touch(self.bindpath)
             cmd = ['/bin/mount', '-n']
             if self.recursive:

```

I think we can afford to do this in general, because if we bind-mount
directories we create the whole directory path.  It sounds sane to do
the same for non-directory bind mounts.

Comment 20 Ralf Corsepius 2020-04-23 13:00:38 UTC
Thanks for the patch. 

At first glance, this appears to work.

Comment 21 Fedora Update System 2020-05-22 12:14:18 UTC
FEDORA-2020-f109b06dad has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-f109b06dad

Comment 22 Fedora Update System 2020-05-22 12:14:56 UTC
FEDORA-2020-16720483b5 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-16720483b5

Comment 23 Fedora Update System 2020-05-22 12:15:27 UTC
FEDORA-2020-ec92166d87 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2020-ec92166d87

Comment 24 Fedora Update System 2020-05-22 12:16:01 UTC
FEDORA-EPEL-2020-9a5f274330 has been submitted as an update to Fedora EPEL 8. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-9a5f274330

Comment 25 Fedora Update System 2020-05-22 12:16:34 UTC
FEDORA-EPEL-2020-9662bcb8a3 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-9662bcb8a3

Comment 26 Fedora Update System 2020-05-23 04:05:44 UTC
FEDORA-2020-16720483b5 has been pushed to the Fedora 31 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-16720483b5`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-16720483b5

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 27 Fedora Update System 2020-05-23 04:26:22 UTC
FEDORA-EPEL-2020-9662bcb8a3 has been pushed to the Fedora EPEL 7 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-9662bcb8a3

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 28 Fedora Update System 2020-05-23 04:32:32 UTC
FEDORA-2020-ec92166d87 has been pushed to the Fedora 30 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-ec92166d87`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-ec92166d87

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 29 Fedora Update System 2020-05-23 04:36:47 UTC
FEDORA-2020-f109b06dad has been pushed to the Fedora 32 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-f109b06dad`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-f109b06dad

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 30 Fedora Update System 2020-05-23 04:53:57 UTC
FEDORA-EPEL-2020-9a5f274330 has been pushed to the Fedora EPEL 8 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-9a5f274330

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 31 Fedora Update System 2020-05-24 03:28:38 UTC
FEDORA-2020-f109b06dad has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 32 Fedora Update System 2020-05-31 03:57:06 UTC
FEDORA-2020-16720483b5 has been pushed to the Fedora 31 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 33 Fedora Update System 2020-06-07 18:43:31 UTC
FEDORA-EPEL-2020-9662bcb8a3 has been pushed to the Fedora EPEL 7 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 34 Fedora Update System 2020-06-08 00:44:59 UTC
FEDORA-EPEL-2020-9a5f274330 has been pushed to the Fedora EPEL 8 stable repository.
If problem still persists, please make note of it in this bug report.