Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1571650 - pip: Update operation's result ignored
Summary: pip: Update operation's result ignored
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python-pip
Version: 28
Hardware: x86_64
OS: Linux
unspecified
low
Target Milestone: ---
Assignee: Tomas Orsava
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-25 09:41 UTC by ricky.tigg
Modified: 2018-05-15 20:00 UTC (History)
13 users (show)

Fixed In Version: python-pip-9.0.3-2.fc28 python-pip-9.0.3-2.fc27 python-pip-9.0.3-2.fc26
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-05-09 21:25:24 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description ricky.tigg 2018-04-25 09:41:09 UTC
Description of problem: Update operation's result is ignored.

Version-Release number of components: python2-pip.noarch 9.0.3-1.fc28 @fedora,         
python3-pip.noarch 9.0.3-1.fc28 @fedora.

Steps to Reproduce:
1. Assumed that some Python packages are non-updated; Verify as follow:
$ pip list --outdated
...
pip (9.0.3) - Latest: 10.0.1 [wheel]
...

2. $ pip install --user -U pip
Collecting pip
  Downloading https://files.pythonhosted.org/packages/0f/74/ecd13431bcc456ed390b44c8a6e917c1820365cbebcb6a8974d1cd045ab4/pip-10.0.1-py2.py3-none-any.whl (1.3MB)
...
Successfully installed pip-10.0.1
You are using pip version 9.0.3, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

3. $ pip install --user --upgrade pip
Traceback (most recent call last):
  File "/usr/bin/pip", line 7, in <module>
    from pip import main
ImportError: cannot import name main

Actual results: As noticeable in step 2., output does assume pip version is still in state non-updated.

Expected results: Pip to be aware of its own operations.

Additional information: –After update took place–
$ dnf list installed|grep -i pip
...
python2-pip.noarch                     9.0.3-1.fc28             @fedora         
python3-pip.noarch                     9.0.3-1.fc28             @fedora

Comment 1 ricky.tigg 2018-04-25 10:02:47 UTC
$ sudo pip install --upgrade pip
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip install --user` instead.
...
      Successfully uninstalled pip-9.0.3
Successfully installed pip-10.0.1

Comment 2 Petr Viktorin 2018-04-25 12:43:32 UTC
As a workaround, you can use the pip module instead of calling "pip" directly:

    $ python -m pip list --outdated

Comment 3 Tomas Orsava 2018-04-30 13:13:23 UTC
This bug is caused by the fact that you don't have ~/.local/bin in your $PATH. Therefore you install new pip sources into your home directory, but you're still using the old access point at /usr/bin/pip that does not work with the new sources.

We can solve this by patching /usr/bin/pip to also look for the new sources, which would be an ugly downstream hack that fixes only this case.

The better solution would be to add ~/.local/bin to the default $PATH. I don't understand why it's not there already. Does anyone have an idea as to why?

Comment 4 Tomas Orsava 2018-04-30 13:48:31 UTC
Hm, so I've verified that ~/.local/bin indeed is in the default $PATH.

Ricky, I believe you have modified your $PATH and now it doesn't contain ~/.local/bin. If you want modules that you pip-installed with --user to work, you need this location in your $PATH.

Comment 5 ricky.tigg 2018-04-30 14:44:16 UTC
Only changes have been made here:
$ gedit &>/dev/null ~/.bashrc

# .bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
	. /etc/bashrc
fi

# User specific aliases and functions

export EDITOR=/usr/bin/gedit

# Aliasten käyttöönotto
alias aliasko='source ~/.bashrc'
... more aliases.

Comment 6 Tomas Orsava 2018-04-30 15:29:12 UTC
Aha! Apologies, it turns out /usr/bin takes precedence over ~/.local/bin in the default Fedora PATH and this is causing the issue.

$ echo $PATH       
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/tom/.local/bin:/home/tom/bin

Comment 7 Miro Hrončok 2018-04-30 17:54:31 UTC
Where is the PATH actually coming from?

$ mock -r fedora-rawhide-x86_64 --shell
<mock-chroot> sh-4.4# echo $PATH
/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin
<mock-chroot> sh-4.4# adduser foo
<mock-chroot> sh-4.4# su foo
<mock-chroot> bash-4.4$ echo $PATH
/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin

Comment 8 ricky.tigg 2018-04-30 19:35:10 UTC
$ echo $PATH
/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/yk/.local/bin:/home/yk/bin

Comment 9 Tomas Orsava 2018-05-01 13:24:09 UTC
(In reply to Miro Hrončok from comment #7)
> Where is the PATH actually coming from?
> 
> $ mock -r fedora-rawhide-x86_64 --shell
> <mock-chroot> sh-4.4# echo $PATH
> /usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin
> <mock-chroot> sh-4.4# adduser foo
> <mock-chroot> sh-4.4# su foo
> <mock-chroot> bash-4.4$ echo $PATH
> /usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin

You need to start the user's shell as a login shell (su - foo) to get the full $PATH:

sh-4.4# adduser foo           
sh-4.4# su - foo
[foo@78595df164a54336b3cd6d4c10e5e017 ~]$ echo $PATH
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/foo/.local/bin:/home/foo/bin

However, I spent some time looking through ~/.bashrc, /etc/bashrc, /etc/profile, and /etc/profile.d/ and I was not able to find out where ~/.local/bin is added to $PATH. There is a function for adding PATH elements called `pathmunge`, but nowhere have I found the ~/.local/bin being added.

Comment 10 Tomas Orsava 2018-05-01 13:38:11 UTC
[Facepalm] Found it! In ~/.bash_profile:

    # User specific environment and startup programs

    PATH=$PATH:$HOME/.local/bin:$HOME/bin

By changing it to:

    PATH=$HOME/.local/bin:$HOME/bin:$PATH

We get the desired ordering:

    $ echo $PATH
    /home/foo/.local/bin:/home/foo/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin


Now the dilemma, is there a reason we shouldn't do this? Should system binaries always have preference over user installed ones?
We can rule out malicious usage, because if someone has access to ~/.local/bin, they can also permanently rewrite $PATH as well. Besides that, if the user wants to install a newer version locally, I say they should be allowed.

Comment 11 Tomas Orsava 2018-05-01 14:17:20 UTC
~/.bash_profile gets created as a copy of /etc/skel/.bash_profile that is owned by the bash package.

Comment 12 Miro Hrončok 2018-05-02 11:12:13 UTC
Let's ask the bash maintainers if we can change the order. It would solve this issue. At least for new installs / users.

Also I'd except that any layer closer to **me** (the user) should have precedence, i.e. IMHO:

1. /usr/local/bin should beat /usr/bin
2. ~/.local/bin should beat /usr/local/bin

Comment 13 Siteshwar Vashisht 2018-05-02 11:59:48 UTC
For the reference, adding ~/.local/bin was previously discussed in this thread[1] and it was added in $PATH based on request by one users[2]. I think we should discuss the change to prepend this path in fedora-devel first and change it only if there is a concrete reason to do so.

[1] https://lists.fedoraproject.org/pipermail/devel/2011-July/154896.html
[2] https://bugzilla.redhat.com/show_bug.cgi?id=699812

Comment 14 Igor Gnatenko 2018-05-02 12:01:15 UTC
> Now the dilemma, is there a reason we shouldn't do this? Should system binaries always have preference over user installed ones?

No.

Comment 15 Miro Hrončok 2018-05-02 12:10:34 UTC
> I think we should discuss the change to prepend this path in fedora-devel first

If we must :)

> change it only if there is a concrete reason to do so

This bug is a concrete reason to do so.

Comment 16 Tomas Orsava 2018-05-02 12:42:37 UTC
I'll send an email to fedora-devel.

Comment 17 Siteshwar Vashisht 2018-05-02 13:19:03 UTC
Miro,

Neither OpenSuse nor Ubuntu add ~/.local/bin in $PATH. It was added as a response to request from one user and should not have been added without some discussion. If a user is installing something in non-standard paths, they should be responsible for setting $PATH to find correct commands.

Comment 19 Amitosh Swain Mahapatra 2018-05-02 13:45:08 UTC
Siteshwar,

>If a user is installing something in non-standard paths, they should be responsible for setting $PATH to find correct commands.

Nowadays, a lot of tools like pip, bundler, and npm assume ~/.local to be the private copy of the LSB FSH standard directories.

It was started by Gnome, Though it's not the standard, the adoption is slowly starting to increase.

Comment 20 Miro Hrončok 2018-05-02 13:52:40 UTC
(In reply to Siteshwar Vashisht from comment #17)
> If a user is installing something in non-standard paths,
> they should be responsible for setting $PATH to find correct commands.

~/.local/bin is quite standard and I'd say XDG complaint. That's what Lennart Poettering said in https://bugzilla.redhat.com/show_bug.cgi?id=699812#c2 and I agree with him.

Comment 21 Siteshwar Vashisht 2018-05-02 13:56:11 UTC
> ~/.local/bin is quite standard and I'd say XDG complaint. That's what Lennart Poettering said in https://bugzilla.redhat.com/show_bug.cgi?id=699812#c2 and I agree with him.

Lennart later clarified his comment[1].

[1] https://lists.fedoraproject.org/pipermail/devel/2011-July/154902.html

Comment 22 Miro Hrončok 2018-05-02 14:05:58 UTC
Yet I still consider ~/.local/bin "standard practice".

Let's continue on devel ML.

Comment 23 Miro Hrončok 2018-05-04 09:06:46 UTC
*** Bug 1569488 has been marked as a duplicate of this bug. ***

Comment 24 Fedora Update System 2018-05-05 15:20:19 UTC
python-pip-9.0.3-2.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-7146cb4e7f

Comment 25 Fedora Update System 2018-05-06 10:09:13 UTC
python-pip-9.0.3-2.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-7146cb4e7f

Comment 26 Fedora Update System 2018-05-07 10:12:10 UTC
python-pip-9.0.3-2.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2018-b795b92cc0

Comment 27 Fedora Update System 2018-05-07 10:24:21 UTC
python-pip-9.0.3-2.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2018-fcbef2ab8c

Comment 28 Fedora Update System 2018-05-07 14:33:53 UTC
python-pip-9.0.3-2.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-b795b92cc0

Comment 29 Fedora Update System 2018-05-08 00:03:24 UTC
python-pip-9.0.3-2.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-fcbef2ab8c

Comment 30 Fedora Update System 2018-05-09 21:25:24 UTC
python-pip-9.0.3-2.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.

Comment 31 Fedora Update System 2018-05-13 20:17:39 UTC
python-pip-9.0.3-2.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.

Comment 32 Fedora Update System 2018-05-15 20:00:20 UTC
python-pip-9.0.3-2.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.