Python's installation tool, pip, has some problems on Debian.
The obvious way to use pip is as root:
sudo pip install packagename.
If you hang out in Python groups at all, you'll quickly find that this
is strongly frowned upon. It can lead to your pip-installed packages
intermingling with the ones installed by Debian's apt-get,
possibly causing problems during apt system updates.
The second most obvious way, as you'll see if you read pip's man page,
pip --user install packagename.
This installs the package with only user permissions, not root,
under a directory called ~/.local. Python automatically checks
.local as part of its PYTHONPATH, and you can add ~/.local/bin to
your PATH, so this makes everything transparent.
Or so I thought until recently, when I discovered that
pip install --user ignores system-installed packages
when it's calculating its dependencies, so you could end up with a
bunch of incompatible versions of packages installed. Plus it takes
forever to re-download and re-install dependencies you already had.
Pip has a clear page describing how pip --user is supposed to work, and that isn't what it's doing. So I filed pip bug 4222; but since pip has 687 open bugs filed against it, I'm not terrifically hopeful of that getting fixed any time soon. So I needed a workaround.
Use virtualenv instead of --user
Fortunately, it turned out that pip install works correctly in a virtualenv if you include the --system-site-packages option. I had thought virtualenvs were for testing, but quite a few people on #python said they used virtualenvs all the time, as part of their normal runtime environments. (Maybe due to pip's deficiencies?) I had heard people speak deprecatingly of --user in favor of virtualenvs but was never clear why; maybe this is why.
So, what I needed was to set up a virtualenv that I can keep around all the time and use by default every time I log in. I called it ~/.pythonenv when I created it:
virtualenv --system-site-packages $HOME/.pythonenv
Normally, the next thing you do after creating a virtualenv is to source a script called bin/activate inside the venv. That sets up your PATH, PYTHONPATH and a bunch of other variables so the venv will be used in all the right ways. But activate also changes your prompt, which I didn't want in my normal runtime environment. So I stuck this in my .zlogin file:
VIRTUAL_ENV_DISABLE_PROMPT=1 source $HOME/.pythonenv/bin/activate
Now I'll activate the venv once, when I log in (and once in every xterm
window since I set XTerm*loginShell: true in my .Xdefaults.
I see my normal prompt, I can use the normal Debian-installed Python
packages, and I can install additional PyPI packages with
pip install packagename (no --user, no sudo).
[ 11:37 Jan 08, 2017 More | permalink to this entry | comments ]