Not able to use pip-accel both globally and inside a virtual environment

[reubano]

I've installed pip-accel globally, but when I try to use it inside a virtual environment I get this error:

Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip_accel/cli.py", line 56, in main
    accelerator = PipAccelerator(config)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip_accel/__init__.py", line 109, in __init__
    self.validate_environment()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip_accel/__init__.py", line 141, in validate_environment
    """, environment=environment, prefix=sys.prefix)
EnvironmentMismatchError: You are trying to install packages in environment #1 which is different from environment #2 where pip-accel is installed! Please install pip-accel under environment #1 to install packages there.

This is caused by validate_environment, which doesn't make much sense. Since pip-accel uses pip under the hood, why should I have to install another version of pip-accel inside every virtual environment? The kicker is that if pip-accel is already installed globally, pip won't even let you install it inside a virtual environment. It's either one or the other.

[xolox]

This is caused by validate_environment, which doesn't make much sense.

Clearly it was a conscious choice to add that code, so just saying "it doesn't make much sense" is not the best way to start this conversation :-).

Since pip-accel uses pip under the hood, why should I have to install another version of pip-accel inside every virtual environment?

When you create a virtual environment the virtualenv tool will automatically install a copy of pip inside the virtual environment. Unfortunately pip-accel doesn't get the same treatment, because virtualenv doesn't know about pip-accel.

In older versions of pip-accel this used to bite unsuspecting users which is why the check was added; mixing global installations and virtual environments can cause a wide variety of issues, some of them very obscure and non-obvious, so shielding users from this mistake seemed like the only reasonable option.

The kicker is that if pip-accel is already installed globally, pip won't even let you install it inside a virtual environment. It's either one or the other.

This is not an absolute truth, it's a bit more subtle (IIUC):

• Old versions of virtualenv will (by default) create virtual environments that can also import packages that are installed system wide. This is the --system-site-packages command line option.
• New versions of virtualenv will (by default) create virtual environments that can't import packages that are installed system wide. This is the --no-site-packages command line option.

I assume that with the new behavior you should not be seeing this problem at all; pip should be able to install pip-accel just fine because it will realize that the pip-accel installed system wide is not visible inside the virtual environment.

[reubano]

Clearly it was a conscious choice to add that code, so just saying "it doesn't make much sense" is not the best way to start this conversation :-).

Sorry about that! Should have added ".. to me."

New versions of virtualenv will (by default) create virtual environments that can't import packages that are installed system wide. This is the --no-site-packages command line option.

Ok, it turns out I am using --no-site-packages but the virtualenv can still see them due to how my PYTHONPATH is setup. So I guess my options are to reset PYTHONPATH or uninstall pip-accel globally.

[xolox]

I have more experience in (ab)using $PYTHONPATH than I care to share here ;-) but on the upside I have run into this exact same issue before, so I can give some hints:

1. Stop customizing $PYTHONPATH because it will bite you again :-). I know this isn't always an option (it wasn't in my use case), but I thought I should mention it anyway!

2. Clear the $PYTHONPATH before invoking pip. One simple way to do so is to run pip as follows:

   $ PYTHONPATH= pip install pip-accel
   

   This will clear the environment variable for this single command, but leave it untouched for the remainder of your shell session.

3. Change your pip installation command as follows:

   $ pip install --ignore-installed pip-accel
   

I hope one of these suggestions helps, please let me know! Maybe I can turn this into a bit of documentation (which is really more relevant to pip than to pip-accel, but given that pip-accel wraps pip maybe I should document this anyway).

[reubano]

Great, thanks for the additional options. I went with the first, since I don't need the path customization any longer. It's working now! However, I think it would be great if there was a way to use pip-accel globally :).