invert site-packages behavior

[mcdonc]

Virtualenv is used, primarily, to create an isolated packaging environment. The current default use-site-packages behavior is suboptimal for this purpose. New users (and some experienced users) invariably fail to pass the flag even when instructions to do so are in blinking red text in a 72 point font, and they wind up in a place where they have conflicting global and virtualenv packages. This is, by far, the most common support issue I've experienced in regards to instructing other people to use virtualev.

It would make a lot more sense if virtualenv defaulted to --no-site-packages behavior. This patch does that, and adds a --use-site-packages flag to get the old behavior back.

[carljm]

Hi @mcdonc - I'm certainly willing to consider this; it's been previously requested. It would be helpful to know what sort of "conflicting package" problems users tend to have, in your experience? Generally packages in the virtualenv should simply override global ones, and there shouldn't be a problem. So I'm wondering if your experiences shed light on other bugs that might be fixable, regardless of whether we invert the default as you suggest.

[carljm]

I've asked on the mailing list about preferences on this: http://groups.google.com/group/python-virtualenv/browse_frm/thread/57e11efac3743149

[mcdonc]

One concrete issue to start with is on MacOSX.

"zope.interface" is present on some path in the system interpreter on most versions of MacOS. It's an older version of this package, apparently used by Apple system packages. When someone installs virtualenv into the system interpreter on such a system, and subsequently creates a virtual environment without "--no-site-packages", installation of Pyramid using the virtualenv bin/easy_install or bin/pip fails. I can't, at the moment, describe exactly how it fails because I don't have a Mac on me. But the general symptom "cant install Pyramid" is reported to us two or threee times a week; when we ask the reporter to use --no-site-packages, it works for them.

That issue aside, I think the supplied patch is a better default. AFAICT, virtualenv is designed to provide an isolated environment, and with the current set of defaults, it just doesn't. This puts folks in a position where they think they're troubleshooting an isolated system, but they aren't.

[mmerickel]

While I'm +1 on changing the default behavior here, the zope.interface issue is actually a bug in pip with how it resolves version requirements in packages.

I'll explain my version of what's going on without looking at pip's source.

# OS X 10.6.8
virtualenv -p python2.6 env

env/bin/python
>>> import pkg_resources
>>> pkg_resources.get_distribution('zope.interface')
zope.interface 3.5.1 (/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python)

Users are able to successfully run env/bin/pip install "pyramid==1.2.1". They then want to create a project and run env/bin/paster create -t pyramid_starter demo. This causes the below error:

pkg_resources.VersionConflict: (zope.interface 3.5.1 (/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python), Requirement.parse('zope.interface>=3.8.0'))

During the install they saw:

Requirement already satisfied (use --upgrade to upgrade): zope.interface>=3.5.1 in /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python (from pyramid==1.2.1)

Explanation:

pyramid pins zope.interface>=3.5.1, zope.component>=3.6.0
zope.component pins zope.interface>=3.8.0

It would appear as if pip is accepting the first "pin" it finds, and ignoring the z.c requirement of 3.8.0, causing issues later. This is not an issue with easy_install, which actually fails while doing easy_install "pyramid==1.2.1":

error: Installed distribution zope.interface 3.5.1 conflicts with requirement zope.interface>=3.8.0

In both cases it's possible to upgrade zope.interface within the virtualenv and then continue on, but for new users it's certainly not obvious.

[carljm]

Thanks for the extended analysis of the zope.interface situation. I believe that's bug pypa/pip#174 - knowing that it's biting people regularly (and that easy_install handles it better, which I didn't realize) definitely ups the priority level on getting that fixed.

Given the feedback on the mailing list thread, I'm not seeing any reason we shouldn't make this change. I may wait a few days for @jezdez to get back from vacation so I can confirm with him.

[jezdez]

Yeah, +1 from me.

[carljm]

Merged - thanks @mcdonc.