老齐教室

Python虚拟环境

导言

本文是针对《Python大学实用教程》《跟老齐学Python:轻松入门》两本书的基础内容之后的提升。

通常,在不同的项目中,会用到不同的Python版本——如果有必要的话,也包括一些库的版本,并非总是最新的就是最适合的或者最好的。对此,解决之道就是创建虚拟环境。

一个项目创建一个虚拟环境,在每个虚拟环境之中,就相当于一个“裸机”,里面所需要的东西任你根据当前的需要自由配置,并且不会影响到其他项目。特别是在诸如Linux系统上,有时候安装某些东西还需要root权限,此时虚拟环境的好处就更明显了。

创建虚拟环境

不论是Python2,还是Python3中,都可以创建虚拟环境——虽然现在广泛使用Python3,但也有项目在Python2上跑着呢。只是两个版本中创建方法稍有区别。

在Python3中创建虚拟环境

下面的方法是Python3中推荐的方法,务必掌握。

1
$ python3 -m venv python3venv

命令行上的-m参数是告诉解释器运行venv模块,此模块是Python3核心发行版的一部分。python3venv是虚拟环境的目录名称。执行之后就会创建python3venv,虚拟环境就在这个目录中。

如果对venv好奇,可以用下面的方式来瞧一瞧。

1
2
3
4
5
6
7
$ python3
Python 3.6.4 (v3.6.4:d48ecebad5, Dec 18 2017, 21:07:28)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import venv
>>> print(venv)
<module 'venv' from '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/venv/__init__.py'>

venv模块的代码在/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/venv/__init__.py,查看此文件(执行cat /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/venv/__init__.py即可看到),并到最底部,会看到如下内容:

1
2
3
4
5
6
7
8
if __name__ == '__main__':
rc = 1
try:
main()
rc = 0
except Exception as e:
print('Error: %s' % e, file=sys.stderr)
sys.exit(rc)

如果要查看命令行的参数,可以:

1
2
3
4
5
$ python3 -m venv
usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear]
[--upgrade] [--without-pip] [--prompt PROMPT]
ENV_DIR [ENV_DIR ...]
venv: error: the following arguments are required: ENV_DIR

在Python3.6及更低版本中创建虚拟环境

现在不推荐这种方法,但是,也介绍一下。因为这个方法主要用于Python3.6及以下版本中。

Python3.6及以下版本有一个名为pyvenv的可执行文件,它本质上是venv模块的二进制包装器。如果按按照下面的方法操作,就可以看到命令行的参数。

1
2
3
4
5
6
$ pyvenv
WARNING: the pyenv script is deprecated in favour of `python3.6 -m venv`
usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear]
[--upgrade] [--without-pip] [--prompt PROMPT]
ENV_DIR [ENV_DIR ...]
venv: error: the following arguments are required: ENV_DIR

使用它创建虚拟目录的方法是:

1
2
$ pyvenv python3venv
WARNING: the pyenv script is deprecated in favour of `python3.6 -m venv`

但是,会告诉我们,这个东西弃用了,后面要使用前面推荐的方法。

在Python2.x中创建虚拟环境

在Python2.x上,用virtualenv模块创建虚拟环境。在不同的操作系统中,对virtualenv可能有不同的处理方式,比如有的默认就安装了这个模块,有的没有安装,这就需要你先安装上。

检查一下它是不是在你的机器上了。

1
2
$ which virtualenv
/Library/Frameworks/Python.framework/Versions/2.7/bin/virtualenv

创建一个Python2.x的虚拟目录,其语法类似于前面的方法:

1
2
3
$ virtualenv python2venv
New python executable in /Users/james/python2venv/bin/python
Installing setuptools, pip, wheel...done.

检查虚拟环境

已经创建了两个虚拟环境的目录,分别是:

1
2
3
4
5
6
$ ls python3venv python2venv/
python2venv/:
bin include lib

python3venv:
bin include lib pyvenv.cfg

这两个目录的却别在于,python3venv中能够看到pyvenv.cfg文件,如果你有兴趣了解有关此文件的更多信息,可以在PEP-0405查看相关的背景知识。

激活虚拟环境

要使用虚拟环境,必须先激活。

注意,虚拟环境激活前后,shell的提示符会发生变化。原来是这样的:

1
2
$ echo $PATH
/Library/Frameworks/Python.framework/Versions/3.6/bin:/Library/Frameworks/Python.framework/Versions/2.7/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

激活虚拟环境之后(执行source python3venv/bin/activate指令后)显示(python3venv) $

1
2
$ source python3venv/bin/activate
(python3venv) $

检查路径后,虚拟环境将优先于前面做同样操作是呈现的路径。

1
2
(python3venv) $ echo $PATH
/Users/james/python3venv/bin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/Library/Frameworks/Python.framework/Versions/2.7/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

执行python

如果我们现在执行pythonpip,由于路径优先(前面的操作已经显示,python3.6优先执行了虚拟环境中),来自虚拟环境命令将优先执行。如果用下面的方式查看,会发现它们都在相应的虚拟环境中。

1
2
3
4
(python3venv) $ which python
/Users/james/python3venv/bin/python
(python3venv) $ which pip
/Users/james/python3venv/bin/pip

比如执行python,本来本地机器上安装两个版本的Python,如果不用虚拟环境,python之后默认会执行Python2,要执行Python3,必须是python3。但是,现在不同了,因为激活了Python3.6的虚拟环境,当执行python命令后,会首先在指定的虚拟环境中搜索。并且,进入到Python3之后,查看sys.path,当前虚拟环境目录也被添加到Python搜索路径中了。

1
2
3
4
5
6
7
(python3venv) $ python
Python 3.6.4 (v3.6.4:d48ecebad5, Dec 18 2017, 21:07:28)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.path)
['', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python36.zip', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload', '/Users/james/python3venv/lib/python3.6/site-packages']

用pip安装软件包

激活虚拟环境后,使用pip会将模块或者第三方包直接安装到虚拟环境。例如,如果我们安装了名为arrow的模块——专门解读日期、时间的模块,比传统的datatime模块使用更方便。

1
2
3
4
5
6
7
8
9
10
11
12
(python3venv) $ pip install arrow
Collecting arrow
Downloading https://files.pythonhosted.org/packages/f4/7f/0360628ba40bb93c10cd89cd289b6a8e9ea87b2db884b8edf32c80ee1c73/arrow-0.13.1-py2.py3-none-any.whl
Collecting python-dateutil (from arrow)
Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB)
100% |████████████████████████████████| 235kB 3.6MB/s
Collecting six>=1.5 (from python-dateutil->arrow)
Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Installing collected packages: six, python-dateutil, arrow
Successfully installed arrow-0.13.1 python-dateutil-2.8.0 six-1.12.0
You are using pip version 9.0.1, however version 19.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

上面的安装过程显示,在安装了arrow的过程中,还安装了sixpython-dateutil两个依赖模块。我们可以再次确认arrow模块在虚拟环境中。

1
2
3
4
5
6
7
(python3venv) $ python
Python 3.6.4 (v3.6.4:d48ecebad5, Dec 18 2017, 21:07:28)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import arrow
>>> print(arrow)
<module 'arrow' from '/Users/james/python3venv/lib/python3.6/site-packages/arrow/__init__.py'>

升级pip

在上面用pip安装的时候,最后有警公告,提示要升级pip,如果看到这个提示,就要必须升级,为什么?请阅读:

必须升级pip,为什么?!

可以按以下方式升级虚拟环境中的pip

1
2
3
4
5
6
7
8
9
10
11
12
(python3venv) $ pip install --upgrade pip
Cache entry deserialization failed, entry ignored
Collecting pip
Using cached https://files.pythonhosted.org/packages/d8/f3/413bab4ff08e1fc4828dfc59996d721917df8e8583ea85385d51125dceff/pip-19.0.3-py2.py3-none-any.whl
Installing collected packages: pip
Found existing installation: pip 9.0.1
Uninstalling pip-9.0.1:
Successfully uninstalled pip-9.0.1
Successfully installed pip-19.0.3

(python3venv) $ pip --version
pip 19.0.3 from /Users/james/python3venv/lib/python3.6/site-packages/pip (python 3.6)

注意:上面显示,不一定是你阅读到本文的时候pip的最新版本。)

集成环境需求

为了让下面的示例更能说明问题,在安装flask——一个用于web开发的框架。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
(python3venv) $ pip install flask
Collecting flask
Downloading https://files.pythonhosted.org/packages/7f/e7/08578774ed4536d3242b14dacb4696386634607af824ea997202cd0edb4b/Flask-1.0.2-py2.py3-none-any.whl (91kB)
100% |████████████████████████████████| 92kB 2.4MB/s
Collecting Jinja2>=2.10 (from flask)
Downloading https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl (126kB)
100% |████████████████████████████████| 133kB 7.6MB/s
Collecting Werkzeug>=0.14 (from flask)
Downloading https://files.pythonhosted.org/packages/20/c4/12e3e56473e52375aa29c4764e70d1b8f3efa6682bef8d0aae04fe335243/Werkzeug-0.14.1-py2.py3-none-any.whl (322kB)
100% |████████████████████████████████| 327kB 13.0MB/s
Collecting itsdangerous>=0.24 (from flask)
Downloading https://files.pythonhosted.org/packages/76/ae/44b03b253d6fade317f32c24d100b3b35c2239807046a4c953c7b89fa49e/itsdangerous-1.1.0-py2.py3-none-any.whl
Collecting click>=5.1 (from flask)
Downloading https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl (81kB)
100% |████████████████████████████████| 81kB 15.2MB/s
Collecting MarkupSafe>=0.23 (from Jinja2>=2.10->flask)
Downloading https://files.pythonhosted.org/packages/f0/00/a6aea33f5598b080b86d6b6d1214b51afe3ffa6100b902d5aa465080083f/MarkupSafe-1.1.1-cp36-cp36m-macosx_10_6_intel.whl
Installing collected packages: MarkupSafe, Jinja2, Werkzeug, itsdangerous, click, flask
Successfully installed Jinja2-2.10 MarkupSafe-1.1.1 Werkzeug-0.14.1 click-7.0 flask-1.0.2 itsdangerous-1.1.0

安装flask的同时,也会安装相关的依赖模块。这样,在虚拟环境中就有了比较多的模块,而且每个模块(或者库)都有一定的版本。

下面按照如下的指令执行,目的是要生成一个名为requirements.txt的文件,在这个文件中,将记录当前虚拟环境中所安装的模块及其版本(注意,requirements.txt文件目录,先下面代码这样指定,是为了后面使用方便。)。

1
2
3
4
5
6
7
8
9
10
(python3venv) $ pip freeze | tee /tmp/requirements.txt
arrow==0.13.1
Click==7.0
Flask==1.0.2
itsdangerous==1.1.0
Jinja2==2.10
MarkupSafe==1.1.1
python-dateutil==2.8.0
six==1.12.0
Werkzeug==0.14.1

有了这个文件,当我们需要重现这个虚拟环境中已经安装的各个模块是,就简单了。

重现虚拟环境配置

前面,曾经创建了一个Python2的虚拟环境,现在要将刚才在Python3的虚拟环境中安装的各个模块(即配置),移植到Python2的虚拟环境中——注意,这种做法并不提倡,但是,这里纯粹是为演示虚拟环境的重现移植。

首先,将现在的python3venv虚拟环境停用。

1
2
(python3venv) $ deactivate
$

然后激活Python2的虚拟环境python2venv

1
2
$ source python2venv/bin/activate
(python2venv) $

升级pip,多数情况下,用pip install –upgrade pip即可,但是,如果遇到了意想不到的事情,可以用下面的方式尝试解决。

1
2
3
4
5
6
7
8
9
10
11
12
$ (python2venv) curl https://bootstrap.pypa.io/get-pip.py | python
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1659k 100 1659k 0 0 2375k 0 --:--:-- --:--:-- --:--:-- 2377k
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Collecting pip
Using cached https://files.pythonhosted.org/packages/d8/f3/413bab4ff08e1fc4828dfc59996d721917df8e8583ea85385d51125dceff/pip-19.0.3-py2.py3-none-any.whl
Installing collected packages: pip
Found existing installation: pip 9.0.1
Uninstalling pip-9.0.1:
Successfully uninstalled pip-9.0.1
Successfully installed pip-19.0.3

现在,要将python3venv中的配置,在当前虚拟环境中重现,方法如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
$ (python2venv) pip install -r /tmp/requirements.txt
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Collecting arrow==0.13.1 (from -r /tmp/requirements.txt (line 1))
Using cached https://files.pythonhosted.org/packages/f4/7f/0360628ba40bb93c10cd89cd289b6a8e9ea87b2db884b8edf32c80ee1c73/arrow-0.13.1-py2.py3-none-any.whl
Collecting Click==7.0 (from -r /tmp/requirements.txt (line 2))
Using cached https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl
Collecting Flask==1.0.2 (from -r /tmp/requirements.txt (line 3))
Using cached https://files.pythonhosted.org/packages/7f/e7/08578774ed4536d3242b14dacb4696386634607af824ea997202cd0edb4b/Flask-1.0.2-py2.py3-none-any.whl
Collecting itsdangerous==1.1.0 (from -r /tmp/requirements.txt (line 4))
Using cached https://files.pythonhosted.org/packages/76/ae/44b03b253d6fade317f32c24d100b3b35c2239807046a4c953c7b89fa49e/itsdangerous-1.1.0-py2.py3-none-any.whl
Collecting Jinja2==2.10 (from -r /tmp/requirements.txt (line 5))
Using cached https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl
Collecting MarkupSafe==1.1.1 (from -r /tmp/requirements.txt (line 6))
Downloading https://files.pythonhosted.org/packages/6d/d2/0ccd2c0e2cd93b35e765d9b3205cd6602e6b202b522fc7997531353715b3/MarkupSafe-1.1.1-cp27-cp27m-macosx_10_6_intel.whl
Collecting python-dateutil==2.8.0 (from -r /tmp/requirements.txt (line 7))
Using cached https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl
Collecting six==1.12.0 (from -r /tmp/requirements.txt (line 8))
Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting Werkzeug==0.14.1 (from -r /tmp/requirements.txt (line 9))
Using cached https://files.pythonhosted.org/packages/20/c4/12e3e56473e52375aa29c4764e70d1b8f3efa6682bef8d0aae04fe335243/Werkzeug-0.14.1-py2.py3-none-any.whl
Collecting backports.functools-lru-cache>=1.2.1; python_version == "2.7" (from arrow==0.13.1->-r /tmp/requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/03/8e/2424c0e65c4a066e28f539364deee49b6451f8fcd4f718fefa50cc3dcf48/backports.functools_lru_cache-1.5-py2.py3-none-any.whl
Installing collected packages: six, python-dateutil, backports.functools-lru-cache, arrow, Click, MarkupSafe, Jinja2, itsdangerous, Werkzeug, Flask
Successfully installed Click-7.0 Flask-1.0.2 Jinja2-2.10 MarkupSafe-1.1.1 Werkzeug-0.14.1 arrow-0.13.1 backports.functools-lru-cache-1.5 itsdangerous-1.1.0 python-dateutil-2.8.0 six-1.12.0

虽然现在是一个Python2的虚拟环境,但是也按照requirement.txt文件中的记录,配置了与Python3虚拟环境相同的各个模块,并且还根据Python2的特定要求,增加了应有的依赖backports.functools-lru-cache==1.5 。于是当前虚拟环境中的配置如下:

1
2
3
4
5
6
7
8
9
10
11
12
(python2venv) $ pip freeze
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
arrow==0.13.1
backports.functools-lru-cache==1.5
Click==7.0
Flask==1.0.2
itsdangerous==1.1.0
Jinja2==2.10
MarkupSafe==1.1.1
python-dateutil==2.8.0
six==1.12.0
Werkzeug==0.14.1

不激活的情况下,使用虚拟环境

一般情况下,按照上面所说的,激活虚拟环境,然后开始使用,这已经很方便了。但是,在某些情况下,或许有不激活虚拟环境的需求。

为此,可以这么做。比如要执行虚拟环境python3venv中的Python3,可以在没有激活任何虚拟环境的情况下,直接执行python3venv/bin/python运行python3venv中的Python。

在下面的操作中,首先要从python2venv的虚拟环境中退出。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
(python2venv) $ deactivate
$ python3venv/bin/python
Python 3.6.4 (v3.6.4:d48ecebad5, Dec 18 2017, 21:07:28)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import arrow
>>> print(arrow)
<module 'arrow' from '/Users/james/python3venv/lib/python3.6/site-packages/arrow/__init__.py'>
>>> ^D

$ python2venv/bin/python
Python 2.7.14 (v2.7.14:84471935ed, Sep 16 2017, 12:01:12)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import arrow
>>> print(arrow)
<module 'arrow' from '/Users/james/python2venv/lib/python2.7/site-packages/arrow/__init__.pyc'>
>>> ^D

在程序文件中,也可以指定所使用的虚拟环境,方法就是用#!发起指令,如下面的代码所示:

1
2
3
4
5
6
7
$ cat <<EOF > test.py
> #!/Users/james/python3venv/bin/python
>
> import arrow
> utc = arrow.utcnow()
> print(utc)
> EOF

设置示例文件test.py可执行,然后执行,可以确认执行了虚拟环境中的模块。

1
2
3
$ chmod u+x test.py
$ ./test.py
2019-03-03T21:17:22.632130+00:00

如此,即可根据特定需求执行程序了。

打包

如果在本地完成了开发,要将程序移植到别的机器上,通常要在目标机器上配置与本地一样的环境。当然,现在有一种容器化的方案,比如使用docker等。如果不那样做,按照上面的方法完成了虚拟环境的配置,并且在虚拟环境中完成了程序编写。那么,就可以将真个虚拟环境打包。当然,这里有一个前提,目标机器上必须安装了与虚拟环境中相同版本的Python。

以前面使用过的python3venv为例,用tar命令将目录python3venv打包。

1
$ tar cf python3venv.tar python3venv

为了演示需要,将当前的python3venv目录删除。

1
$ rm -rf python3venv

然后从python3venv.tar中将目录提取出来。

1
$ tar xf python3venv.tar

继续使用test.py测试一下,python3venv虚拟目录及其内部安装的各模块,都已经恢复。

1
2
$ ./test.py
2019-03-03T21:26:47.082092+00:00

用pip管理虚拟环境

pip是一个好工具,它为管理和维护虚拟环境及其依赖提供简便方法。比如,对于虚拟环境python3venv中的模块,如果要删除arrowflask,如下操作所示,使用pip uninstall packagename即可,不过,安装时候所安装的那些依赖项会依然保留。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
(python3venv) $ pip uninstall arrow flask
Uninstalling arrow-0.13.1:
Would remove:
/Users/james/python3venv/lib/python3.6/site-packages/arrow-0.13.1.dist-info/*
/Users/james/python3venv/lib/python3.6/site-packages/arrow/*
Proceed (y/n)? y
Successfully uninstalled arrow-0.13.1
Uninstalling Flask-1.0.2:
Would remove:
/Users/james/python3venv/bin/flask
/Users/james/python3venv/lib/python3.6/site-packages/Flask-1.0.2.dist-info/*
/Users/james/python3venv/lib/python3.6/site-packages/flask/*
Proceed (y/n)? y
Successfully uninstalled Flask-1.0.2
(python3venv) $ pip freeze
Click==7.0
itsdangerous==1.1.0
Jinja2==2.10
MarkupSafe==1.1.1
python-dateutil==2.8.0
six==1.12.0
Werkzeug==0.14.1

除了上面简单的方法之外,还可以通过pip-tools中的工具,实现对模块更灵活的管理。

首先,要退出当前的虚拟环境,并删除python3venv目录。然后,重新创建虚拟环境——从这就可以看出虚拟环境的优势,你可以任意删除和重建,每次重建之后它都是“干干净净”的。

进入到新建的虚拟环境之后,升级pip——这是常规套路。再安装pip-tools

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
(python3venv) $ deactivate
$ rm -rf python3venv
$ python3 -m venv python3venv
$ source python3venv/bin/activate
(python3venv) $ pip install --upgrade pip
Cache entry deserialization failed, entry ignored
Collecting pip
Using cached https://files.pythonhosted.org/packages/d8/f3/413bab4ff08e1fc4828dfc59996d721917df8e8583ea85385d51125dceff/pip-19.0.3-py2.py3-none-any.whl
Installing collected packages: pip
Found existing installation: pip 9.0.1
Uninstalling pip-9.0.1:
Successfully uninstalled pip-9.0.1
Successfully installed pip-19.0.3

(python3venv) $ pip install pip-tools
Collecting pip-tools
Downloading https://files.pythonhosted.org/packages/58/7a/f93b24807b7ac2d9d0bd6b8a886bcbe67eb39c1b1184b985dd5e0e2eca92/pip_tools-3.4.0-py2.py3-none-any.whl (43kB)
100% |████████████████████████████████| 51kB 2.3MB/s
Collecting six (from pip-tools)
Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting click>=6 (from pip-tools)
Using cached https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl
Installing collected packages: six, click, pip-tools
Successfully installed click-7.0 pip-tools-3.4.0 six-1.12.0

创建一个名为requirements.in的文件,在其中写上需要的模块及其版版本,比如arrowflask,版本如下:

1
2
3
4
(python3venv) $ cat <<EOF > requirements.in
> arrow==0.13.1
> Flask==1.0.2
> EOF

然后,使用pip-compile (是pip-tools中提供的),依据requirements.in创建requirements.txt文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
(python3venv) $ pip-compile --generate-hashes --output-file requirements.txt requirements.in
#
# This file is autogenerated by pip-compile
# To update, run:
#
# pip-compile --generate-hashes --output-file requirements.txt requirements.in
#
arrow==0.13.1 \
--hash=sha256:3397e5448952e18e1295bf047014659effa5ae8da6a5371d37ff0ddc46fa6872 \
--hash=sha256:6f54d9f016c0b7811fac9fb8c2c7fa7421d80c54dbdd75ffb12913c55db60b8a
click==7.0 \
--hash=sha256:2335065e6395b9e67ca716de5f7526736bfa6ceead690adf616d925bdc622b13 \
--hash=sha256:5b94b49521f6456670fdb30cd82a4eca9412788a93fa6dd6df72c94d5a8ff2d7 \
# via flask
flask==1.0.2 \
--hash=sha256:2271c0070dbcb5275fad4a82e29f23ab92682dc45f9dfbc22c02ba9b9322ce48 \
--hash=sha256:a080b744b7e345ccfcbc77954861cb05b3c63786e93f2b3875e0913d44b43f05
itsdangerous==1.1.0 \
--hash=sha256:321b033d07f2a4136d3ec762eac9f16a10ccd60f53c0c91af90217ace7ba1f19 \
--hash=sha256:b12271b2047cb23eeb98c8b5622e2e5c5e9abd9784a153e9d8ef9cb4dd09d749 \
# via flask
jinja2==2.10 \
--hash=sha256:74c935a1b8bb9a3947c50a54766a969d4846290e1e788ea44c1392163723c3bd \
--hash=sha256:f84be1bb0040caca4cea721fcbbbbd61f9be9464ca236387158b0feea01914a4 \
# via flask
markupsafe==1.1.1 \
--hash=sha256:00bc623926325b26bb9605ae9eae8a215691f33cae5df11ca5424f06f2d1f473 \
--hash=sha256:09027a7803a62ca78792ad89403b1b7a73a01c8cb65909cd876f7fcebd79b161 \
--hash=sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235 \
--hash=sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5 \
--hash=sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff \
--hash=sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b \
--hash=sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1 \
--hash=sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e \
--hash=sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183 \
--hash=sha256:535f6fc4d397c1563d08b88e485c3496cf5784e927af890fb3c3aac7f933ec66 \
--hash=sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1 \
--hash=sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1 \
--hash=sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e \
--hash=sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b \
--hash=sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905 \
--hash=sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735 \
--hash=sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d \
--hash=sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e \
--hash=sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d \
--hash=sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c \
--hash=sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21 \
--hash=sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2 \
--hash=sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5 \
--hash=sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b \
--hash=sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6 \
--hash=sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f \
--hash=sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f \
--hash=sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7 \
# via jinja2
python-dateutil==2.8.0 \
--hash=sha256:7e6584c74aeed623791615e26efd690f29817a27c73085b78e4bad02493df2fb \
--hash=sha256:c89805f6f4d64db21ed966fda138f8a5ed7a4fdbc1a8ee329ce1b74e3c74da9e \
# via arrow
six==1.12.0 \
--hash=sha256:3350809f0555b11f552448330d0b52d5f24c91a322ea4a15ef22629740f3761c \
--hash=sha256:d16a0141ec1a18405cd4ce8b4613101da75da0e9a7aec5bdd4fa804d0e0eba73 \
# via python-dateutil
werkzeug==0.14.1 \
--hash=sha256:c3fd7a7d41976d9f44db327260e263132466836cef6f91512889ed60ad26557c \
--hash=sha256:d5da73735293558eb1651ee2fddc4d0dedcfa06538b8813a2e20011583c9e49b \
# via flask

从上述信息中可以看到,每个依赖都有相应的哈希值,这样能够保证所安装的模块彼此之间没有冲突。

有了requirements.txt文件之后,就可以使用pip-sync(也是pip-tools中的一个工具)将虚拟环境中的配置与之同步。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
(python3venv) $ pip-sync
Collecting arrow==0.13.1 (from -r /var/folders/xk/p46swdnn3b56rbd01qwdjhdh0000gn/T/tmpfwkk4j83 (line 1))
Using cached https://files.pythonhosted.org/packages/f4/7f/0360628ba40bb93c10cd89cd289b6a8e9ea87b2db884b8edf32c80ee1c73/arrow-0.13.1-py2.py3-none-any.whl
Collecting flask==1.0.2 (from -r /var/folders/xk/p46swdnn3b56rbd01qwdjhdh0000gn/T/tmpfwkk4j83 (line 4))
Using cached https://files.pythonhosted.org/packages/7f/e7/08578774ed4536d3242b14dacb4696386634607af824ea997202cd0edb4b/Flask-1.0.2-py2.py3-none-any.whl
Collecting itsdangerous==1.1.0 (from -r /var/folders/xk/p46swdnn3b56rbd01qwdjhdh0000gn/T/tmpfwkk4j83 (line 7))
Using cached https://files.pythonhosted.org/packages/76/ae/44b03b253d6fade317f32c24d100b3b35c2239807046a4c953c7b89fa49e/itsdangerous-1.1.0-py2.py3-none-any.whl
Collecting jinja2==2.10 (from -r /var/folders/xk/p46swdnn3b56rbd01qwdjhdh0000gn/T/tmpfwkk4j83 (line 10))
Using cached https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl
Collecting markupsafe==1.1.1 (from -r /var/folders/xk/p46swdnn3b56rbd01qwdjhdh0000gn/T/tmpfwkk4j83 (line 13))
Using cached https://files.pythonhosted.org/packages/f0/00/a6aea33f5598b080b86d6b6d1214b51afe3ffa6100b902d5aa465080083f/MarkupSafe-1.1.1-cp36-cp36m-macosx_10_6_intel.whl
Collecting python-dateutil==2.8.0 (from -r /var/folders/xk/p46swdnn3b56rbd01qwdjhdh0000gn/T/tmpfwkk4j83 (line 42))
Using cached https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl
Collecting werkzeug==0.14.1 (from -r /var/folders/xk/p46swdnn3b56rbd01qwdjhdh0000gn/T/tmpfwkk4j83 (line 45))
Using cached https://files.pythonhosted.org/packages/20/c4/12e3e56473e52375aa29c4764e70d1b8f3efa6682bef8d0aae04fe335243/Werkzeug-0.14.1-py2.py3-none-any.whl
Requirement already satisfied: click>=5.1 in ./python3venv/lib/python3.6/site-packages (from flask==1.0.2->-r /var/folders/xk/p46swdnn3b56rbd01qwdjhdh0000gn/T/tmpfwkk4j83 (line 4)) (7.0)
Requirement already satisfied: six>=1.5 in ./python3venv/lib/python3.6/site-packages (from python-dateutil==2.8.0->-r /var/folders/xk/p46swdnn3b56rbd01qwdjhdh0000gn/T/tmpfwkk4j83 (line 42)) (1.12.0)
Installing collected packages: python-dateutil, arrow, werkzeug, itsdangerous, markupsafe, jinja2, flask
Successfully installed arrow-0.13.1 flask-1.0.2 itsdangerous-1.1.0 jinja2-2.10 markupsafe-1.1.1 python-dateutil-2.8.0 werkzeug-0.14.1
(python3venv) $ pip freeze
arrow==0.13.1
Click==7.0
Flask==1.0.2
itsdangerous==1.1.0
Jinja2==2.10
MarkupSafe==1.1.1
pip-tools==3.4.0
python-dateutil==2.8.0
six==1.12.0
Werkzeug==0.14.1

如果删除一个模块,例如删除Flask,只需要更新requirements.in文件,如下所示。然后重复上面的过程,使用pip-sync,就可以从当前虚拟环境中删除Flask,并且,在安装的时候,随着一同安装上的各种依赖模块,也同时被移除。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
(python3venv) $ cat <<EOF > requirements.in
> arrow==0.13.1
> EOF
(python3venv) $ pip-compile --generate-hashes --output-file requirements.txt requirements.in
#
# This file is autogenerated by pip-compile
# To update, run:
#
# pip-compile --generate-hashes --output-file requirements.txt requirements.in
#
arrow==0.13.1 \
--hash=sha256:3397e5448952e18e1295bf047014659effa5ae8da6a5371d37ff0ddc46fa6872 \
--hash=sha256:6f54d9f016c0b7811fac9fb8c2c7fa7421d80c54dbdd75ffb12913c55db60b8a
python-dateutil==2.8.0 \
--hash=sha256:7e6584c74aeed623791615e26efd690f29817a27c73085b78e4bad02493df2fb \
--hash=sha256:c89805f6f4d64db21ed966fda138f8a5ed7a4fdbc1a8ee329ce1b74e3c74da9e \
# via arrow
six==1.12.0 \
--hash=sha256:3350809f0555b11f552448330d0b52d5f24c91a322ea4a15ef22629740f3761c \
--hash=sha256:d16a0141ec1a18405cd4ce8b4613101da75da0e9a7aec5bdd4fa804d0e0eba73 \
# via python-dateutil
(python3venv) $ pip-sync
Uninstalling Flask-1.0.2:
Successfully uninstalled Flask-1.0.2
Uninstalling itsdangerous-1.1.0:
Successfully uninstalled itsdangerous-1.1.0
Uninstalling Jinja2-2.10:
Successfully uninstalled Jinja2-2.10
Uninstalling MarkupSafe-1.1.1:
Successfully uninstalled MarkupSafe-1.1.1
Uninstalling Werkzeug-0.14.1:
Successfully uninstalled Werkzeug-0.14.1
(python3venv) $ pip freeze
arrow==0.13.1
Click==7.0
pip-tools==3.4.0
python-dateutil==2.8.0
six==1.12.0

指定PyPi库

如果需要将虚拟环境与特定的PyPi库关联起来,即要求安装指定的PyP中的模块或者库,为此需要在虚拟环境的根目录下创建一个名为pip.conf的配置文件,在这个文件中声明安装依赖的属性。下面是一个pip.conf的例子,要根据需要修改hostname.spurin.com

1
2
3
4
5
[global]
disable-pip-version-check = True
trusted-host = hostname.spurin.com
index = https://hostname.spurin.com/repository/pypi-all/pypi
index-url = https://hostname.spurin.com/repository/pypi-all/simple

创建虚拟环境中模块的轮子

必须升级pip,为什么?!中曾经介绍了.whl文件的作用,虚拟环境中安装的模块,也可以编译为.whil文件,如果在另外一台机器上再安装同样模块,只需要用pip安装此文件即可,从而让虚拟环境的移植更简单了。

具体操作流程如下所示。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
(python3venv) $ pip wheel --wheel-dir=/tmp/wheel-dir -r requirements.txt
Collecting arrow==0.13.1 (from -r requirements.txt (line 7))
Using cached https://files.pythonhosted.org/packages/f4/7f/0360628ba40bb93c10cd89cd289b6a8e9ea87b2db884b8edf32c80ee1c73/arrow-0.13.1-py2.py3-none-any.whl
Saved /private/tmp/wheel-dir/arrow-0.13.1-py2.py3-none-any.whl
Collecting python-dateutil==2.8.0 (from -r requirements.txt (line 10))
Using cached https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl
Saved /private/tmp/wheel-dir/python_dateutil-2.8.0-py2.py3-none-any.whl
Collecting six==1.12.0 (from -r requirements.txt (line 14))
Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Saved /private/tmp/wheel-dir/six-1.12.0-py2.py3-none-any.whl

(python3venv) $ ls /tmp/wheel-dir
arrow-0.13.1-py2.py3-none-any.whl python_dateutil-2.8.0-py2.py3-none-any.whl six-1.12.0-py2.py3-none-any.whl
(python3venv) $ pip uninstall arrow python_dateutil six
Uninstalling arrow-0.13.1:
Would remove:
/Users/james/python3venv/lib/python3.6/site-packages/arrow-0.13.1.dist-info/*
/Users/james/python3venv/lib/python3.6/site-packages/arrow/*
Proceed (y/n)? y
Successfully uninstalled arrow-0.13.1
Uninstalling python-dateutil-2.8.0:
Would remove:
/Users/james/python3venv/lib/python3.6/site-packages/dateutil/*
/Users/james/python3venv/lib/python3.6/site-packages/python_dateutil-2.8.0.dist-info/*
Proceed (y/n)? y
Successfully uninstalled python-dateutil-2.8.0
Uninstalling six-1.12.0:
Would remove:
/Users/james/python3venv/lib/python3.6/site-packages/six-1.12.0.dist-info/*
/Users/james/python3venv/lib/python3.6/site-packages/six.py
Proceed (y/n)? y
Successfully uninstalled six-1.12.0
(python3venv) $ pip install --find-links=/tmp/wheel-dir -r requirements.txt
Looking in links: /tmp/wheel-dir
Collecting arrow==0.13.1 (from -r requirements.txt (line 7))
Collecting python-dateutil==2.8.0 (from -r requirements.txt (line 10))
Collecting six==1.12.0 (from -r requirements.txt (line 14))
Installing collected packages: six, python-dateutil, arrow
Successfully installed arrow-0.13.1 python-dateutil-2.8.0 six-1.12.0

结束语

创建虚拟环境的目的,就是隔离特定模块的版本,从而满足不同项目的特定需要。

另外,除了上述创建虚拟环境的方法之外,现在也流行着一个名为pipenv的模块,有兴趣的可以试试,代码仓库地址:https://github.com/pypa/pipenv 。当然,对它也是仁者见仁智者见智。

参考文献

[1]. https://spurin.com/2019/03/12/Python-Virtual-Environments/

使用支付宝打赏
使用微信打赏

若你觉得我的文章对你有帮助,欢迎点击上方按钮对我打赏

关注微信公众号,读文章、听课程,提升技能