对于不同方式下载numpy的性能测试

242 阅读6分钟

测试环境

  • 系统:Windows 11
  • CPU:Intel i9-12900K
  • RAM:64GB
  • Python:python3.12

1. 使用 conda install numpy

终端信息:

(test) C:\Users\Guo18>conda install numpy
Channels:
 - defaults
Platform: win-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: C:\Users\Guo18\.conda\envs\test

  added / updated specs:
    - numpy


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    blas-1.0                   |              mkl           6 KB
    intel-openmp-2023.1.0      |   h59b6b97_46320         2.7 MB
    mkl-2023.1.0               |   h6b88ed4_46358       155.9 MB
    mkl-service-2.4.0          |  py312h827c3e9_2          64 KB
    mkl_fft-1.3.11             |  py312h827c3e9_0         169 KB
    mkl_random-1.2.8           |  py312h0158946_0         252 KB
    numpy-2.3.1                |  py312h5f75535_0          12 KB
    numpy-base-2.3.1           |  py312h23d94f8_0         8.3 MB
    setuptools-72.1.0          |  py312haa95532_0         2.9 MB
    tbb-2021.8.0               |       h59b6b97_0         149 KB
    ------------------------------------------------------------
                                           Total:       170.4 MB

The following NEW packages will be INSTALLED:

  blas               pkgs/main/win-64::blas-1.0-mkl
  intel-openmp       pkgs/main/win-64::intel-openmp-2023.1.0-h59b6b97_46320
  mkl                pkgs/main/win-64::mkl-2023.1.0-h6b88ed4_46358
  mkl-service        pkgs/main/win-64::mkl-service-2.4.0-py312h827c3e9_2
  mkl_fft            pkgs/main/win-64::mkl_fft-1.3.11-py312h827c3e9_0
  mkl_random         pkgs/main/win-64::mkl_random-1.2.8-py312h0158946_0
  numpy              pkgs/main/win-64::numpy-2.3.1-py312h5f75535_0
  numpy-base         pkgs/main/win-64::numpy-base-2.3.1-py312h23d94f8_0
  tbb                pkgs/main/win-64::tbb-2021.8.0-h59b6b97_0

The following packages will be DOWNGRADED:

  setuptools                         78.1.1-py312haa95532_0 --> 72.1.0-py312haa95532_0

结果

(test) C:\work\python\test>python test_numpy.py
Using NumPy version: 2.3.1

Matrix multiplication    : 0.045153 seconds (avg over 3 runs)
Matrix inversion         : 0.099050 seconds (avg over 3 runs)
SVD decomposition        : 0.996503 seconds (avg over 3 runs)

查看安装的numpy信息

(test) C:\work\python\test>pip show numpy
Name: numpy
Version: 2.3.1
Summary: Fundamental package for array computing in Python
Home-page: https://numpy.org
Author: Travis E. Oliphant et al.
Author-email:
License: Copyright (c) 2005-2025, NumPy Developers.
 All rights reserved.

 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions are
 met:

     * Redistributions of source code must retain the above copyright
        notice, this list of conditions and the following disclaimer.

     * Redistributions in binary form must reproduce the above
        copyright notice, this list of conditions and the following
        disclaimer in the documentation and/or other materials provided
        with the distribution.

     * Neither the name of the NumPy Developers nor the names of any
        contributors may be used to endorse or promote products derived
        from this software without specific prior written permission.

 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Location: C:\Users\Guo18\.conda\envs\test\Lib\site-packages
Requires:
Required-by: mkl_fft, mkl_random

显然从 Required-by 可以看到依赖并链接了 MKL 库。

2. 使用 pip install numpy

终端信息:

(test-pip) C:\work\python\test>pip install numpy
Collecting numpy
  Downloading numpy-2.3.1-cp312-cp312-win_amd64.whl.metadata (60 kB)
Downloading numpy-2.3.1-cp312-cp312-win_amd64.whl (12.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.7/12.7 MB 20.0 MB/s eta 0:00:00
Installing collected packages: numpy
Successfully installed numpy-2.3.1

结果

(test-pip) C:\work\python\test>python test_numpy.py
Using NumPy version: 2.3.1

Matrix multiplication    : 0.045519 seconds (avg over 3 runs)
Matrix inversion         : 0.177157 seconds (avg over 3 runs)
SVD decomposition        : 1.588028 seconds (avg over 3 runs)

查看安装的numpy信息

(test-pip) C:\work\python\test>pip show numpy
Name: numpy
Version: 2.3.1
Summary: Fundamental package for array computing in Python
Home-page: https://numpy.org
Author: Travis E. Oliphant et al.
Author-email:
License: Copyright (c) 2005-2025, NumPy Developers.
 All rights reserved.

 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions are
 met:

     * Redistributions of source code must retain the above copyright
        notice, this list of conditions and the following disclaimer.

     * Redistributions in binary form must reproduce the above
        copyright notice, this list of conditions and the following
        disclaimer in the documentation and/or other materials provided
        with the distribution.

     * Neither the name of the NumPy Developers nor the names of any
        contributors may be used to endorse or promote products derived
        from this software without specific prior written permission.

 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

 ----

 The NumPy repository and source distributions bundle several libraries that are
 compatibly licensed.  We list these here.

 Name: lapack-lite
 Files: numpy/linalg/lapack_lite/*
 License: BSD-3-Clause
   For details, see numpy/linalg/lapack_lite/LICENSE.txt

 Name: dragon4
 Files: numpy/_core/src/multiarray/dragon4.c
 License: MIT
   For license text, see numpy/_core/src/multiarray/dragon4.c

 Name: libdivide
 Files: numpy/_core/include/numpy/libdivide/*
 License: Zlib
   For license text, see numpy/_core/include/numpy/libdivide/LICENSE.txt


 Note that the following files are vendored in the repository and sdist but not
 installed in built numpy packages:

 Name: Meson
 Files: vendored-meson/meson/*
 License: Apache 2.0
   For license text, see vendored-meson/meson/COPYING

 Name: spin
 Files: .spin/cmds.py
 License: BSD-3
   For license text, see .spin/LICENSE

 Name: tempita
 Files: numpy/_build_utils/tempita/*
 License: MIT
   For details, see numpy/_build_utils/tempita/LICENCE.txt

 ----

 This binary distribution of NumPy also bundles the following software:


 Name: OpenBLAS
 Files: numpy.libs\libscipy_openblas*.dll
 Description: bundled as a dynamically linked library
 Availability: https://github.com/OpenMathLib/OpenBLAS/
 License: BSD-3-Clause
   Copyright (c) 2011-2014, The OpenBLAS Project
   All rights reserved.

   Redistribution and use in source and binary forms, with or without
   modification, are permitted provided that the following conditions are
   met:

      1. Redistributions of source code must retain the above copyright
         notice, this list of conditions and the following disclaimer.

      2. Redistributions in binary form must reproduce the above copyright
         notice, this list of conditions and the following disclaimer in
         the documentation and/or other materials provided with the
         distribution.
      3. Neither the name of the OpenBLAS project nor the names of
         its contributors may be used to endorse or promote products
         derived from this software without specific prior written
         permission.

   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
   AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
   IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
   ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
   LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
   DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
   CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
   OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
   USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
....

Location: C:\Users\Guo18\.conda\envs\test-pip\Lib\site-packages
Requires:
Required-by:

尽管没有显示任何依赖信息,但是在配置信息中可以看到使用了 OpenBLAS 的链接库。

总结

使用 conda install numpy 安装的numpy性能会优于使用 pip install numpy 的numpy性能。原因是 conda 会去自动连接因特尔的MKL优化库,在一些场景下比默认numpy使用的OpenBLAS库性能更好。

Q & A

MKL和OpenBLAS是什么?为什么numpy会使用?为什么会出现性能差距?

MKL和OpenBLAS都是科学计算的数学库,它们都实现了标准的 BLAS(Basic Linear Algebra Subprograms)和 LAPACK 接口用于向量/矩阵加法和乘法、求解线性方程组、奇异值分解、特征值分解等线性代数运算。 MKL全称Intel Math Kernel Library,主要由Intel开发维护,针对 Intel CPU 深度优化,支持多线程,性能强大。 OpenBLAS全称Open Basic Linear Algebra Subprograms,主要由开源社区开发维护,针对多种CPU架构优化(如AMD、ARM),兼容性强。 NumPy 本身 不实现高性能数学计算的底层细节,而是依赖像 MKL 或 OpenBLAS 来加速底层操作。 因为MKL 对 Intel CPU 使用 AVX512、AVX2、SSE 等指令集做深度优化,而 OpenBLAS 只能使用较为通用的优化手段,所以一般Intel CPU上MKL的性能要优于OpenBLAS。

如果不想使用conda但还是想要使用numpy-mkl,应该怎么做?

可以选择手动编译链接,但是更推荐使用命令 python -m pip install -i https://pypi.anaconda.org/intel/simple numpy

AMD CPU可以使用MKL吗?性能也会更好吗?

AMD CPU 可以使用 Intel MKL,Intel MKL(Math Kernel Library)是通用的 x86 数学计算库,只要是兼容 x86_64 指令集的处理器都可以使用。 虽然能运行,但在 AMD CPU 上,MKL 可能会故意降级使用“低性能指令”,尤其是在旧版本中。MKL 会根据 CPUID 检测 CPU 是不是 Intel 的。如果不是,它可能强制使用老旧的 SSE2 指令集(而不是更快的 AVX2、AVX512),绕过对 AMD 支持的调度逻辑。这会导致即使 AMD CPU 性能很强,MKL 也不会用最快的代码路径。