.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "examples/benchmarks/plot_benchmarks.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. or to run this example in your browser via Binder .. rst-class:: sphx-glr-example-title .. _sphx_glr_examples_benchmarks_plot_benchmarks.py: Backends comparison =================== Numerical backends performace comparison. .. GENERATED FROM PYTHON SOURCE LINES 14-30 .. code-block:: Python import matplotlib.pyplot as plt import numpy as np markers = ["o", "s", "d", "v", "^", ">"] figsize = (2, 2) threads = [1, 2, 4, 8, 16] devices = ["cpu", "gpu"] # backends = ["numpy", "scipy", "autograd", "jax", "torch"] # we skip jax as it is complicated to deal with multithreading so a fair comparison is impossible backends = ["numpy", "scipy", "autograd", "torch"] colors = ["#3b9dd4", "#ecd142", "#e87c40", "#b33dd1", "#50ba61", "#cd2323"] .. GENERATED FROM PYTHON SOURCE LINES 31-33 Time vs. number of harmonics ------------------------------ .. GENERATED FROM PYTHON SOURCE LINES 33-79 .. code-block:: Python for num_threads in threads: plt.figure(figsize=figsize) i = 0 for backend in backends: for device in devices: g = "cuda" if device == "gpu" else device if device != "gpu" or backend not in [ "numpy", "scipy", "autograd", "jax", ]: arch = np.load( f"{num_threads}/benchmark_{backend}_{g}.npz", allow_pickle=True ) NH = arch["real_nh"] plt.plot( arch["real_nh"], arch["times"], f"-{markers[i]}", c=colors[i], label=f"{backend} {device}", ) times_all = np.array(arch["times_all"]) times_std = np.std(times_all, axis=1) plt.errorbar( arch["real_nh"], arch["times"], times_std, c=colors[i], capsize=1, ) i += 1 plt.legend() plt.yscale("log") plt.xscale("log") plt.xlabel("number of harmonics") plt.ylabel("time (s)") plt.title(f"backends comparison {num_threads} threads") plt.tight_layout() .. rst-class:: sphx-glr-horizontal * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_001.png :alt: backends comparison 1 threads :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_001.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_002.png :alt: backends comparison 2 threads :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_002.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_003.png :alt: backends comparison 4 threads :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_003.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_004.png :alt: backends comparison 8 threads :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_004.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_005.png :alt: backends comparison 16 threads :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_005.png :class: sphx-glr-multi-img .. GENERATED FROM PYTHON SOURCE LINES 80-82 Speedup vs. number of harmonics ----------------------------------- .. GENERATED FROM PYTHON SOURCE LINES 82-126 .. code-block:: Python for num_threads in threads: plt.figure(figsize=figsize) arch_np = np.load(f"{num_threads}/benchmark_numpy_cpu.npz", allow_pickle=True) i = 1 for backend in backends: for device in devices: g = "cuda" if device == "gpu" else device if device != "gpu" or backend == "torch": arch = np.load( f"{num_threads}/benchmark_{backend}_{g}.npz", allow_pickle=True ) if backend != "numpy": speedup = np.array(arch_np["times"]) / np.array(arch["times"]) plt.plot( arch["real_nh"], speedup, f"-{markers[i]}", c=colors[i], label=f"{backend} {device}", ) speedup_all = np.array(arch_np["times_all"]) / np.array( arch["times_all"] ) speedup_std = np.std(speedup_all, axis=1) plt.errorbar( arch["real_nh"], speedup, speedup_std, c=colors[i], capsize=1, ) i += 1 plt.legend() # plt.yscale("log") # plt.xscale("log") plt.xlabel("number of harmonics") plt.ylabel("speedup vs. numpy") plt.title(f"backends comparison {num_threads} threads") plt.tight_layout() .. rst-class:: sphx-glr-horizontal * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_006.png :alt: backends comparison 1 threads :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_006.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_007.png :alt: backends comparison 2 threads :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_007.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_008.png :alt: backends comparison 4 threads :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_008.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_009.png :alt: backends comparison 8 threads :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_009.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_010.png :alt: backends comparison 16 threads :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_010.png :class: sphx-glr-multi-img .. GENERATED FROM PYTHON SOURCE LINES 127-129 Time vs. number of threads ----------------------------- .. GENERATED FROM PYTHON SOURCE LINES 129-178 .. code-block:: Python for inh in range(len(NH)): plt.figure(figsize=figsize) i = 0 for backend in backends: for device in devices: t_threads = [] t_threads_all = [] for num_threads in threads: if device != "gpu" or backend == "torch": g = "cuda" if device == "gpu" else device arch = np.load( f"{num_threads}/benchmark_{backend}_{g}.npz", allow_pickle=True ) t = arch["times"] # t = np.array(t) t_threads.append(t) t_threads_all.append(arch["times_all"]) if t_threads != []: t_threads = np.array(t_threads) plt.plot( threads, t_threads[:, inh], f"-{markers[i]}", c=colors[i], label=f"{backend} {device}", ) times_all = np.array(t_threads_all)[:, inh] times_std = np.std(times_all, axis=1) plt.errorbar( threads, t_threads[:, inh], times_std, c=colors[i], capsize=1, ) i += 1 plt.xticks(threads) plt.legend(ncol=2) plt.yscale("log") # plt.xscale("log") plt.xlabel("number of threads") plt.ylabel("time (s)") plt.title(f"backends comparison {NH[inh]} harmonics") plt.tight_layout() .. rst-class:: sphx-glr-horizontal * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_011.png :alt: backends comparison 97 harmonics :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_011.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_012.png :alt: backends comparison 197 harmonics :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_012.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_013.png :alt: backends comparison 385 harmonics :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_013.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_014.png :alt: backends comparison 593 harmonics :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_014.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_015.png :alt: backends comparison 797 harmonics :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_015.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_016.png :alt: backends comparison 997 harmonics :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_016.png :class: sphx-glr-multi-img .. GENERATED FROM PYTHON SOURCE LINES 179-181 Speedup vs. number of threads ----------------------------- .. GENERATED FROM PYTHON SOURCE LINES 181-238 .. code-block:: Python for inh in range(len(NH)): plt.figure(figsize=figsize) i = 1 for backend in backends: for device in devices: speedup_threads = [] speedup_threads_all = [] for num_threads in threads: if device != "gpu" or backend == "torch": g = "cuda" if device == "gpu" else device arch = np.load( f"{num_threads}/benchmark_{backend}_{g}.npz", allow_pickle=True ) arch_np = np.load( f"{num_threads}/benchmark_numpy_cpu.npz", allow_pickle=True ) if backend != "numpy": t = arch["times"] speedup = np.array(arch_np["times"]) / np.array(arch["times"]) speedup_threads.append(speedup) speedup_all = np.array(arch_np["times_all"]) / np.array( arch["times_all"] ) speedup_threads_all.append(speedup_all) if speedup_threads != []: speedup_threads = np.array(speedup_threads) if backend != "numpy": plt.plot( threads, speedup_threads[:, inh], f"-{markers[i]}", c=colors[i], label=f"{backend} {device}", ) speedup_std = np.std(np.array(speedup_threads_all)[:, inh], axis=1) plt.errorbar( threads, speedup_threads[:, inh], speedup_std, c=colors[i], capsize=1, ) i += 1 plt.xticks(threads) # plt.ylim(0.25, 3.8) plt.legend(ncol=2) # plt.yscale("log") # plt.xscale("log") plt.xlabel("number of threads") plt.ylabel("speedup vs. numpy") plt.title(f"backends comparison {NH[inh]} harmonics") plt.tight_layout() .. rst-class:: sphx-glr-horizontal * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_017.png :alt: backends comparison 97 harmonics :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_017.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_018.png :alt: backends comparison 197 harmonics :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_018.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_019.png :alt: backends comparison 385 harmonics :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_019.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_020.png :alt: backends comparison 593 harmonics :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_020.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_021.png :alt: backends comparison 797 harmonics :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_021.png :class: sphx-glr-multi-img * .. image-sg:: /examples/benchmarks/images/sphx_glr_plot_benchmarks_022.png :alt: backends comparison 997 harmonics :srcset: /examples/benchmarks/images/sphx_glr_plot_benchmarks_022.png :class: sphx-glr-multi-img .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 5.271 seconds) **Estimated memory usage:** 520 MB .. _sphx_glr_download_examples_benchmarks_plot_benchmarks.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: binder-badge .. image:: images/binder_badge_logo.svg :target: https://mybinder.org/v2/gh/nannos/nannos.gitlab.io/doc?filepath=notebooks/examples/benchmarks/plot_benchmarks.ipynb :alt: Launch binder :width: 150 px .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_benchmarks.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_benchmarks.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_benchmarks.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_