Time and Profile Your Code

Inspired by: Diagnose slow Python code (YouTube)

Introduction

Timing and profiling you code (or program, or algorithm) can give you meaningful information about your code.

Sometimes your code runs too slow. Sometimes you need to make your code more efficient. Sometimes you need to know how well your code will scale to larger datasets.

Python has several modules that can be used to collect performance statistics. Two of which are shown in the code examples below.

Project #1

Time and/or profile one of your programs or algorithms. You should to run it long enough to to collect meaningful statistics.

For small programs, you might execute it 1,000 times in a loop. Be careful that your looping code is short enough so you are collecting data on your program and not the loop code.

Project #2

If your are analyzing how well your code scales, test various size datasets and plot the resulting statistics.

Time and Profile Code Examples

import time # ------------------------------------------------------------------- # ---- test code # ------------------------------------------------------------------- def test001(): ii = 0 for i in range(1000000): ii += i return ii def test002(): ff = 0.0 for i in range(1000000): ff += float(i) return ff def test_001_and_002(): test001() test002() # ------------------------------------------------------------------- # ---- main # -------------------------------------------------------------------- start = time.perf_counter() test_001_and_002() end = time.perf_counter() elapsed = end - start print(f'Completed in {elapsed:.2f} seconds')

import cProfile import pstats # ------------------------------------------------------------------- # ---- test code # ------------------------------------------------------------------- def test001(): ii = 0 for i in range(1000000): i += i return ii def test002(): ff = 0.0 for i in range(1000000): ff += float(i) return ff def test_001_and_002(): test001() test002() # ------------------------------------------------------------------- # ---- main # ------------------------------------------------------------------- with cProfile.Profile() as pr: test_001_and_002() stats = pstats.Stats(pr) stats.sort_stats(pstats.SortKey.TIME) stats.print_stats() ##stats.dump_stats(filename='needs_profiling.prof')

Create your own timing decorator