inanzzz | Using golang bench, benchstat and benchcmp to measure performance

In golang applications, you can use bench command to run benchmark tests and see performance results. You can also use benchstat or benchcmp commands to compare past results to new ones. In example below, we are going to use two different functions to validate yyy-mm-dd format and write benchmark tests for both. I suggest you to read a very valuable High Performance Go Workshop post.

Example commands

// All benchmarks in the project
go test -v -bench=. -benchmem ./...
go test -v -bench=. ./... -benchmem

// All benchmarks in the package
go test -v -bench=. -benchmem ./internal/validator/

Code

// internal/validator/datetime.go

package validator

import (
	"regexp"
	"time"
)

// DateRegex validates date format `yyyy-mm-dd`.
func DateRegex(v string) bool {
	return regexp.MustCompile(`([12]\d{3}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01]))`).MatchString(v)
}

// TimeParse validates date format `yyyy-mm-dd`.
func TimeParse(v string) bool {
	if _ , e := time.Parse("2006-01-02", v); e != nil {
		return false
	}
	return true
}

Test with single input

// internal/validator/datetime_test.go

package validator

import (
	"testing"
)

func BenchmarkDateRegex(b *testing.B) {
	for i := 0; i < b.N; i++ {
		_ = DateRegex("2020-01-01")
	}
}

func BenchmarkTimeParse(b *testing.B) {
	for i := 0; i < b.N; i++ {
		_ = TimeParse("2020-01-01")
	}
}

Run

$ go test -v -bench=. -benchmem ./internal/validator/

BenchmarkDateRegex-4   	  133645	      8860 ns/op	    7047 B/op	      62 allocs/op
BenchmarkTimeParse-4   	 7844908	       150 ns/op	       0 B/op	       0 allocs/op

Test with various inputs

// internal/validator/datetime_test.go

package validator

import (
	"testing"
)

func benchmarkDateRegex(v string, b *testing.B) {
	for i := 0; i < b.N; i++ {
		_ = DateRegex(v)
	}
}

func BenchmarkDateRegex_1(b *testing.B) { benchmarkDateRegex("", b) }
func BenchmarkDateRegex_2(b *testing.B) { benchmarkDateRegex("     ", b) }
func BenchmarkDateRegex_3(b *testing.B) { benchmarkDateRegex("2020-01-01", b) }

Run

$ go test -v -bench=. -benchmem ./internal/validator/

BenchmarkDateRegex_1-4   	  122085	      8483 ns/op	    7032 B/op	      62 allocs/op
BenchmarkDateRegex_2-4   	  132746	      8445 ns/op	    7032 B/op	      62 allocs/op
BenchmarkDateRegex_3-4   	  132991	      8802 ns/op	    7048 B/op	      62 allocs/op

Benchmark measurement

Here we are comparing past results to current results. Make sure you run go get golang.org/x/perf/cmd/benchstat and go get golang.org/x/tools/cmd/benchcmp to install packages.

$ go test -v -bench=. -benchmem ./internal/validator/ > old.out

// Do some changes to your code

$ go test -v -bench=. -benchmem ./internal/validator/ > new.out

Measure

$ benchstat old.out new.out

name         old time/op    new time/op    delta
DateRegex-4    8.98µs ± 0%    8.75µs ± 0%   ~     (p=1.000 n=1+1)
TimeParse-4     151ns ± 0%     152ns ± 0%   ~     (p=1.000 n=1+1)

name         old alloc/op   new alloc/op   delta
DateRegex-4    7.05kB ± 0%    7.05kB ± 0%   ~     (p=1.000 n=1+1)
TimeParse-4     0.00B          0.00B        ~     (all equal)

name         old allocs/op  new allocs/op  delta
DateRegex-4      62.0 ± 0%      62.0 ± 0%   ~     (all equal)
TimeParse-4      0.00           0.00        ~     (all equal)

$ benchcmp old.out new.out

benchmark                old ns/op     new ns/op     delta
BenchmarkDateRegex-4     8978          8752          -2.52%
BenchmarkTimeParse-4     151           152           +0.66%

benchmark                old allocs     new allocs     delta
BenchmarkDateRegex-4     62             62             +0.00%
BenchmarkTimeParse-4     0              0              +0.00%

benchmark                old bytes     new bytes     delta
BenchmarkDateRegex-4     7052          7046          -0.09%
BenchmarkTimeParse-4     0             0             +0.00%