Overview

This article describes the Benchmark tool, including its testing principles and some relatively complex test implementations.

A simple benchmark

A simple benchmark is really quite simple, for example

  1. [root@liqiang.io]# cat benchmark_test.go
  2. func BenchmarkTest(b *testing.B) {
  3. for i := 0; i < b.N; i++ {
  4. _ = add(1, 2)
  5. }
  6. }
  7. func add(a, b int) int {
  8. return a + b
  9. }

Then run it and see.

  1. [root@liqiang.io]# go test -test.bench ^BenchmarkTest$ -test.run ^$ .
  2. goos: darwin
  3. goarch: amd64
  4. pkg: xxxxxx
  5. cpu: xxxxxx
  6. BenchmarkTest-12 1000000000 0.2499 ns/op
  7. PASS
  8. ok xxxxxx 0.402s

As you can see, the line returned here is.

  1. BenchmarkTest-12 1000000000 0.2499 ns/op

The first column shows which function was benchmarked, the second column shows how many times the function was run (N in b.N), and then the last one is the result of the benchmark, where the time per call is 0.2499 ns, which is very fast. So can we see other benchmarks? For example, the time may vary depending on the cpu or how busy the machine is; in addition, very often, we look at whether a piece of code is fast or not on the one hand, and whether the memory occupation is high or not is also a very critical factor, and whether IO is frequent is also a point to consider (of course, IO may be reflected in the speed), so here we can add a memory-related analysis, I slightly modify Note that here I added one more running parameter: -benchmem

  1. [root@liqiang.io]# go test -test.bench ^BenchmarkTest$ -test.run ^$ . -benchmem
  2. goos: darwin
  3. goarch: amd64
  4. pkg: xxxxxx
  5. cpu: xxxxxx
  6. BenchmarkTest-12 100000000 10.30 ns/op 3 B/op 1 allocs/op
  7. PASS
  8. ok xxxxxx 2.034s

Since I modified the code, we ignore the time, and we can find two more columns in the result, which are how much memory was allocated for each operation (function called inside for) and how many times memory was requested for each operation.

Custom report metrics

For example, if you want to benchmark the complexity of the sorting algorithm that comes with it, then you can see how many times it was compared during the sorting process, so you can look at it like this.

  1. [root@liqiang.io]# cat sort_slice_test.go
  2. func BenchmarkTest(b *testing.B) {
  3. compare := 0
  4. for i := 0; i < b.N; i++ {
  5. sortSlice := []int{1, 2, 3, 4, 5}
  6. sort.Slice(sortSlice, func(i, j int) bool {
  7. compare++
  8. return sortSlice[i] < sortSlice[j]
  9. })
  10. }
  11. b.ReportMetric(float64(compare)/float64(b.N), "compare/op")
  12. }

Then run it to see the result.

  1. [root@liqiang.io]# go test -test.bench ^BenchmarkTest$ -test.run ^$ . -benchmem
  2. goos: darwin
  3. goarch: amd64
  4. pkg: xxxxxxx
  5. cpu: xxxxxxx
  6. BenchmarkTest-12 8820426 123.3 ns/op 4.000 compare/op 104 B/op 3 allocs/op
  7. PASS
  8. ok xxxxxxx 2.139s

Cross-Sectional Comparison

For example, if we write an SDK and want to compare the performance with other SDKs of the same type, we can run the benchmark manually several times, one at a time, and then compare the results of these multiple runs; for me, I would prefer to run the benchmark for multiple For me, I’d rather run the benchmark for multiple SDKs at once and then directly see the results of similar operations for multiple SDKs, so I can do this.

  1. [root@liqiang.io]# cat multi_bench_test.go
  2. func BenchmarkSDK1(b *testing.B) {
  3. for i := 0; i < b.N; i++ {
  4. SDK1.func()
  5. }
  6. }
  7. func BenchmarkSDK2(b *testing.B) {
  8. for i := 0; i < b.N; i++ {
  9. SDK2.func()
  10. }
  11. }

It seems to work, but the relationship is not that strong, I would prefer something like this operation.

  1. [root@liqiang.io]# cat multi_bench_test.go
  2. func BenchmarkFunc(b *testing.B) {
  3. for i := 0; i < b.N; i++ {
  4. SDK1.func()
  5. }
  6. for i := 0; i < b.N; i++ {
  7. SDK2.func()
  8. }
  9. }

However, if you write it that way directly, you will find that the result is not what you want, I give an example here.

  1. [root@liqiang.io]# cat multi_bench_test.go
  2. func BenchmarkAdd1(b *testing.B) {
  3. for i := 0; i < b.N; i++ {
  4. add(3, 5)
  5. }
  6. }
  7. func BenchmarkAdd2(b *testing.
  8. for i := 0; i < b.N; i++ {
  9. add2(3, 5)
  10. }
  11. }
  12. func BenchmarkTwoAdd(b *testing.
  13. for i := 0; i < b.N; i++ {
  14. add2(3, 5)
  15. }
  16. for i := 0; i < b.N; i++ {
  17. add(3, 5)
  18. }
  19. }

Run it, and the result looks like this, which actually adds up the two results.

  1. [root@liqiang.io]# go test -test.bench '. *' -test.run ^$ . -benchmem
  2. goos: darwin
  3. goarch: amd64
  4. pkg: xxxxxxxxx
  5. cpu: xxxxxxxxx
  6. BenchmarkAdd1-12 50087434 21.98 ns/op 10 B/op 2 allocs/op
  7. BenchmarkAdd2-12 100000000 10.84 ns/op 5 B/op 1 allocs/op
  8. BenchmarkTwoAdd-12 36373838 32.60 ns/op 16 B/op 3 allocs/op
  9. PASS
  10. ok xxxxxxxxx 3.579s

The correct way to write ####

The correct way to write it is as follows.

  1. [root@liqiang.io]# cat multi_bench_test.go
  2. func BenchmarkTwoAdd(b *testing.B) {
  3. var funcs = []func(a, b int) int{add, add2}
  4. for idx, f := range funcs {
  5. b.Run("function "+strconv.Itoa(idx), func(b *testing.B) {
  6. for i := 0; i < b.N; i++ {
  7. f(3, 5)
  8. }
  9. })
  10. }
  11. }

Then we run it to see the results.

  1. [root@liqiang.io]# go test -test.bench '^BenchmarkTwoAdd$' -test.run ^$ . -benchmem
  2. goos: darwin
  3. goarch: amd64
  4. pkg: xxxxxxxx
  5. cpu: xxxxxxxx
  6. BenchmarkTwoAdd/function_0-12 44406841 24.75 ns/op 10 B/op 2 allocs/op
  7. BenchmarkTwoAdd/function_1-12 90792690 12.98 ns/op 5 B/op 1 allocs/op
  8. PASS
  9. ok xxxxxxxx 3.325s

You can find that this is not two function calls added together, but two independent functions, but you can find that the runtime here is slightly more than running alone, but no harm done, we are concerned about the side-by-side comparison, and it is clear here.

Summary

OK, that’s the introduction and summary of Go’s own Benchmark, hope it helps you.

Ref