Skip to content

Commit

Permalink
Merge pull request #15 from janisz/refactor_tests
Browse files Browse the repository at this point in the history 
Refactor tests
  • Loading branch information
@bgalek
bgalek committed Mar 17, 2023
2 parents ab9a1f0 + af505a3 commit 08dc779
Show file tree
Hide file tree
Showing 4 changed files with 108 additions and 55 deletions.
77 changes: 54 additions & 23 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,40 +4,71 @@ Benchmarks for BigCache project

### GC pause time
```
go version
go version go1.13 linux/amd64
# go version
go version go1.20.2 linux/amd64
# go run caches_gc_overhead_comparison.go -cache bigcache
Cache: bigcache
Number of entries: 20000000
Number of repeats: 50
Value size: 100
GC pause for startup: 3.591222ms
GC pause for bigcache: 273.027743ms
# go run caches_gc_overhead_comparison.go -cache stdmap
Cache: stdmap
Number of entries: 20000000
Number of repeats: 50
GC pause for startup: 20.811µs
GC pause for warmup: 3.715446ms
GC pause for freecache: 38.607127ms
GC pause for bigcache: 159.066277ms
GC pause for map: 54.543926ms
Value size: 100
GC pause for startup: 2.893986ms
GC pause for stdmap: 111.621318ms
```

### Writes and reads
```
# go version
go version go1.13.5 linux/amd64
go version go1.20.2 linux/amd64
# go test -bench=. -benchmem -benchtime=4s ./... -timeout 30m
goos: linux
goarch: amd64
pkg: github.com/allegro/bigcache-bench
BenchmarkMapSet-8 12373646 392 ns/op 201 B/op 3 allocs/op
BenchmarkConcurrentMapSet-8 4108234 1389 ns/op 344 B/op 8 allocs/op
BenchmarkFreeCacheSet-8 8555763 783 ns/op 342 B/op 2 allocs/op
BenchmarkBigCacheSet-8 8892351 554 ns/op 308 B/op 2 allocs/op
BenchmarkMapGet-8 15402984 329 ns/op 24 B/op 1 allocs/op
BenchmarkConcurrentMapGet-8 12573116 496 ns/op 24 B/op 2 allocs/op
BenchmarkFreeCacheGet-8 8183350 781 ns/op 136 B/op 2 allocs/op
BenchmarkBigCacheGet-8 10523967 478 ns/op 152 B/op 4 allocs/op
BenchmarkBigCacheSetParallel-8 31305084 152 ns/op 319 B/op 3 allocs/op
BenchmarkFreeCacheSetParallel-8 20131963 266 ns/op 341 B/op 3 allocs/op
BenchmarkConcurrentMapSetParallel-8 18498877 280 ns/op 200 B/op 6 allocs/op
BenchmarkBigCacheGetParallel-8 52064035 101 ns/op 152 B/op 4 allocs/op
BenchmarkFreeCacheGetParallel-8 44531870 180 ns/op 136 B/op 3 allocs/op
BenchmarkConcurrentMapGetParallel-8 25893908 197 ns/op 24 B/op 2 allocs/op
cpu: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
BenchmarkMapSetForStruct-8 2523 1887529 ns/op 696961 B/op 19746 allocs/op
BenchmarkSyncMapSetForStruct-8 934 4957356 ns/op 1851193 B/op 69774 allocs/op
BenchmarkOracamanMapSetForStruct-8 1513 2870550 ns/op 1226702 B/op 20195 allocs/op
BenchmarkFreeCacheSetForStruct-8 688 6652622 ns/op 6983119 B/op 40287 allocs/op
BenchmarkBigCacheSetForStruct-8 760 5973545 ns/op 3772362 B/op 42204 allocs/op
BenchmarkMapSetForBytes-8 1708 2826723 ns/op 2096269 B/op 29749 allocs/op
BenchmarkSyncMapSetForBytes-8 814 5821909 ns/op 3133981 B/op 80034 allocs/op
BenchmarkOracamanMapSetForBytes-8 1372 3579288 ns/op 2976597 B/op 30276 allocs/op
BenchmarkFreeCacheSetForBytes-8 852 5575751 ns/op 7864114 B/op 30538 allocs/op
BenchmarkBigCacheSetForBytes-8 906 5206221 ns/op 4653427 B/op 32457 allocs/op
BenchmarkMapGetForStruct-8 23807222 201.1 ns/op 23 B/op 1 allocs/op
BenchmarkSyncMapGetForStruct-8 17040469 238.3 ns/op 23 B/op 1 allocs/op
BenchmarkOracamanMapGetForStruct-8 18912418 238.2 ns/op 23 B/op 1 allocs/op
BenchmarkFreeCacheGetForStruct-8 5201450 935.8 ns/op 295 B/op 9 allocs/op
BenchmarkBigCacheGetForStruct-8 5397729 875.0 ns/op 287 B/op 9 allocs/op
BenchmarkMapGetForBytes-8 23130621 207.2 ns/op 23 B/op 1 allocs/op
BenchmarkSyncMapGetForBytes-8 18846595 237.4 ns/op 23 B/op 1 allocs/op
BenchmarkOracamanMapGetForBytes-8 19344432 246.4 ns/op 23 B/op 1 allocs/op
BenchmarkFreeCacheGetForBytes-8 11845938 395.3 ns/op 159 B/op 3 allocs/op
BenchmarkBigCacheGetForBytes-8 12868870 346.8 ns/op 151 B/op 3 allocs/op
BenchmarkSyncMapSetParallelForStruct-8 6717847 694.8 ns/op 70 B/op 5 allocs/op
BenchmarkOracamanMapSetParallelForStruct-8 21306747 218.4 ns/op 31 B/op 2 allocs/op
BenchmarkFreeCacheSetParallelForStruct-8 18417992 266.4 ns/op 54 B/op 4 allocs/op
BenchmarkBigCacheSetParallelForStruct-8 17324250 290.5 ns/op 203 B/op 4 allocs/op
BenchmarkSyncMapSetParallelForBytes-8 6361482 773.0 ns/op 199 B/op 6 allocs/op
BenchmarkOracamanMapSetParallelForBytes-8 20362628 234.1 ns/op 139 B/op 3 allocs/op
BenchmarkFreeCacheSetParallelForBytes-8 19983202 237.1 ns/op 143 B/op 3 allocs/op
BenchmarkBigCacheSetParallelForBytes-8 17371614 280.4 ns/op 443 B/op 3 allocs/op
BenchmarkSyncMapGetParallelForStruct-8 24168936 193.5 ns/op 23 B/op 1 allocs/op
BenchmarkOracamanMapGetParallelForStruct-8 21529862 186.5 ns/op 23 B/op 1 allocs/op
BenchmarkFreeCacheGetParallelForStruct-8 14119288 350.2 ns/op 295 B/op 9 allocs/op
BenchmarkBigCacheGetParallelForStruct-8 13292924 368.0 ns/op 287 B/op 9 allocs/op
BenchmarkSyncMapGetParallelForBytes-8 25061154 188.6 ns/op 23 B/op 1 allocs/op
BenchmarkOracamanMapGetParallelForBytes-8 25262446 190.0 ns/op 23 B/op 1 allocs/op
BenchmarkFreeCacheGetParallelForBytes-8 20815525 232.5 ns/op 159 B/op 3 allocs/op
BenchmarkBigCacheGetParallelForBytes-8 21059220 220.7 ns/op 151 B/op 3 allocs/op
PASS
ok github.com/allegro/bigcache-bench 254.953s
ok github.com/allegro/bigcache-bench 183.467s
```
70 changes: 41 additions & 29 deletions caches_gc_overhead_comparison.go
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
package main

import (
"flag"
"fmt"
"os"
"runtime"
"runtime/debug"
"time"
Expand All @@ -21,48 +23,56 @@ func gcPause() time.Duration {
return pause
}

const (
entries = 20000000
valueSize = 100
repeat = 50
)

func main() {

c := ""
entries := 0
repeat := 0
valueSize := 0
flag.StringVar(&c, "cache", "bigcache", "cache to bench.")
flag.IntVar(&entries, "entries", 20000000, "number of entries to test")
flag.IntVar(&repeat, "repeat", 50, "number of repetitions")
flag.IntVar(&valueSize, "value-size", 100, "size of single entry value in bytes")
flag.Parse()

debug.SetGCPercent(10)
fmt.Println("Cache: ", c)
fmt.Println("Number of entries: ", entries)
fmt.Println("Number of repeats: ", repeat)
fmt.Println("Value size: ", valueSize)

var benchFunc func(entries, valueSize int)

switch c {
case "freecache":
benchFunc = freeCache
case "bigcache":
benchFunc = bigCache
case "stdmap":
benchFunc = stdMap
default:
fmt.Printf("unknown cache: %s", c)
os.Exit(1)
}

benchFunc(entries, valueSize)
fmt.Println("GC pause for startup: ", gcPause())

stdMap()
freeCache()
bigCache()

fmt.Println("GC pause for warmup: ", gcPause())

for i := 0; i < repeat; i++ {
freeCache()
}
fmt.Println("GC pause for freecache: ", gcPause())
for i := 0; i < repeat; i++ {
bigCache()
benchFunc(entries, valueSize)
}
fmt.Println("GC pause for bigcache: ", gcPause())
for i := 0; i < repeat; i++ {
stdMap()
}
fmt.Println("GC pause for map: ", gcPause())

fmt.Printf("GC pause for %s: %s\n", c, gcPause())
}

func stdMap() {
func stdMap(entries, valueSize int) {
mapCache := make(map[string][]byte)
for i := 0; i < entries; i++ {
key, val := generateKeyValue(i, valueSize)
mapCache[key] = val
}
}

func freeCache() {
func freeCache(entries, valueSize int) {
freeCache := freecache.NewCache(entries * 200) //allocate entries * 200 bytes
for i := 0; i < entries; i++ {
key, val := generateKeyValue(i, valueSize)
Expand All @@ -72,14 +82,15 @@ func freeCache() {
}

firstKey, _ := generateKeyValue(1, valueSize)
checkFirstElement(freeCache.Get([]byte(firstKey)))
v, err := freeCache.Get([]byte(firstKey))
checkFirstElement(valueSize, v, err)

if freeCache.OverwriteCount() != 0 {
fmt.Println("Overwritten: ", freeCache.OverwriteCount())
}
}

func bigCache() {
func bigCache(entries, valueSize int) {
config := bigcache.Config{
Shards: 256,
LifeWindow: 100 * time.Minute,
Expand All @@ -95,10 +106,11 @@ func bigCache() {
}

firstKey, _ := generateKeyValue(1, valueSize)
checkFirstElement(bigcache.Get(firstKey))
v, err := bigcache.Get(firstKey)
checkFirstElement(valueSize, v, err)
}

func checkFirstElement(val []byte, err error) {
func checkFirstElement(valueSize int, val []byte, err error) {
_, expectedVal := generateKeyValue(1, valueSize)
if err != nil {
fmt.Println("Error in get: ", err.Error())
Expand Down
9 changes: 6 additions & 3 deletions go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,12 @@ module github.com/allegro/bigcache-bench
go 1.19

require (
github.com/allegro/bigcache/v2 v2.1.3
github.com/coocood/freecache v1.1.0
github.com/allegro/bigcache/v2 v2.2.5
github.com/coocood/freecache v1.2.3
github.com/orcaman/concurrent-map/v2 v2.0.1
)

require github.com/cespare/xxhash v1.1.0 // indirect
require (
github.com/cespare/xxhash v1.1.0 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
)
7 changes: 7 additions & 0 deletions go.sum
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,17 @@ github.com/OneOfOne/xxhash v1.2.2 h1:KMrpdQIwFcEqXDklaen+P1axHaj9BSKzvpUUfnHldSE
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/allegro/bigcache/v2 v2.1.3 h1:8g6YBowSY3qw5+7OFy41EPNmM58slrMrX39Di41XE0M=
github.com/allegro/bigcache/v2 v2.1.3/go.mod h1:NE2LouLwhvFzJ55Wtx2x8uaOHfj4G19RRGPC8Kz3UUA=
github.com/allegro/bigcache/v2 v2.2.5 h1:mRc8r6GQjuJsmSKQNPsR5jQVXc8IJ1xsW5YXUYMLfqI=
github.com/allegro/bigcache/v2 v2.2.5/go.mod h1:FppZsIO+IZk7gCuj5FiIDHGygD9xvWQcqg1uIPMb6tY=
github.com/cespare/xxhash v1.1.0 h1:a6HrQnmkObjyL+Gs60czilIUGqrzKutQD6XZog3p+ko=
github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
github.com/cespare/xxhash/v2 v2.1.2/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/coocood/freecache v1.1.0 h1:ENiHOsWdj1BrrlPwblhbn4GdAsMymK3pZORJ+bJGAjA=
github.com/coocood/freecache v1.1.0/go.mod h1:ePwxCDzOYvARfHdr1pByNct1at3CoKnsipOHwKlNbzI=
github.com/coocood/freecache v1.2.3 h1:lcBwpZrwBZRZyLk/8EMyQVXRiFl663cCuMOrjCALeto=
github.com/coocood/freecache v1.2.3/go.mod h1:RBUWa/Cy+OHdfTGFEhEuE1pMCMX51Ncizj7rthiQ3vk=
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/orcaman/concurrent-map/v2 v2.0.1 h1:jOJ5Pg2w1oeB6PeDurIYf6k9PQ+aTITr/6lP/L/zp6c=
Expand Down

0 comments on commit 08dc779

Please sign in to comment.