Panic: concurrent map writes of ParallelNestedLoopApplyExec's SimpleLRUCache

[yibin87]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

From customer case, not sure how to reproduce it, just the panic stack
fatal error: concurrent map writes

goroutine 349945042302 [running]:
github.com/pingcap/tidb/util/kvcache.(*SimpleLRUCache).Put(0xc6eb5946c0, {0x5d72e00, 0xc82454dab8}, {0x52a62a0?, 0xc1d757f730})
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/kvcache/simple_lru.go:114 +0x396
github.com/pingcap/tidb/executor.(*applyCache).put(0xcabeb98aa0, {0xc1c8be0060?, 0xb12eb0f14f0b7?, 0x8a12c80?}, {0x52a62a0, 0xc1d757f730})
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/apply_cache.go:66 +0xe5
github.com/pingcap/tidb/executor.(*applyCache).Set(0xcabeb98aa0, {0xc1c8be0060, 0x2e, 0x2e}, 0xc1d757f730)
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/apply_cache.go:93 +0x132
github.com/pingcap/tidb/executor.(*ParallelNestedLoopApplyExec).fetchAllInners(0xc617193d40, {0x5d98390, 0xc60643b440}, 0x9)
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/parallel_apply.go:338 +0xa8f
github.com/pingcap/tidb/executor.(*ParallelNestedLoopApplyExec).fillInnerChunk(0xc617193d40, {0x5d98390, 0xc60643b440}, 0x9, 0xc3bde2ebe0)
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/parallel_apply.go:384 +0x32f
github.com/pingcap/tidb/executor.(*ParallelNestedLoopApplyExec).innerWorker(0xc617193d40, {0x5d98390, 0xc60643b440}, 0xc713dd9e60?)
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/parallel_apply.go:247 +0x1ba
created by github.com/pingcap/tidb/executor.(*ParallelNestedLoopApplyExec).Next
/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/parallel_apply.go:146 +0x328
tidb_stderr.log

4. What is your TiDB version? (Required)

v7.1.1

[SeaRise]

/cc @SeaRise

[SeaRise]

/assign @SeaRise

[SeaRise]

tidb/pkg/executor/internal/applycache/apply_cache.go

Line 87 in 17d65b9

evictedKey, evictedValue, evicted := c.cache.RemoveOldest()

seems lock should be added here.

[SeaRise]

/sig compute

[ti-chi-bot]

@SeaRise: The label(s) sig/compute cannot be applied, because the repository doesn't have them.

In response to this:

/sig compute

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[SeaRise]

/sig execution

[SeaRise]

/label affects-7.5

[SeaRise]

/label affects-7.1

[SeaRise]

/label affects-6.5

[SeaRise]

/label affects-6.1

[SeaRise]

/label affects-5.4

[SeaRise]

/label affects-5.3

[ti-chi-bot]

@SeaRise: The label(s) affects-5.3 cannot be applied. These labels are supported: fuzz/sqlancer, challenge-program, compatibility-breaker, first-time-contributor, contribution, good first issue, correctness, duplicate, proposal, security, ok-to-test, needs-ok-to-test, needs-more-info, needs-cherry-pick-release-5.4, needs-cherry-pick-release-6.1, needs-cherry-pick-release-6.5, needs-cherry-pick-release-7.1, needs-cherry-pick-release-7.5, needs-cherry-pick-release-7.6, affects-5.4, affects-6.1, affects-6.5, affects-7.1, affects-7.5, affects-7.6, may-affects-5.4, may-affects-6.1, may-affects-6.5, may-affects-7.1, may-affects-7.5, may-affects-7.6.

In response to this:

/label affects-5.3

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[SeaRise]

/severity major