Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CTEExec should not be deadlocked with defer function after panic action triggered #49096

Closed
AilinKid opened this issue Dec 1, 2023 · 0 comments · Fixed by #49192
Closed

CTEExec should not be deadlocked with defer function after panic action triggered #49096

AilinKid opened this issue Dec 1, 2023 · 0 comments · Fixed by #49192
Assignees
@AilinKid
Labels
affects-5.4 This bug affects 5.4.x versions. affects-6.1 affects-6.5 affects-7.1 affects-7.5 severity/major sig/execution SIG execution type/bug This issue is a bug.

Comments

@AilinKid
Copy link
Contributor

AilinKid commented Dec 1, 2023

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

sync.runtime_SemacquireMutex(0x0?, 0x25?, 0xc0bfd74ca8?)
	/usr/local/go/src/runtime/sema.go:77 +0x25
sync.(*Mutex).lockSlow(0xc12cf36108)
	/usr/local/go/src/sync/mutex.go:171 +0x165
sync.(*Mutex).Lock(...)
	/usr/local/go/src/sync/mutex.go:90
github.com/pingcap/tidb/util/cteutil.(*StorageRC).Lock(0x0?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/cteutil/storage.go:211 +0x33
github.com/pingcap/tidb/executor.resetCTEStorageMap({0x55508f8, 0xc06d4b2a00})
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:1459 +0x159
github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelay.func1()
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:751 +0x5d
panic({0x4581a60, 0xc66ac508c0})
	/usr/local/go/src/runtime/panic.go:884 +0x212
github.com/pingcap/tidb/util/memory.(*PanicOnExceed).Action(0xc158421040, 0xc11ca39ae0)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/memory/action.go:148 +0x11f
github.com/pingcap/tidb/util/memory.(*Tracker).Consume.func2(0x46d8260?, 0xc000713f80?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/memory/tracker.go:452 +0xff
github.com/pingcap/tidb/util/memory.(*Tracker).Consume(0x46d8080?, 0x448b2812c)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/memory/tracker.go:464 +0x2c2
github.com/pingcap/tidb/util/memory.(*Tracker).remove(0xc0465905a0, 0xc125a28500)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/memory/tracker.go:355 +0x105
github.com/pingcap/tidb/util/memory.(*Tracker).Detach(0xc125a28500)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/memory/tracker.go:327 +0x1b6
github.com/pingcap/tidb/executor.(*hashRowContainer).Close(0xc001371600?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/hash_table.go:506 +0x9e
github.com/pingcap/tidb/parser/terror.Call(0xc0d801ba40?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/parser/terror/terror.go:306 +0x31
github.com/pingcap/tidb/executor.(*HashJoinExec).Close(0xc0923a2d00)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/join.go:174 +0x265
github.com/pingcap/tidb/executor.(*baseExecutor).Close(0xc06e6bcee8?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:214 +0x7c
github.com/pingcap/tidb/executor.(*ProjectionExec).Close(0xc07e332ea0)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/projection.go:334 +0x1d4
github.com/pingcap/tidb/executor.(*ShuffleExec).Close(0xc185c93420)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/shuffle.go:190 +0x35b
github.com/pingcap/tidb/executor.(*baseExecutor).Close(0x54e0e50?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:214 +0x7c
github.com/pingcap/tidb/executor.(*SelectionExec).Close(0xc185c935e0?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:1572 +0x65
github.com/pingcap/tidb/executor.(*ShuffleExec).Close(0xc185c93340)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/shuffle.go:190 +0x35b
github.com/pingcap/tidb/executor.(*baseExecutor).Close(0x18?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:214 +0x7c
github.com/pingcap/tidb/executor.(*SelectionExec).Close(0xc185c937a0?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:1572 +0x65
github.com/pingcap/tidb/executor.(*baseExecutor).Close(0xc06e6bcee8?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:214 +0x7c
github.com/pingcap/tidb/executor.(*ProjectionExec).Close(0xc07e333c20)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/projection.go:334 +0x1d4
github.com/pingcap/tidb/executor.(*cteProducer).closeProducer(0xc0190ff100)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/cte.go:218 +0x2e
github.com/pingcap/tidb/executor.(*CTEExec).Close(0xc13d0df4a0)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/cte.go:119 +0x52
github.com/pingcap/tidb/executor.(*baseExecutor).Close(0x18?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:214 +0x7c
github.com/pingcap/tidb/executor.(*SelectionExec).Close(0xc185c93880?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:1572 +0x65
github.com/pingcap/tidb/executor.(*baseExecutor).Close(0xc06e6bcee8?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:214 +0x7c
github.com/pingcap/tidb/executor.(*ProjectionExec).Close(0xc0f1326240)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/projection.go:334 +0x1d4
github.com/pingcap/tidb/executor.(*baseExecutor).Close(0xc0913c3f20?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:214 +0x7c
github.com/pingcap/tidb/executor.(*HashAggExec).Close(0xc1624f9b00)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/aggregate.go:292 +0x2e5
github.com/pingcap/tidb/executor.(*baseExecutor).Close(0xc06e6bcee8?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:214 +0x7c
github.com/pingcap/tidb/executor.(*ProjectionExec).Close(0xc0f1326480)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/projection.go:334 +0x1d4
github.com/pingcap/tidb/executor.(*baseExecutor).Close(0xc0ad1f7aa0?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:214 +0x7c
github.com/pingcap/tidb/executor.(*HashJoinExec).Close(0xc0923a3520)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/join.go:194 +0x4da
github.com/pingcap/tidb/executor.(*baseExecutor).Close(0xc0928aa000?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:214 +0x7c
github.com/pingcap/tidb/executor.(*HashAggExec).Close(0xc10b278000)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/aggregate.go:292 +0x2e5
github.com/pingcap/tidb/executor.(*baseExecutor).Close(0xc06e6bcee8?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:214 +0x7c
github.com/pingcap/tidb/executor.(*ProjectionExec).Close(0xc0f1327b00)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/projection.go:334 +0x1d4
github.com/pingcap/tidb/executor.(*UnionExec).Close(0xc1471f2480)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:1936 +0x122
github.com/pingcap/tidb/executor.(*baseExecutor).Close(0xc08f63d5c0?)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:214 +0x7c
github.com/pingcap/tidb/executor.(*HashAggExec).Close(0xc10b2786c0)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/aggregate.go:292 +0x2e5
github.com/pingcap/tidb/executor.(*ReplaceExec).Close(0xc1cd83c530)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/replace.go:51 +0x136
github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelayExecutor.func1()
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:909 +0x2b
github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelayExecutor(0xc076e92e10, {0x54dfe90, 0xc03a63c450}, {0x54e4c20?, 0xc1cd83c530})
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:929 +0x60e
github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelay(0xc076e92e10, {0x54dfe90, 0xc03a63c450}, {0x54e4c20?, 0xc1cd83c530?}, 0x0)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:776 +0x26d
github.com/pingcap/tidb/executor.(*ExecStmt).Exec(0xc076e92e10, {0x54dfe90, 0xc03a63c450})
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:571 +0xbe5
github.com/pingcap/tidb/session.runStmt({0x54dfe90, 0xc15d68c930}, 0xc06d4b2a00, {0x54f1d60, 0xc076e92e10?})
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/session/session.go:2373 +0x565
github.com/pingcap/tidb/session.(*session).ExecuteStmt(0xc06d4b2a00, {0x54dfe90, 0xc15d68c750}, {0x54f9330?, 0xc0ccc02600})
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/session/session.go:2237 +0xab8
github.com/pingcap/tidb/server.(*TiDBContext).ExecuteStmt(0xc06e6bcd20, {0x54dfe90, 0xc15d68c750}, {0x54f9330?, 0xc0ccc02600})
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/driver_tidb.go:252 +0xaf
github.com/pingcap/tidb/server.(*clientConn).handleStmt(0xc06e386180, {0x54dfde8, 0xc0adf786c0}, {0x54f9330, 0xc0ccc02600}, {0x7e04638, 0x0, 0x0}, 0x1)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:2122 +0x167
github.com/pingcap/tidb/server.(*clientConn).handleQuery(0xc06e386180, {0x54dfde8, 0xc0adf786c0}, {0xc12e792801, 0x1606})
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1972 +0x93d
github.com/pingcap/tidb/server.(*clientConn).dispatch(0xc06e386180, {0x54dfe90?, 0xc05bb17500?}, {0xc12e792800, 0x1607, 0x1607})
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1403 +0x1005
github.com/pingcap/tidb/server.(*clientConn).Run(0xc06e386180, {0x54dfe90, 0xc05bb17500})
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1152 +0x24e
github.com/pingcap/tidb/server.(*Server).onConn(0xc002dcf8c0, 0xc06e386180)
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/server.go:648 +0xac7
created by github.com/pingcap/tidb/server.(*Server).startNetworkListener
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/server.go:483 +0x72c

2. What did you expect to see? (Required)

closeCTE should be conflict with defer function after panic triggered

3. What did you see instead (Required)

dead lock

4. What is your TiDB version? (Required)

master
@AilinKid AilinKid added type/bug This issue is a bug. sig/execution SIG execution labels Dec 1, 2023
@AilinKid AilinKid mentioned this issue Dec 5, 2023
Merged
13 tasks
@AilinKid AilinKid self-assigned this Dec 5, 2023
ti-chi-bot bot pushed a commit that referenced this issue Dec 5, 2023
@AilinKid
executor: fix deadlock in dml statement with cte when oom panic actio…
0c7659c 
…n was triggered (#49192)

close #49096
This was referenced Dec 5, 2023
Closed
Closed
Merged
Merged
Merged
ti-chi-bot bot pushed a commit that referenced this issue Dec 6, 2023
@ti-chi-bot
executor: fix deadlock in dml statement with cte when oom panic actio…
003d0d4 
…n was triggered (#49192) (#49202)

close #49096
ti-chi-bot bot pushed a commit that referenced this issue Dec 8, 2023
@ti-chi-bot
executor: fix deadlock in dml statement with cte when oom panic actio…
bb5be54 
…n was triggered (#49192) (#49201)

close #49096
ti-chi-bot bot pushed a commit that referenced this issue Dec 20, 2023
@ti-chi-bot
executor: fix deadlock in dml statement with cte when oom panic actio…
5d1665d 
…n was triggered (#49192) (#49203)

close #49096
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@AilinKid AilinKid

Labels
affects-5.4 This bug affects 5.4.x versions. affects-6.1 affects-6.5 affects-7.1 affects-7.5 severity/major sig/execution SIG execution type/bug This issue is a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

executor: fix deadlock in dml statement with cte when oom panic action was triggered AilinKid/tidb
2 participants
@XuHuaiyu @AilinKid