Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reduce pipelined DML RPC count in some case to optimize performance #51625

Closed
crazycs520 opened this issue Mar 8, 2024 · 0 comments · Fixed by #51652
Closed

reduce pipelined DML RPC count in some case to optimize performance #51625

crazycs520 opened this issue Mar 8, 2024 · 0 comments · Fixed by #51652

Comments

@crazycs520
Copy link
Contributor

Enhancement

#50389 introduce pipelined DML, we can enhancement the performance in some situation. See following test:

func TestPipelinedDMLInsertRPC(t *testing.T) {
	store := realtikvtest.CreateMockStoreAndSetup(t)
	tk := testkit.NewTestKit(t, store)
	tk.MustExec("use test")
	tk.MustExec("drop table if exists t1")
	tk.MustExec("create table t1 (a int, b int, unique index idx(b))")
	res := tk.MustQuery("explain analyze insert ignore into t1 values (1,1), (2,2), (3,3), (4,4), (5,5)")
	explain := getExplainResult(res)
	require.Regexp(t, "Insert.* check_insert: {total_time: .* rpc:{BatchGet:{num_rpc:1, total_time:.*}}}.*", explain)
	tk.MustExec("set session tidb_dml_type = bulk")
	res = tk.MustQuery("explain analyze insert ignore into t1 values (1,1), (2,2), (3,3), (4,4), (5,5)")
	explain = getExplainResult(res)
	// TODO: try to optimize the rpc count, when use bulk dml, insert ignore will use 5 BufferBatchGet and  1 BatchGet rpcs.
	// but without bulk dml, it will only use 1 BatchGet rpcs.
	require.Regexp(t, "Insert.* check_insert: {total_time: .* rpc:{.*BufferBatchGet:{num_rpc:5, total_time:.*}}}.*", explain)
	require.Regexp(t, "Insert.* check_insert: {total_time: .* rpc:{.*BatchGet:{num_rpc:1, total_time:.*}}}.*", explain)
	tk.MustExec("delete from t1")
}

func getExplainResult(res *testkit.Result) string {
	resBuff := bytes.NewBufferString("")
	for _, row := range res.Rows() {
		_, _ = fmt.Fprintf(resBuff, "%s\t", row)
	}
	return resBuff.String()
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant