panic: runtime error: slice bounds out of range

[stratosgear]

I suddenly start getting a:

❯ cliphist list
panic: runtime error: slice bounds out of range [::3771442824] with length 2147483647

goroutine 1 [running]:
go.etcd.io/bbolt.unsafeByteSlice(...)
	go.etcd.io/[email protected]/unsafe.go:27
go.etcd.io/bbolt.(*leafPageElement).value(...)
	go.etcd.io/[email protected]/page.go:143
go.etcd.io/bbolt.(*Cursor).keyValue(0xc000069e50?)
	go.etcd.io/[email protected]/cursor.go:374 +0x172
go.etcd.io/bbolt.(*Cursor).Last(0xc000069e50)
	go.etcd.io/[email protected]/cursor.go:82 +0x18a
main.list(0x1?, {0x55cbb70a9cf8, 0xc000012018})
	go.senan.xyz/cliphist/cliphist.go:226 +0x105
main.glob..func2({0x55cbb7094d40?, 0xc0001101b0?, 0x7ffdff9c008c?})
	go.senan.xyz/cliphist/cliphist.go:73 +0xc5
main.main()
	go.senan.xyz/cliphist/cliphist.go:42 +0x149

that I do not know how to debug.

I thought some items in the history might trigger it, but trying to wipe all fails again:

❯ cliphist wipe
panic: runtime error: slice bounds out of range [::3771442824] with length 2147483647

goroutine 1 [running]:
go.etcd.io/bbolt.unsafeByteSlice(...)
	go.etcd.io/[email protected]/unsafe.go:27
go.etcd.io/bbolt.(*leafPageElement).value(...)
	go.etcd.io/[email protected]/page.go:143
go.etcd.io/bbolt.(*node).read(0xc001579d50, 0x7fbe4dbe6000)
	go.etcd.io/[email protected]/node.go:172 +0x374
go.etcd.io/bbolt.(*Bucket).node(0xc000112140, 0x8?, 0xc001578000)
	go.etcd.io/[email protected]/bucket.go:680 +0x1a5
go.etcd.io/bbolt.(*node).childAt(0xc001578000?, 0x1?)
	go.etcd.io/[email protected]/node.go:77 +0x45
go.etcd.io/bbolt.(*Cursor).node(0xc000104db0?)
	go.etcd.io/[email protected]/cursor.go:393 +0xc7
go.etcd.io/bbolt.(*Bucket).Delete(0xc000112140, {0x7fbe4dbe6050, 0x8, 0x8})
	go.etcd.io/[email protected]/bucket.go:332 +0xf5
main.wipe(0x0?)
	go.senan.xyz/cliphist/cliphist.go:318 +0x165
main.glob..func6({0x55b3bc999d40?, 0xc0001081b0?, 0x7ffc3684608c?})
	go.senan.xyz/cliphist/cliphist.go:132 +0xb4
main.main()
	go.senan.xyz/cliphist/cliphist.go:42 +0x149

So I am stuck.

Running under latest version of hyprland on latest arch. Not sure what other dbug info you might need.

Thanks!

[sentriz]

wow haven't seen that one before. how big is your database?

$ ls -lh $XDG_CACHE_HOME/cliphist/db

looks like you might be running into etcd-io/bbolt#576

[stratosgear]

Whaaaat?

➜  pwd
/home/stratos/.cache/cliphist
➜  ls -alh
total 6.9G
drwx------ 1 stratos stratos    4 Oct 28 16:08 .
drwxr-xr-x 1 stratos stratos 2.3K Nov  9 12:04 ..
-rw-r--r-- 1 stratos stratos 6.9G Nov  9 12:32 db

I have only installed it last week! No idea what is inside there!

[stratosgear]

I renamed the file to db.off and cliphist started working again.

Still worth investigating what really made it grew that much, and maybe provide a better hint for future users (pretty much what they discuss in the link above)

[sentriz]

wow that's big. are you storing videos as well as images in there? couldn't be just text

[sentriz]

ill track the upstream issues for the panic. if you see the db growing again, you could try $ cliphist list | grep binary data and see what the file sizes are. that may be a hint

[sentriz]

my 12 week old db is 54 mb for reference 😁

[stratosgear]

I might have copied an occasional image or two, but not videos...

Hmmm, I could also have done a couple of copy and paste on files to another folder with pcmanfm. Are these copied in the db too...? That's weird!

It would be interesting if you could whip up a script that tries to print the contents of the db (although I believe the underlying db library ALSO fails to read in there...)

[stratosgear]

Well in regards to cliphist list | grep binary data. It is TOO late now. I cannot run it on the huge db, as it fails.

[sentriz]

Hmmm, I could also have done a couple of copy and paste on files to another folder with pcmanfm. Are these copied in the db too...? That's weird!

hmm im not sure how pcmanfm interacts with the clipboard, never tried it

It would be interesting if you could whip up a script that tries to print the contents of the db (although I believe the underlying db library ALSO fails to read in there...)

yeah unfortunately it looks like the database is unusable (maybe corrupt?) until upstream is fixed

---

also i just added a check to store to just ignore anything coming in >5MB

[stratosgear]

I did some tests now with pcmanfm.

After copying a binary file, all I see in the clipboard history if the filename only as file:///tmp/somefile and the db size does NOT increase by the size of the file. I'll keep an eye out!. Is the 7GB file of any use to you? I would like to delete it!

[sentriz]

i don't wanna take 7GB of potentially very personal information 😁 i would just delete it. and it probably wont be an issue going forward with the 5MB check

[stratosgear]

Ok will do.

Haven't checked, but how big is an image copy from a large 4K monitor? I am only running two monitors of 1920x1080 and another at 2560x1080, but I would hate to lose an image copy from a full screen resolution...

Maybe 5MB is too conservative? some test might be in order...

[sentriz]

ah i see. can you take a screenshot to clipboard, then $ wl-paste >test; ls -lh test and see how big?

[stratosgear]

Well it was slightly more involving :)

➜  wl-paste > f.img
Clipboard content is not available as inferred output type "application/octet-stream"
Use "wl-paste --list-types" to view available types. Use "--type" to explicitly specify a type.

stratos@iocaine 
/tmp 
❯ wl-paste --list-types
image/png

stratos@iocaine 
/tmp 
➜  wl-paste --type image/png > img.img

stratos@iocaine 
/tmp 
➜  ls -al img.img
-rw-r--r-- 1 stratos stratos 264086 Nov  9 13:12 img.img

The copied image was pulled from grim and slurp and it is very close to the max resolution of my 2560x1080 monitor...

And while I have you here, may I abuse this thread to ask if you are aware of any scripts/utils that might allow me to preview what images I might have copied in cliphist...?

[sentriz]

-rw-r--r-- 1 stratos stratos 264086 Nov 9 13:12 img.img

okay 264086 bytes is 264.09 kB so that should be fine

And while I have you here, may I abuse this thread to ask if you are aware of any scripts/utils that might allow me to preview what images I might have copied in cliphist...?

i believe there is something like that in #61 from @chmouel 👀

[chmouel]

yes! let me clean this up and submit a PR!

[sentriz]

we also have an rofi image script from @Sonico98 too 👍

[stratosgear]

Tried that script but it failed on me! Opened a separate issue #71

[stratosgear]

@chmouel I believe your gist has a typo somewhere no...? I get a:

line 33: unexpected EOF while looking for matching `"'

Is there any updated version somewhere... :)

[chmouel]

@stratosgear hey sorry I had some issues to run cliphist to capture the image (I am not sure what is the issue, it doesn't capture when running as exec in sway but it does when I run cliphist from the command line)

This script works fine with me, their may be some problem with quoting unfortunately, i don't think there is much way to avoid quoting issues while using shell script:

#!/usr/bin/env bash
TMP=$(mktemp /tmp/.mm.XXXXXX)
clean() {
	ueberzugpp cmd -s "$SOCKET" -a exit
	rm -f $TMP
	pkill -f kitty.*fzcliphist
}
trap clean EXIT

query=
[[ -n ${1-""} ]] && query="--query=$*"

case "$(uname -a)" in
*Darwin*) UEBERZUG_TMP_DIR="$TMPDIR" ;;
*) UEBERZUG_TMP_DIR="/tmp" ;;
esac

UB_PID_FILE="$UEBERZUG_TMP_DIR/.$(uuidgen)"
ueberzugpp layer --no-stdin --silent --use-escape-codes --pid-file "$UB_PID_FILE"
UB_PID=$(cat "$UB_PID_FILE")

export SOCKET="$UEBERZUG_TMP_DIR"/ueberzugpp-"$UB_PID".socket
export X=$(($(tput cols) / 2 + 1))

# --max-width $FZF_PREVIEW_COLUMNS --max-height $FZF_PREVIEW_LINES
# if FZF_PREVIEW_COLUMNS exists, use it, otherwise use tput cols
[ -z "$FZF_PREVIEW_COLUMNS" ] && max_width=$(($(tput cols) - 2)) || max_width=$FZF_PREVIEW_COLUMNS
[ -z "$FZF_PREVIEW_LINES" ] && max_height=$(($(tput lines) - 2)) || max_height=$FZF_PREVIEW_LINES

rm -f /tmp/.copied
# run fzf with preview
output=$(cliphist list | fzf ${query} --with-nth=2.. --reverse --preview="echo {}|cliphist decode > $TMP;[[ {} == *'binary data image/png' ]] && { ueberzugpp cmd -s $SOCKET -i fzfpreview -a add -x $X -y 1 --max-width $max_width --max-height $max_height -f $TMP ;} || bat ${TMP} ")
[[ -n "$output" ]] && {
	echo "$output" | cliphist decode | wl-copy -n
	touch /tmp/.copied
}

(ah yeah another problem which i didn't see before but just appeared for me on kitty latest version with ueberzugpp is that it doesn't clear the screen after showing the image)