Is there a way to download models without engaging GPU memory?

[murtaza-nasir]

For example, I'd like to download this, but right now it gives an error, perhaps because this requires the safetensors file to be joined manually. Is there any way to download models externally and add them to the model download directory?

[AlpinDale]

Yes, you may download the model with huggingface_hub first.

pip install huggingface_hub hf_transfer
export HF_HUB_ENABLE_HF_TRANSFER=1
huggingface-cli download LoneStriker/MiquMaid-v2-2x70B-DPO-GPTQ --local-dir-use-symlinks False --local-dir ./MiquMaid-v2-2x70B-DPO-GPTQ

You can also use the --cache-dir flag to specify the cache directory, if ~/.cache doesn't have enough space to accommodate each model chunk.

[murtaza-nasir]

Thank you, will try this!