Any plan to support Llama 405B (Llama 3.1)

[yanjunplay]

I've been waiting for a while, haha..

Today's the Llama3.1 405B model is officially announced https://llama.meta.com/. Do we have any plan to support this soon? I assume we need to adjust Eagle model accordingly.

Thanks a lot!!

[hongyanz]

As an academic group, we don't have enough GPUs to proceed the inference/training on LLama 405B (which should require 8x H100s for inference). If any industry would like to support us, please do let us know. Thanks.

[yanjunplay]

Thanks @hongyanz for the reply. Let me do some study and see how we can collaborate.

[anilmurty]

hey @hongyanz - I'd be interested in seeing if we can support you training 405B on https://akash.network/. We're able to run Llama-3.1 405B for inference. Are you interested in discussing this?