Add llm endpoint creation and inference sample code to self hosting d…

[ruizehung-scale]

Add llm endpoint creation and inference sample code to self hosting doc play with it section.

Related Issue

#141

Test Plan

(launch) ➜  devbox ~ curl -X POST 'http://localhost:5000/v1/llm/model-endpoints' \
    -H 'Content-Type: application/json' \
    -d '{
        "name": "llama-7b",
        "model_name": "llama-7b",
        "source": "hugging_face",
        "inference_framework": "text_generation_inference",
        "inference_framework_image_tag": "0.9.3",
        "num_shards": 4,
        "endpoint_type": "streaming",
        "cpus": 32,
        "gpus": 4,
        "memory": "40Gi",
        "storage": "40Gi",
        "gpu_type": "nvidia-ampere-a10",
        "min_workers": 1,
        "max_workers": 12,
        "per_worker": 1,
        "labels": {},
        "metadata": {}
    }' \
    -u test_user_id:
{"endpoint_creation_task_id":"8d323344-b1b5-497d-a851-6d6284d2f8e4"}%                                                                               

(launch) ➜  devbox ~ curl -X POST 'http://localhost:5000/v1/llm/completions-sync?model_endpoint_name=llama-7b' \
    -H 'Content-Type: application/json' \
    -d '{
        "prompts": ["Tell me a joke about AI"],
        "max_new_tokens": 30,
        "temperature": 0.1
    }' \
    -u test-user-id:
{"status":"SUCCESS","outputs":[{"text":". Tell me a joke about AI. Tell me a joke about AI. Tell me a joke about AI. Tell me","num_completion_tokens":30}],"traceback":null}%                                                                                                                          

[yixu34]

Hmm I think we'll want a separate guide that explains the server abstractions (maybe outside the scope of this PR), so that people understand what name vs model_name mean.

Also, I think we want docs per field, but that's more of an API reference, which doesn't need to go here.

[yixu34]

subtle nit around causality: Is seeing READY an indicator that the endpoint is ready? In other words, I might reword this to "You can tell that it's ready by listing pods and..."

Though alternatively, we may want to curl a status endpoint, to keep things at the API level.

[ruizehung-scale]

The status in model endpoint record is independent from the status of the pods .... Maybe at some point we might want to add an api that checks the endpoint pod status?

[seanshi-scale]

We do have something that returns the number of available/unavailable workers. Not sure if this exposed on the llm endpoint get/list routes though

[yixu34]

Oh right nm.

[yixu34]

Hmm @seanshi-scale @yunfeng-scale I think status and traceback should be gone?

[yixu34]

Oh oops they're still there in this repo. Will create a separate issue: #159