-
Notifications
You must be signed in to change notification settings - Fork 43
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: Add Deployments for E2E Tests (#60)
- Loading branch information
1 parent
64daef4
commit 4801723
Showing
36 changed files
with
460 additions
and
267 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
File renamed without changes.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
14 changes: 14 additions & 0 deletions
14
presets/k8s/falcon-7b-instruct/falcon-7b-instruct-service.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
apiVersion: v1 | ||
kind: Service | ||
metadata: | ||
name: falcon-7b-instruct | ||
spec: | ||
selector: | ||
app: falcon | ||
statefulset.kubernetes.io/pod-name: falcon-7b-instruct-0 | ||
ports: | ||
- protocol: TCP | ||
port: 80 | ||
targetPort: 5000 | ||
type: LoadBalancer | ||
publishNotReadyAddresses: true |
51 changes: 51 additions & 0 deletions
51
presets/k8s/falcon-7b-instruct/falcon-7b-instruct-statefulset.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
apiVersion: apps/v1 | ||
kind: StatefulSet | ||
metadata: | ||
name: falcon-7b-instruct | ||
spec: | ||
replicas: 1 | ||
selector: | ||
matchLabels: | ||
app: falcon | ||
podManagementPolicy: Parallel | ||
template: | ||
metadata: | ||
labels: | ||
app: falcon | ||
spec: | ||
containers: | ||
- name: falcon-container | ||
image: REPO_HERE.azurecr.io/falcon-7b-instruct:TAG_HERE | ||
command: | ||
- /bin/sh | ||
- -c | ||
- accelerate launch --config_file config.yaml --num_processes 1 --num_machines 1 --use_deepspeed --machine_rank 0 --gpu_ids all inference-api.py | ||
livenessProbe: | ||
httpGet: | ||
path: /healthz | ||
port: 5000 | ||
initialDelaySeconds: 600 # 10 Min | ||
periodSeconds: 10 | ||
readinessProbe: | ||
httpGet: | ||
path: /healthz | ||
port: 5000 | ||
initialDelaySeconds: 30 | ||
periodSeconds: 10 | ||
volumeMounts: | ||
- name: dshm | ||
mountPath: /dev/shm | ||
volumes: | ||
- name: dshm | ||
emptyDir: | ||
medium: Memory | ||
tolerations: | ||
- effect: NoSchedule | ||
key: sku | ||
operator: Equal | ||
value: gpu | ||
- effect: NoSchedule | ||
key: nvidia.com/gpu | ||
operator: Exists | ||
nodeSelector: | ||
pool: on7binstruct |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
apiVersion: v1 | ||
kind: Service | ||
metadata: | ||
name: falcon-7b | ||
spec: | ||
selector: | ||
app: falcon | ||
statefulset.kubernetes.io/pod-name: falcon-7b-0 | ||
ports: | ||
- protocol: TCP | ||
port: 80 | ||
targetPort: 5000 | ||
type: LoadBalancer | ||
publishNotReadyAddresses: true | ||
|
Oops, something went wrong.