❗️ DISCLAIMER: This project's architecture should work, but doesn't meet the requirement: using GPU serverlessly to lower the cost, which make this project become a experimental demostration.
Feel free to clone or fork AND USE AT YOUR OWN RISK! 😅
There are lots of ways to build serverless applications, but at this time (2024 Q1), GPU is still not supported by Amazon Lambda (althought the rumor said is's still on the development roadmap).
This project accomplish a method to run the workload by Managed ECS computes, but invoked by AWS Batch. When users upload text file to S3 bucket, Batch Job Queue will be invoked and creates many Batch Job, then launch ECS computes to process them.
We choose EC2 as ECS's launch type so GPU accelarator can be attached on, then process example task: Input a prompt into LLM model with GPU.
The source prompt is created as a text file and stored in Source S3 Bucket, and the processed file will be put into Destination S3 Bucket.
This following chart shows the whole picture:
cdk deployUpload a .txt file contains the prompt to the "Source S3 Bucket", the result will be produced by the container, rename the original filename with a "`processed-" prefix, then send to "Deestination S3 Bucket".
cdk destroyWe choose only g5g.xlarge instance class to cut the cost to run EC2, and the architecture design is serverless because the files are processed by a short-lived container. We also apply the strategy to use Spot instances with a very high bid percentage.
But the cost still far from compitable with managed service. Take Machine Learning workloads for example, the cost of SageMaker, Bedrock is much lower than EC2.
Although the example we made is about Machine Learning, it's recommended to apply this project to run other workloads that consumes GPU, and the managed service is not yet exists on the earch.
During first deployment, the CodeBuild Project will be triggered along with the CDK project, so will produce the first Docker Image into a new ECR repo. After that, the CodeBuild project only be triggered when project related resources modified, which is the Lambda function. Otherwise, it won't be triggered even the documents like Dockerfile, buildspec.yaml edited.
If we gonna trigger another CodeBuild process, either modify the Lambda function before deploy CDK project, or cline the CodeCommit repo and commit the changes.