README.md

HPC Recipes for AWS

This page shows all available recipes. They are organized by theme (e.g. Database management and Networking). Each recipe also has tags that indicate the key technologies (i.e. parallelcluster and rds) it references.

Making use of recipes

You can use an HPC Recipe for AWS in several ways:

• Learn from it. Read the README file and inspect its assets directory to understand how it works.
• Launch resources with it. Navigate to its README page and follow the instructions. There is often a quick-launch link to the AWS CloudFormation console.
• Incorporate it. Recipe assets are permissively licensed so you can use them in your own builds. You can also bring assets in by URL

Recipe List

➡️ aws: General AWS (default)

There are currently no recipes in this namespace.

➡️ batch: AWS Batch

whisper_transcription_awsbatch_inferentia

• About: Whisper audio transcription powered by AWS Batch and AWS Inferentia
• Usage: README.md

➡️ db: Database management

slurm_accounting_db

• About: Set up a replicated Amazon Aurora database that can support Slurm accounting.
• Usage: README.md

➡️ dir: Directory services

demo_managed_ad

• About: Stand up a basic AWS Managed Microsoft AD for use with AWS ParallelCluster.
• Usage: README.md

demo_openldap

• About: Simple OpenLDAP service running on ECS
• Usage: README.md

➡️ env: User environment

spack

• About: Install and configure Spack + Spack configs on shared storage
• Usage: README.md

➡️ iam: Identity Access and Management

There are currently no recipes in this namespace.

➡️ ide: IDEs and GUIs

There are currently no recipes in this namespace.

➡️ net: Networking

hpc_basic

• About: Networking configuration for HPC on AWS. Can use an existing VPC or create a new one.
• Usage: README.md

hpc_large_scale

• About: Networking configuration for large-scale HPC on AWS. Creates a new VPC.
• Usage: README.md

➡️ pcluster: AWS ParallelCluster

decoupled_storage

• About: Demonstrate decoupled shared storage using Amazon EFS.
• Usage: README.md

isolated-clusters

• About: Provides templates to configure AWS ParallelCluster for isolated environments/those with no Internet access.
• Usage: README.md

latest

• About: Quick demo of the latest ParallelCluster release
• Usage: README.md

login_nodes

• About: Demonstrates the new Login Nodes feature in ParallelCluster 3.7.2
• Usage: README.md

login_nodes_ami_for_res

• About: Create an AMI of a ParallelCluster LoginNode compatible with Research and Engineering Studio
• Usage: README.md

multi_az

• About: Create a cluster that can launch instances in multiple Availability Zones
• Usage: README.md

multi_user

• About: Creates a multi-user instance of AWS ParallelCluster using AWS Managed AD as the directory service.
• Usage: README.md

pcui

• About: Create an AWS ParallelCluster with ParallelClusterUI to manage it.
• Usage: README.md

slurm_accounting

• About: Create an instance of AWS ParallelCluster with Slurm accounting enabled, using Amazon RDS as the DBMS.
• Usage: README.md

slurm_accounting_with_email

• About: Create an instance of AWS ParallelCluster with Slurm accounting enabled and e-mail notifications using Slurm-Mail, using Amazon RDS as the DBMS.
• Usage: README.md

stig

• About: Includes files to accelerate STIG compliance for Parallelcluster images as well as scripts to verify OSCAP results
• Usage: README.md

try_hpc6a

• About: Create a ParallelCluster system to try out Hpc6a instances.
• Usage: README.md

try_hpc6id

• About: Create a ParallelCluster system to try out Hpc6id instances.
• Usage: README.md

try_hpc7a

• About: Create a ParallelCluster system to try out Hpc7a instances.
• Usage: README.md

try_hpc7g

• About: Create a ParallelCluster system to try out Hpc7g instances.
• Usage: README.md

try_trn1

• About: Create a ParallelCluster system to try out Trn1 instances.
• Usage: README.md

➡️ pcs: AWS Parallel Computing Service

byo_login

• About: Connect standalone access nodes to AWS PCS
• Usage: README.md

cfd_cluster

• About: Run OpenFOAM on PCS
• Usage: README.md

cloudwatch

• About: Enable CloudWatch Logs on PCS instances
• Usage: README.md

enable_efa

• About: Use EFA with AWS PCS compute node groups
• Usage: README.md

getting_started

• About: Supports "Getting Started with AWS PCS" in AWS PCS user guide.
• Usage: README.md

multiuser_demo

• About: Example for configuration for multi-user AWS PCS using LDAPS
• Usage: README.md

spack_for_pcs

• About: Install Spack on AWS PCS
• Usage: README.md

➡️ res: Research and Engineering Studio on AWS

entra_id

• About: Set up Entra ID with RES
• Usage: README.md

res_demo_env

• About: Research and Engineering Studio (RES) on AWS demo environment
• Usage: README.md

➡️ security: Security configuration

public_certs

• About: Certificate creation for public domain. Creates secrets for pem/key files for a public cert for a Route53 owned domain.
• Usage: README.md

➡️ scheduler: HPC scheduler

There are currently no recipes in this namespace.

➡️ storage: Storage

efs_simple

• About: Create an Amazon EFS filesystem and mount targets in three Availability Zones.
• Usage: README.md

fsx_lustre

• About: Creates either a scratch or persistent FSxL filesystem and a security group for accessing the filesystem.
• Usage: README.md

fsx_lustre_s3_dra

• About: Demonstrates an FSx for Lustre filesystem with an S3 data repository association
• Usage: README.md

fsx_openzfs

• About: Provision an FSx for OpenZFS filesystem and a security groups for accessing the filesystem
• Usage: README.md

mountpoint_s3

• About: Configure AWS ParallelCluster to mount S3 buckets to directories using mountpoint-s3
• Usage: README.md

s3_demo

• About: Create an Amazon S3 bucket using CloudFormation.
• Usage: README.md

➡️ training: Teaching and training recipes

try_recipes_1

• About: Cluster example using manual configuration.
• Usage: README.md

try_recipes_2

• About: Cluster example using CloudFormation imports.
• Usage: README.md

try_recipes_3

• About: Cluster example using nested CloudFormation stacks.
• Usage: README.md

---

Incorporating recipe assets

You can access recipe assets using HTTPS or S3 protocols. They are mirrored to an S3 bucket so they can used with AWS CloudFormation.

Here are example URLs for the CloudFormation launch template for the "latest" ParallelCluster recipe (recipes/pcluster/latest):

• HTTPS Template: https://aws-hpc-recipes.s3.us-east-1.amazonaws.com/main/recipes/pcluster/latest/assets/launch.yaml
• S3 Template: s3://aws-hpc-recipes/main/recipes/pcluster/latest/assets/launch.yaml

In both URLs, /main/ is the HPC Recipes on AWS version. At present, the only supported version is main.

You can use the HTTPS link in a couple of contexts.

1. You can download the file it references
   • curl -O HTTPS_URL
2. If it's a Cloudformation template, you can import it into the CloudFormation console when creating a stack
3. You can embed it in a CloudFormation quick-launch link.
   • Template: https://console.aws.amazon.com/cloudformation/home?region=REGION#/stacks/create/review?templateURL=HTTPS_URL

The S3 URL is useful anywhere you want to use native S3 commands for data access.

• Template: aws s3 cp s3://aws-hpc-recipes/main/recipes/NAMESPACE/RECIPE/assets/FILE.yaml .