Overview GPU slicing (time-slicing) enables efficient GPU resource sharing on Amazon EKS clusters, particularly for AI workloads. By dividing GPU access into smaller time intervals, multiple tasks or processes can share GPU resources, leading to cost optimization and improved utilization. Amazon EKS supports GPU slicing through NVIDIA’s Kubernetes device plugin, which exposes GPU resources to Kubernetes, allowing the scheduler to manage GPU allocation dynamically. Here’s how to enable GPU slicing on EKS clusters. ...
Overview This repository contains Terraform code to deploy an EKS cluster on AWS, utilizing Karpenter for efficient node autoscaling. The cluster supports both x86 (amd64) and Graviton (arm64) architectures. Additionally, the configuration leverages Spot Instances for cost efficiency and includes a demonstration deployment for users to test the setup. Prerequisites Before using this Terraform repository, ensure you have the following installed: Terraform (latest version) AWS CLI (configured with credentials for your AWS account) kubectl (for interacting with the EKS cluster) It needs to be made sure that service-linked role is created for creation of spot instances. It can be created with command. ...