Configure IAM Role Access for Amazon S3 Plugin in Kubernetes (EKS)

Beginning in v7.1.2, the Amazon S3 plugin supports accessing S3 buckets in Kubernetes using IAM Roles for Service Accounts (IRSA) instead of static access keys. This article describes how to configure IAM Roles for Service Accounts in an Amazon EKS cluster to enable the Amazon S3 plugin to access S3 buckets and KMS-encrypted objects.

PREREQUISITES:

AWS CLI: Installed and configured with IAM permissions to manage roles and policies;
kubectl: Configured to access your EKS cluster;
eksctl: Installed;
An existing EKS cluster;
IAM permissions to create and manage IAM roles and policies.

1. Create IAM Policy

Define the required permissions for accessing S3 and decrypting with KMS. Create a file named s3-kms-policy.json with the following content:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowReadAccessToSpecificBucket",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::<your-bucket-name>", // Replace <your-bucket-name> with your S3 bucket name
        "arn:aws:s3:::<your-bucket-name>/*" // Replace <your-bucket-name> with your S3 bucket name
      ]
    },
    {
      "Sid": "AllowDecryptWithKMS",
      "Effect": "Allow",
      "Action": [
        "kms:Decrypt",
        "kms:DescribeKey"
      ],
      "Resource": "arn:aws:kms:<your-region>:<your-aws-account-id>:key/<your-kms-key-id>"   // Replace <your-region>, <your-aws-account-id>, and <your-kms-key-id> with your actual values 
    }
  ]
}

Create the IAM policy:

aws iam create-policy \
  --policy-name S3KMSSpecificAccess \
  --policy-document file://s3-kms-policy.json

Expected output includes:

Policy ARN: arn:aws:iam::123456789012:policy/S3KMSSpecificAccess

2. Associate OIDC Provider with Cluster

Associate the IAM OIDC identity provider with your EKS cluster to enable IAM roles for service accounts:

eksctl utils associate-iam-oidc-provider \
  --region <your-region> \ # Replace with your AWS region 
  --cluster <your-eks-cluster-name> \ # Replace with your EKS cluster
  --approve

3. Create Service Account with IAM Role

Create a Kubernetes service account that is associated with the IAM policy:

eksctl create iamserviceaccount \
--name s3-kms-access-sa \
--namespace default \
--cluster <your-eks-cluster-name> \ # Replace with your EKS cluster
--attach-policy-arn arn:aws:iam::<account-id>:policy/S3KMSSpecificAccess \ # Replace with your 12-digit AWS account ID 
--approve

4. Verify Service Account Creation

Confirm that the service account was created successfully:

kubectl get sa --namespace default

You should see the s3-kms-access-sa service account in the output.

5. Verify S3 Access with Test Pod

5.1. Create Test Pod

Use a temporary pod to validate S3 and KMS access using the IRSA-enabled ServiceAccount.

Create a test pod manifest named test-pod.yaml:

apiVersion: v1
kind: Pod
metadata:
  name: s3-kms-test
  namespace: default
spec:
  serviceAccountName: s3-kms-access-sa
  containers:
    - name: app
      image: public.ecr.aws/aws-cli/aws-cli:latest
      command: ["sh", "-c", "sleep 3600"]

Apply the pod:

kubectl apply -f test-pod.yaml

5.2. Test Listing AWS Bucket and Downloading Object

Test listing the S3 bucket:

aws s3 ls s3://<your-bucket-name>/ # Replace with the name of your S3 bucket

Test downloading an object:

aws s3 cp s3://<your-bucket-name>/test-file.txt /tmp/ # Replace with the name of your S3 bucket

5.3. Delete Test Pod

kubectl delete pod s3-kms-test

6. Update and Reapply Application.Yml

Once validation is complete, update application.yml by adding serviceAccountName: s3-kms-access-sa to the metricinsights-dataprocessor deployment under spec.template.spec:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: metricinsights-dataprocessor
  labels:
    app: dataprocessor
spec:
  replicas: 1
  selector:
    matchLabels:
      app: dataprocessor
  template:
    metadata:
      labels:
        app: dataprocessor
        name: metricinsights-dataprocessor
    spec:
      serviceAccountName: s3-kms-access-sa

Re-apply the manifest:

kubectl apply -f application.yml