Mount Windows Shared Folder on AWS EKS

After running Metric Insights on AWS EKS, it is required to mount a Windows shared folder into Metric Insights pods. Windows shared folder has to be mounted on /opt/mi/shared by the user www-data:www-data or 33:33 inside the following pods:

  1. web (master and slave)
  2. dataprocessor

This process comprises the following steps:

  1. Install Driver
  2. Create Kubernetes Secret
  3. Configure Custom Storage Class
  4. Configure Persistent Volume
  5. Configure Persistent Volume Claim
  6. Update Metric Insights Deployment Manifest
  7. Check the Mount Point

1. Install Driver

Note: Ensure that kubectl can be run with admin access in the Kubernetes cluster.

AWS EKS does not support native mounting of SMB share into pods. To enable mounting it is required to install the CSI driver in the Kubernetes cluster. Execute the following command from the Linux environment where kubectl is installed and configured to work with the Kubernetes cluster:

$ curl -skSL https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/v1.9.0/deploy/install-driver.sh | bash -s v1.9.0 --

2. Create Kubernetes Secret

Note: The secret has to be created in the default namespace.

Ensure that this command is not kept in history or remove this command from history if it has been stored.

Kubernetes cluster needs read/write permissions for the Windows share to mount it on /opt/mi/shared. Use the following command to create a Kubernetes secret that stores credentials for accessing the Windows share:

$ kubectl create secret generic metricinsights-shared-credentials -n default --from-literal domain=domain --from-literal username=user --from-literal password=password

Replace the following values:

domain - Windows Server domain,

user and password - credentials of the user with read/write access to the shared folder.

3. Configure Custom Storage Class

  1. Create a file metricinsights-smb-sc.yml with the content provided in the code below:
$ cat metricinsights-smb-sc.yml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: metricinsights-shared-smb
provisioner: smb.csi.k8s.io
parameters:
  source: "//remote server/share name" # Use the FQDN provided by Amazon FSx
  # if csi.storage.k8s.io/provisioner-secret is provided, will create a sub directory
  # with PV name under source
  csi.storage.k8s.io/provisioner-secret-name: "metricinsights-shared-credentials"
  csi.storage.k8s.io/provisioner-secret-namespace: "default"
  csi.storage.k8s.io/node-stage-secret-name: "metricinsights-shared-credentials"
  csi.storage.k8s.io/node-stage-secret-namespace: "default"
reclaimPolicy: Delete  # available values: Delete, Retain
volumeBindingMode: Immediate
mountOptions:
  - dir_mode=0755
  - file_mode=0644
  - uid=33

Replace the following values:

remote server - Windows server,

share name - full path to the shared folder.

  1. Execute the following command to create a custom storage class: kubectl apply -f metricinsights-smb-sc.yml

4. Configure Persistent Volume

The next layer between Kubernetes cluster and Windows Server is a Persistent Volume.

  1. Create a file metricinsights-shared-pv.yml with information on mounting options provided in the code below:
$ cat metricinsights-shared-pv.yml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: metricinsights-shared
spec:
  capacity:
    storage: 100Gi # PV-Size
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: metricinsights-shared-smb
  mountOptions:
    - dir_mode=0755
    - file_mode=0644
    - uid=33
    - gid=33
  csi:
    driver: smb.csi.k8s.io
    readOnly: false
    volumeHandle: VOLUME_ID # Must be unique in the EKS Cluster, eg: PV-1
    volumeAttributes:
      source: "//remote server/share name"
    nodeStageSecretRef:
      name: metricinsights-shared-credentials
      namespace: default 

Replace the following values:

domain - Windows Server domain,

user and password - credentials of the user with read/write access to the shared folder.

  1. Execute the following command to create a Persistent Volume: kubectl apply -f metricinsights-shared-pv.yml

5. Configure Persistent Volume Claim

The last layer between a Kubernetes pod and Windows server is a Persistent Volume Claim. It is an abstract layer that will be connected to the pod directly.

  1. Create a file metricinsights-pvc.yml with the content provided in the code below:
$ cat metricinsights-pvc.yml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: metricinsights-shared
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 99Gi # Size of the PVC, must be lower then the PV-Size.
  volumeName: metricinsights-shared
  storageClassName: metricinsights-shared-smb
  1. Execute the following command to create a Persistent Volume Claim: kubectl apply -f metricinsights-pvc.yml

6. Update Metric Insights Deployment Manifest

In this step it is required to add new volumes into the  web-master , web-slave, and dataprocessor pods. Update the following blocks in the according deployment manifest's sections:

$ cat deployment.yml
...
        volumeMounts:
          - name: shared
            mountPath: "/opt/mi/shared"
      volumes:
        - name: shared
          persistentVolumeClaim:
            claimName: metricinsights-shared
...

This is the example of the updated deployment manifest's section for web master/slave pods:

Ensure, that these changes were applied for metricinsights-web-master,  metricinsights-web-slave and metricinsights-dataprocessor.

7. Check the Mount Point

User www-user or user with the uid 33 requires read/write access to /opt/mi/shared in web-master, web-slave, and dataprocessor pods. Use the following approach to check read/write access:

  1. Access the required pod
  2. Access /opt/mi/
  3. Use ls -l | grep shared to ensure that the owner of the directory is www-data:www-data
  4. Use df -h to ensure that the Windows shared folder has been mounted on /opt/mi/shared correctly

This operation needs to be repeated for web-master, web-slave, and dataprocessor pods.