Storage configuration

Provision FSx for ONTAP

FSx for NetApp ONTAP provides fully managed shared storage with features like snapshots, cloning, and data tiering.
It’s integrated with AWS, allowing for seamless cloud storage solutions.
In this step, you’re setting up the FSx for ONTAP file system which will be used as the backend storage for your OpenShift applications.
We will create a single-AZ FSx for ONTAP file system in the same VPC as the ROSA cluster.

Get Subnet and VPC IDs

Let’s take a look at the VPC subnets.

From the bastion, run the following command.

aws ec2 describe-subnets --query 'Subnets[*].{SubnetId:SubnetId,VpcId:VpcId,CidrBlock:CidrBlock}' \
   --output table

Output from commands will be showin in a box like below.

-------------------------------------------------------------------------
|                             DescribeSubnets                           |
---------------------------------------------------------------------
|    CidrBlock   |         SubnetId           |          VpcId          |
---------------------------------------------------------------------
|  10.0.0.0/18   |  subnet-0c1e3b083f692a17f  |  vpc-0994809fd6f55252b  |
|  192.168.0.0/24|  subnet-01b8fa59d97657eca  |  vpc-0a4106cf5b3b895b5  |
|  10.0.128.0/18 |  subnet-0299fe13ba470aa9f  |  vpc-0994809fd6f55252b  |
---------------------------------------------------------------------

Run the following command to assign the SubnetId to the SUBNETID variable.

export SUBNETID=$(aws ec2 describe-subnets --query 'Subnets[?CidrBlock==`10.0.0.0/18`].SubnetId' \
   --output json | jq -r '.[0]') && echo $SUBNETID

Assign the VpcId to the VPCID variable.

export VPCID=$(aws ec2 describe-subnets --query 'Subnets[?CidrBlock==`10.0.0.0/18`].VpcId' \
   --output json | jq -r '.[0]') && echo $VPCID

Create the FSx stack

This step is option, please ask your instructor

Create the stack

This can take 20 - 30 minutes

aws cloudformation create-stack \
  --stack-name ROSA-FSXONTAP \
  --template-body file://./FSxONTAP.yml \
  --region us-east-2 \
  --parameters \
     ParameterKey=Subnet1ID,ParameterValue=$SUBNETID \
     ParameterKey=myVpc,ParameterValue=$VPCID \
     ParameterKey=FileSystemName,ParameterValue=ROSA-myFSxONTAP \
     ParameterKey=ThroughputCapacity,ParameterValue=512 \
     ParameterKey=FSxAllowedCIDR,ParameterValue=10.0.0.0/16 \
     ParameterKey=FsxAdminPassword,ParameterValue=Rosa12345 \
     ParameterKey=SvmAdminPassword,ParameterValue=Rosa12345 \
  --capabilities CAPABILITY_NAMED_IAM

You can monitor the progress with the following command. You may have to run it a few times before the stack is fully configured.

aws cloudformation describe-stacks --stack ROSA-FSXONTAP | jq -r '.Stacks[] | select(.StackName=="ROSA-FSXONTAP") | .StackStatus'

CREATE_IN_PROGRESS

Once the stack is ready, you will se the CREATE_COMPLETE status

CREATE_COMPLETE

Verify your file system and storage virtual machine (SVM1) have been created.

aws fsx describe-file-systems

{
    "FileSystems": [
        {
            "OwnerId": "760156770233",
            "CreationTime": "2024-05-31T10:28:50.754000+00:00",
            "FileSystemId": "fs-0dd8fac60f4d9d0e5",
            "FileSystemType": "ONTAP",
            "Lifecycle": "AVAILABLE",
            "StorageCapacity": 1024,
            "StorageType": "SSD",
...

aws fsx describe-storage-virtual-machines

{
    "StorageVirtualMachines": [
        {
            "CreationTime": "2024-05-31T10:30:51.208000+00:00",
            "Endpoints": {
                "Iscsi": {
                    "DNSName": "iscsi.svm-0ffc7291efcc7ff99.fs-0dd8fac60f4d9d0e5.fsx.eu-west-1.amazonaws.com",
                    "IpAddresses": [
                        "10.0.2.48",
                        "10.0.34.116"
                    ]
                },
                "Management": {
                    "DNSName": "svm-0ffc7291efcc7ff99.fs-0dd8fac60f4d9d0e5.fsx.eu-west-1.amazonaws.com",
                    "IpAddresses": [
                        "10.0.46.136"
                    ]
                },
                "Nfs": {
                    "DNSName": "svm-0ffc7291efcc7ff99.fs-0dd8fac60f4d9d0e5.fsx.eu-west-1.amazonaws.com",
                    "IpAddresses": [
                        "10.0.46.136"
                    ]
                }
            },
...

Install and Configure the Trident CSI driver for ROSA

Trident is NetApp’s dynamic storage orchestrator for OpenShift.
It automates and manages storage resources for containers.
By installing Trident, you’re enabling your ROSA cluster to dynamically provision and manage storage resources on FSx for ONTAP, providing a robust and scalable storage solution for your applications.

Install Trident

To begin, add the Astra Trident Helm repository

helm repo add netapp-trident https://netapp.github.io/trident-helm-chart

Use helm install to install the Trident driver in the trident namespace. You may see a warning about Pod Security. It can be ignored.

helm install trident netapp-trident/trident-operator --create-namespace --namespace trident

NAME: trident
LAST DEPLOYED: Fri May 31 15:12:00 2024
NAMESPACE: trident
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Thank you for installing trident-operator, which will deploy and manage NetApp's Trident CSI
storage provisioner for Kubernetes.

Your release is named 'trident' and is installed into the 'trident' namespace.
Please note that there must be only one instance of Trident (and trident-operator) in a Kubernetes cluster.

To configure Trident to manage storage resources, you will need a copy of tridentctl, which is
available in pre-packaged Trident releases.  You may find all Trident releases and source code
online at https://github.com/NetApp/trident.

To learn more about the release, try:

  $ helm status trident
  $ helm get all trident

Verify the Trident driver installation.

helm status trident -n trident | grep "NAME:" -A 5

NAME: trident
LAST DEPLOYED: Mon Nov  6 20:52:31 2023
NAMESPACE: trident
STATUS: deployed
REVISION: 1
TEST SUITE: None

Helm deployes the Astra Trident Operator, to verify the Operator status:

oc get tridentorchestrators -n trident -oyaml | grep "status: " -A 1

    status: Installed
    version: v24.02.0

Create a secret to store the SVM username and password in the ROSA cluster

Download and view the svm-secret.yml file. Take note of the password.

Be sure to inspect any files before applying them.

cd ~/rosa-fsx/fsx
cat svm-secret.yml

apiVersion: v1
kind: Secret
metadata:
  name: backend-fsx-ontap-nas-secret
  namespace: trident
type: Opaque
stringData:
  username: vsadmin
  password: Rosa12345

Add the secret to the ROSA cluster.

oc apply -f svm-secret.yml

Verify the secret has been added to the ROSA cluster.

oc get secrets -n trident backend-fsx-ontap-nas-secret

NAME                                 TYPE                                  DATA   AGE
backend-fsx-ontap-nas-secret         Opaque                                2      24h

Install and Configure the Trident CSI Backend to FSx for ONTAP

The Trident backend configuration tells Trident how to communicate with the storage system, in this case, FSx for ONTAP.
We willl use the ontap-nas driver to provision storage volumes.
We are going to edit backend-ontap-nas.yml so it has the IP from the ManagementLIF and DataLIF IP addresses of the FSx Server Virtual Mancine.

Create the Trident Backend

Assign the IP from the SVM to a variable.

export SVMIP=$(aws fsx describe-storage-virtual-machines | jq -r '.StorageVirtualMachines[].Endpoints.Management.IpAddresses[]') && echo $SVMIP

And then update backend-ontap-nas.yml with the IP.

sed -i "s/<<management-ip>>/$SVMIP/g" backend-ontap-nas.yml

Review the contents of the file.

cat backend-ontap-nas.yml

Example:

apiVersion: trident.netapp.io/v1
kind: TridentBackendConfig
metadata:
  name: backend-fsx-ontap-nas
  namespace: trident
spec:
  version: 1
  backendName: fsx-ontap
  storageDriverName: ontap-nas
  managementLIF: 10.0.50.139
  dataLIF: 10.0.50.139
  svm: SVM1
  credentials:
    name: backend-fsx-ontap-nas-secret

Next, execute the following commands to create the Trident backend in the ROSA cluster.

oc apply -f backend-ontap-nas.yml

Verify the backend configuration.

oc get tridentbackendconfigs -n trident

NAME                    BACKEND NAME   BACKEND UUID                           PHASE   STATUS
backend-fsx-ontap-nas   fsx-ontap      1f490bf3-492c-4ef7-899e-9e7d8711c82f   Bound   Success

Create storage class in ROSA cluster

A storage class defines how storage is dynamically provisioned, specifying attributes like size and performance.
A storage class automates the creation of storage volumes when applications request storage through PVCs.
This configures a storage class to work with Trident, ensuring efficient management of FSx for NetApp ONTAP as backend storage.

Create the new `trident-csi` storage class.

oc apply -f storage-class-csi-nas.yml

Verify the status of the trident-csi storage class creation.

oc get storageclass trident-csi

NAME            PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
trident-csi     csi.trident.netapp.io   Retain          Immediate              true                   1h58m