Deployment on AWS- Elastic Kubernetes Service

Amazon Elastic Kubernetes Service (EKS) is a managed Kubernetes service that makes it easy to run Kubernetes on AWS without needing to install and operate your own Kubernetes control plane. eksctl is a command-line tool that simplifies the process of creating and managing EKS clusters.

This guide will walk you through the steps to create an EKS cluster using eksctl, from prerequisites to clean-up. By the end of this guide, you will have a fully functional EKS cluster with a managed node group.

Prerequisites

Before you begin, ensure you have the following tools installed and configured:

1. AWS CLI: The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services. Install it from the AWS CLI installation guide.
2. eksctl: eksctl is a CLI tool for creating and managing EKS clusters. Install it by following the instructions from the eksctl official documentation.
3. kubectl: kubectl is a command-line tool for interacting with your Kubernetes cluster. Install it from the Kubernetes documentation.

Install AWS, Kubectl & Eksctl CLI’s

Install AWS CLI

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"

unzip awscliv2.zip

sudo ./aws/install

Reference: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html

Configure AWS Command Line using Security Credentials

1. Go to AWS Management Console –> Services –> IAM
2. Select the IAM User: Pranav
3. Important Note: It is recommended to use only IAM user to generate Security Credentials. Using Root User is highly discouraged.
4. Click on Security credentials tab
5. Click on Create access key
6. Copy Access ID and Secret access key
7. Go to command line and provide the required details

aws configure

AWS Access Key ID [None]: ABCDEFGHIAZBERTUCNGG

AWS Secret Access Key [None]: uMe7fumK1IdDB094q2sGFhM5Bqt3HQRw3IHZzBDTm

Default region name [None]: us-east-1

Default output format [None]: json

Install Kubectl CLI

1. Kubectl binaries for EKS please prefer to use from Amazon (Amazon EKS-vended kubectl binary)
2. This will help us to get the exact Kubectl client version based on our EKS Cluster version. You can use the below documentation link to download the binary.

Reference: https://docs.aws.amazon.com/eks/latest/userguide/install-kubectl.html

 # Download the eksctl binary:

Curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_Linux_amd64.tar.gz" | tar xz -C /tmp

# Move the binary to a directory in your PATH:

sudo mv /tmp/eksctl /usr/local/bin

 # Verify the installation:

 eksctl versionInstall eksctlCLI

Create an EKS Cluster

eksctl is a simple CLI tool for creating and managing clusters on EKS – Amazon’s managed Kubernetes service for EC2. It is written in Go, uses CloudFormation, was created by Weaveworks.

To create an eks cluster, copy the below content in a file named cluster-config.yaml.

apiVersion: eksctl.io/v1alpha5

kind: ClusterConfig

availabilityZones:

- us-east-1a

- us-east-1b

- us-east-1c

metadata:

  name: sandbox

  region: us-east-1

  version: '1.27'

  tags:

    created-by: pranav-vunet

    env: sandbox-eks

iam:

  withOIDC: true

vpc:

  cidr: 10.42.0.0/16

  clusterEndpoints:

    privateAccess: false

    publicAccess: true

    #addons:

    #- name: vpc-cni

    #  version: 1.14.1

    #  configurationValues:  "{\"env\":{\"ENABLE_PREFIX_DELEGATION\":\"true\", \"ENABLE_POD_ENI\":\"true\", \"POD_SECURITY_GROUP_ENFORCING_MODE\":\"standard\"},\"enableNetworkPolicy\": \"true\"}"

    #  resolveConflicts: overwrite

managedNodeGroups:

- name: sandbox

  desiredCapacity: 4

  volumeSize: 180

  minSize: 4

  maxSize: 4

  ssh:

    allow: true

    publicKeyName: Sandbox

  iam:

    withAddonPolicies:

      externalDNS: true

      albIngress: true

  instanceType: c5n.4xlarge

  privateNetworking: false

  releaseVersion: 1.27.3-20230816

  updateConfig:

    maxUnavailablePercentage: 50

  labels:

    eks-demo-vunet: 'yes'

Run this command eksctl create cluster -f cluster-config.yaml to initiate the creation of your EKS cluster. The process may take several minutes to complete.

Verify the Cluster Creation

Once the cluster creation process is complete, you can verify that your cluster and nodes are up and running.

List the Clusters

To list all clusters in the specified region:

 eksctl get cluster --region us-east-1

This command will display information about the clusters, including their names and statuses.

List Nodes

To list the nodes in your cluster:

 kubectl get nodes

Before you can use kubectl to interact with your cluster, you need to update your kubeconfig file to use the new cluster:

aws eks update-kubeconfig --region us-east-1 --name my-cluster

This command configures kubectl to use the EKS cluster you created.

Create an EBS CSI driver

Next, you need to create an Amazon EBS CSI driver IAM role and Add the Amazon EBS CSI add-on.

eksctl create iamserviceaccount \

 --region us-east-1 \ (replace the region name)

 --name ebs-csi-controller-sa \

 --namespace kube-system \

 --cluster eksdemo \ (replace with your cluster-name)

 --attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \

 --approve \

 --role-only \

 --role-name AmazonEKS_EBS_CSI_DriverRole

This command deploys an AWS CloudFormation stack that creates an IAM role, attaches the IAM policy to it, and annotates the existing ebs-csi-controller-sa service account with the Amazon Resource Name (ARN) of the IAM role.

Now we can finally add the EBS CSI add-on

eksctl create addon --name aws-ebs-csi-driver --cluster eksdemo --service-account-role-arn arn:aws:iam::$(aws sts get-caller-identity --query Account --output text):role/AmazonEKS_EBS_CSI_DriverRole --force

Downloading the binary

1. Create a folder in the home directory and download the NG installation binary using the below command

• wget https://download.vunetsystems.com/_Downloads_/_vuDocker_/vuSmartMaps_NG_2_11.tar.gz  –user=<username> –password=<password>  –no-check-certificate

💡Note: If you don’t have access to the download server, download the binaries directly from this URL 

Please check with [email protected] for  getting the credentials for Download server.

2. Extract the tar file using

• • tar -xvzf vuSmartMaps_NG_2_11.tar.gz

3. Once extracted, start the launcher, using

• •  ./build/launcher_linux_x86_64

4. Once the launcher has started successfully, access the launcher User Interface from a web browser using the link available.

Welcome Page

• This will be the starting page for Installation.
• Click the Proceed to install button, to start with the actual NG installation.  

Upload License 

1. Here you need to provide a valid license. This license file will contain the services that are going to be installed along with required resources.
2. Upload the valid license and click on Continue. 

💡Note: Please get the updated license files from [email protected].

Please mention the number of nodes when requesting the license you’re using in case of multi node deployment.

Installation Environment

1. Here, you will be prompted to select your installation environment choice
2. Select installation environment as Google Cloud.
3. Click the Continue button to proceed further.

Upload Kubeconfig file

1. Upload kubeconfig which has the super-admin access to the kubernetes cluster.

💡Note: Only YAML file should be uploaded here.

2. On clicking the Continue button, vuLauncher will verify the access to the cluster and get the details of the nodes.

K8s Nodes Selection

1. Here, you can exclude some of the nodes where you don’t want to run the services.
2. This is helpful in case we don’t want to schedule our services on Master node OR in case of AKS we have a reserved pool of nodes where by default, it doesn’t allow scheduling of any pods. Click on Continue once the details are updated.

Configure IP

1. In case you have a Loadbalancer available in your kubernetes cluster(mostly available in case of Managed Kubernetes), you can create a static IP address which will be then used to expose the services.
2. If you don’t have the Load Balancer, we can expose the services on worker nodes. In this case you can skip this step.

Configure Disk 

1. Based on the previous step, 3 storage classes will be configured for each type of storage. Accordingly we need to assign the storage class and encryption setting for each disk.
2. Along with the disk storage, select Encrypted option here for the Hyperscale disk settings and click on Continue.

Mapping

VM to Service Mapping (with advanced Configuration)

1. Here you can override the Service mapping to VM.
2. By default, the vulauncher installation script will allocate resources to the available VM in the best possible way. Click on Continue if you’re fine with the default allocation.
3. If the user wishes to override, they can click the ‘Edit’ button. This will prompt them with a list of VMs, where they can increase or decrease the count as needed.
4. In the Advanced Configuration section, choose the set of VMs for a service. If a VM goes down, then kubernetes will choose the given set of VMs to bring up this service.  By default, all the nodes are chosen.

Customize

1. Here users can override the port that the service is running on.
2. There may be cases where your enterprise requires you to run standard services on non standard ports. Please configure the port for these services here.
3. To override, click the edit button of the respective service, and then write the required port number.
4. Click Continue to proceed.

Install

1. Here this page shows the summary of the information that the user provided.
2. You can click the edit button on the details page, to move back to their respective section and override the change.
3. You can also click the name of the stepper window to move.
4. Then click Continue, to start the deployment procedure.

💡Note: Once you start the deployment, you cannot edit the configuration you provided.

Installation Process

1. The installation shows each event that is going to be performed.
2. Users can click Cancel Installation to stop the ongoing installation. Additionally, they can retry if the process is halted or if the installation stops.
3. Once the installation is successful, a prompt will open. Here, users can click Go to vuSmartMaps, and it will redirect to the vuSmartMaps login page.
4. Use the Login credentials displayed here, to login to the UI.

Create a LoadBalancer Services

AWS Load balancer services should be created so that external access to traefik for Web traffic and to Kafka for data ingestion can be done. In case of Traefik, one AWS Load Balancer service is to be created while for Kafka, N number of load balancer services are to be created where N is the number of nodes in the Kafka cluster.

Follow the below steps to create LoadBalancer services to expose Traefik and Kafka.

Traefik LoadBalancer Service

Create a YAML file named traefik-lb.yaml with the following content:

apiVersion: v1

kind: Service

metadata:

 name: traefik-lb

 namespace: vsmaps

spec:

 type: LoadBalancer

 selector:

   app.kubernetes.io/instance: traefik-vsmaps

   app.kubernetes.io/name: traefik

 ports:

 - name: websecure

   port: 4433

   protocol: TCP

   targetPort: websecure

Apply the service using kubectl apply -f traefik-lb.yaml

Kafka LoadBalancer Service for 1 Node Kafka Cluster

Create a YAML file named broker-lb.yaml with the following content:

apiVersion: v1

kind: Service

metadata:

  name: broker-lb

  namespace: vsmaps

spec:

  type: LoadBalancer

  ports:

  - name: broker

    port: 31092

    protocol: TCP

    targetPort: 31092

  selector:

    app: cp-kafka

    organization: vunet

    release: kafka-cluster

Apply the service using kubectl apply -f broker-lb.yaml

To handle a Kafka cluster with multiple nodes, you can create a separate LoadBalancer service for each Kafka broker. Below is an updated broker-lb.yaml file template that you can duplicate and modify for each broker node.

Kafka LoadBalancer Service for 3 Node Kafka Cluster

Create the LoadBalancer YAML file for each Kafka broker node:

For Broker 1 (broker-lb-1.yaml):

apiVersion: v1

kind: Service

metadata:

 name: broker-lb-1

 namespace: vsmaps

spec:

 type: LoadBalancer

 ports:

 - name: broker-1

   port: 31092

   protocol: TCP

   targetPort: 31092

 selector:

   app: cp-kafka

   release: kafka-cluster

   statefulset.kubernetes.io/pod-name: kafka-cluster-cp-kafka-0

For Broker 2 (broker-lb-2.yaml):

apiVersion: v1

kind: Service

metadata:

 name: broker-lb-2

 namespace: vsmaps

spec:

 type: LoadBalancer

 ports:

 - name: broker-2

   port: 31092

   protocol: TCP

   targetPort: 31092

     #selector:

     #app: cp-kafka

     #organization: vunet

     #release: kafka-cluster

 selector:

   app: cp-kafka

   release: kafka-cluster

   statefulset.kubernetes.io/pod-name: kafka-cluster-cp-kafka-1

For Broker 3 (broker-lb-3.yaml

apiVersion: v1

kind: Service

metadata:

 name: broker-lb-3

 namespace: vsmaps

spec:

 type: LoadBalancer

 ports:

 - name: broker-3

   port: 31092

   protocol: TCP

   targetPort: 31092

 selector:

   app: cp-kafka

   release: kafka-cluster

   statefulset.kubernetes.io/pod-name: kafka-cluster-cp-kafka-2

After creating the YAML files, you can apply them to your Kubernetes cluster by running the following commands:

 kubectl apply -f broker-lb-1.yaml

 kubectl apply -f broker-lb-2.yaml

 kubectl apply -f broker-lb-3.yaml

Verify the Services

Check the status of the services to ensure they are up and running with external IPs assigned.

kubectl get svc -n vsmaps

We should see the EXTERNAL-IP for both traefik-lb and broker-lb services. These IPs can be used to access Traefik and Kafka externally.

Post Deployment Steps

1. Follow the below steps in the master node, once the deployment is successful.
2. To find out which node is the master node, execute the following command in the node where the Kubernetes cluster is running. These details are provided during the initial deployment.

• kubectl get nodes -n vsmaps

In the above output, the e2e-69-187 node is the master node, since the Role is assigned as Master.

3. Run the following command to own the kube config file

sudo chown -R vunet:vunet /etc/kubernetes/admin.conf

Along with the above, please verify the below scenarios

Default Timezone 

Each vuSmartMaps installation will have a default timezone configured in the About page. By default, this is set to UTC. This time zone serves as the base timezone for the platform and can only be updated by the Admin. The default timezone is used for:

• The user interface (observability): viewing Alerts, Dashboards, Reports, Log Analytics, and downloading Reports and Dashboards as PDFs with a Global timeselector.
• Scheduling: The timezone for the scheduled time for Alerts and Reports.
• Distributed channels: Timezone of the content sent via Emails, SMS, WhatsApp,ITSM, etc.

User specific Timezone

User-specific timezones can also be configured by each user from the Profile page,allowing customization of the timezone settings for individual preferences while the platform-wide operations adhere to the default timezone.

To specify the user-specific timezone, navigate to the User-Specific Timezone icon at the top right, which displays the timezone set by the user in their profile.

You can change this timezone by navigating to the profile section.

Select your desired timezone from the User Specific Timezone dropdown menu, and the system will update to reflect the chosen timezone.

Default Retention Settings

Each vuSmartMaps installation will have default data retention settings available under Platform Settings -> Data Retention -> Hyperscale DataStore.

Update the default settings accordingly as per the requirements