Intra Cluster Communication

Within the vuSmartMaps cluster, various services including the Kubernetes control plane interact across worker nodes.  It is preferred that all the VMs are set up within the same VLAN/subnet for unrestricted access. However, if access control policies are enforced, the following communication ports must be opened.

Ports/Communication between vuSiteManager VM and vuSmartMaps VMs

Ports/Communication between all vuSmartMaps VMs

💡Note:For a single node deployment, ports should be opened internally on the same node. In the case of multi-node deployment, ports should be opened internally on all the nodes.

Configuration Files

Below is the list of configuration files included in the build. These files are client-specific and may vary for each deployment.

In summary, there are two primary YAML files per client that require thorough verification before initiating the deployment process:

• Environment Configuration File: Contains virtual machine (VM) access credentials and cluster details.
• Sizing File: Specifies the list of VMs, service-to-VM mappings, and their respective resource limits.

Environment Config File

The environment configuration file is located at the relative path:
launcher/static-files/input.yaml.

This file includes generic, environment-specific settings such as VM credentials and load balancer IP addresses. For VM credentials, either a password or a private key is required.

# specify if kubernetes cluster is being shared across other user
is_cluster_shared: false

# ssh credentials for all the nodes that are provided in the sizing file.
vm_credentials:
 username: "user"
 password: "password"
 private_key: "path to privatekey" # private key file path

# service name and its loadbalancer
loadbalancer:
 service_name: "loadbalancer address for it"

is_cluster_shared: When deploying vuSmartMaps in an existing Kubernetes cluster that is also utilized by others, this parameter should be set to true.

vm_credentials: Specify the username, password, or private key required to access all nodes in the cluster. The configuration assumes a common set of credentials for accessing all nodes.

loadbalancer: This section will have load balancer addresses for each external facing service of vuSmartMaps.  This configuration applies exclusively to managed Kubernetes clusters hosted on cloud platforms.

Sizing File

The sizing file is unique to each environment and client installation. It provides control over resource limits for services, service grouping across nodes, and additional configurations.

It has the following sections:

primary_images section contains the credentials to access the docker images directly from the ghcr. If the client can access ghcr, we will fetch images during runtime. If the VM doesn’t have a connection to ghcr, then we will ship the images with the build, if so then one ignores the credentials here. Both the username and password are in encrypted mode.

primary_images:
location:"https://ghcr.io"
username:"*******"
password:"*******"
email:"[email protected]"

category section contains a list of tags, which will be used to group the nodes. If the user wants to create a new category its name should be specified here.

category:
   - "data-store-optimized"
   - "data-hub-optimized"
   - "core-services-optimized"

Env_name parameter specifies the type of environment being used. If we are deploying in VM (without Kubernetes cluster), then the value should be “vm” or else for existing Kubernetes cluster, then the value should be “cloud”

env_name: "vm"

node section is applicable only for the “vm” environment (i.e. when env_name: “vm”). This section contains the details of all nodes necessary for installing the Kubernetes cluster.

• ip_address: Specifies the IP address of the VM node.
• storage: Defines the mount points available on each node. For each data tier, we can provide a list of mount points, that will be considered by longhorn

• • vusmart: This data tier doesn’t correspond to any storage class. It will be used for containerd and kubelet. By default, Kubernetes uses a host mount partition to store this information. We will use this data tier to create a soft link to containerd and kubelet 
  • default: This corresponds to the longhorn-default storage class. For this tier data replica will be 1.
  • default-replica: This corresponds to the longhorn-default-replica storage class. For this tier data replica will be 2.
  • hot: This corresponds to the longhorn-hot storage class. Here the data replica will be 1. Currently, only clickhouse will this mount point
  • warm: This corresponds to the longhorn-warm storage class. Here data replica will be 1. Currently, only clickhouse will use this mount point.
  • cold: This corresponds to the longhorn-cold storage class. Here data replica will be 1. Minio object storage service will use this mount point.

• is_master: Set this to true for one of the nodes. This node will be considered to be the Kubernetes master node.
• tags: Here you can specify the name of the group for each node.

node:
   - ip_address: "10.1.1.2"
     tags:
       - "core-services-optimized"
     storage:
       default: ["/data1"]
       hot: ["/data2", "/data3"]

   - ip_address: "10.1.1.2"
     storage:
       default: ["/data1"]

vusmart: "/data2"
       default-replica: ["/data"]
       hot: ["/data2"]
       warm: ["/data3"]
     tags:
       - "core-services-optimized"

   - ip_address: "10.1.1.1"
     is_master: true
     tags:
       - "core-services-optimized"
     storage:
       default: ["/data1"]

For the “cloud” environment, the following will be the node section.

• node_name: This will be the existing Kubernetes node name. One could get a list of nodes via the command: kubectl get nodes
• tags: This will be the corresponding group name of the nodes.

node:
   - node_name: "node-1"
     tags:
       - "core-services-optimized"

   - node_name: "node-2"
     tags:
       - "core-services-optimized"

   - node_name: "node-3"
     tags:
       - "core-services-optimized"

This section will be available only in a “cloud” type environment. Here, the storage class for each tier must be specified. The default tier is mandatory, while the others are optional.

storage_class:
   default: "storageclass-1"
   hot: "storageclass-2"
   cold: "storageclass-3"
   warm: "storageclass-4"
   default-replica: ""

services section will provide a list of all services to be deployed as part of the vuSmartMaps installation. Each entry will include the following elements:

• name:  Name of the service.
• tags: Group categories for node selection. If we specify the category here, we will consider only those nodes with the respective category
• resource_limit: Defines minimum and maximum CPU and memory for the pod
• replica: Number of pods to deploy.
• disk: Specifies required storage for each data tier.
• config_override: if there is any section in values.yaml that the launcher needs to be updated before deployment, then we can provide it here. Here the key will be the yml path till we get it in values.yaml, the value will be the one that gets updated with. If the key doesn’t exist then the launcher will create a new key with a given path.

services:
  - name: "service_name"
    tags:
       - "core-services-optimized"
    resource_limit:
       min:
         cpu: "0.25"
         memory: "500Mi"
       max:
         cpu: "0.4"
         memory: "2Gi"
    replica: 1
    disk:
       default: 2Gi
       default-replica: 2Gi
    config_override:
       defaultSettings.key1: "value1"        defaultSettings.key2: “value2”