Create and Deploy Managed K8s

Objective

This document provides instructions on how to create managed K8s cluster and deploy it on the VoltStack site. Managed K8s cluster is similar in principle to regular K8s and you can use tools like kubectl to perform operations that are common to regular K8s. Volterra provides mechanism to easily deploy applications using managed K8s across VoltStack sites forming DC clusters. To know more about deploying VoltStack site, see Create VoltStack Site.

Using the instructions provided in this guide, you can create a managed K8s cluster, associate it with a VoltStack site, and deploy applications using its Kubeconfig.

Note: Managed K8s is also known as physical K8s.


Prerequisites


Restrictions

The following restrictions apply :

  • Using vK8s and managed K8s in the same site is supported. However, if the namespace is already used local to a managed K8s cluster, then any object creation in that namespace using vK8s is not supported for that site. Conversely, if a namespace is already used by vK8s, then operations on local managed K8s cluster are not supported.
  • Managed K8s is supported only for VoltStack site and not supported for other sites.
  • Managed K8s can be enabled by applying it before provisioning the VoltStack site. It is not supported for enabling by updating existing VoltStack site.
  • Managed K8s cannot be disabled once it is enabled.
  • In case of managed K8s, Role and RoleBinding operations are supported via kubectl. However, ClusterRoleBinding, PodSecurityPolicy, and ClusterRole are not supported for kubectl. These can be configured only through VoltConsole.

Configuration

Enabling a managed K8s cluster on a VoltStack site requires you to first create it and apply it during VoltStack site creation. You can also create a new managed K8s cluster as part of VoltStsck site creation. This example shows creating a K8s cluster separately and attaches it to VoltStack site during creation.

Create Managed K8s Cluster

Perform the following steps to create managed K8s cluster

Step 1: Log into VoltConsole and start K8s cluster object creation.

Navigate to Manage -> Site Management in the system namespace and select K8s Clusters in the page groups. Click Add K8s Cluster.

add pk8s
Figure: K8s Cluster Creation

Step 2: Configure metadata and access sections.
  • Enter a name in the metadata section. Optionally, set labels and add description.
  • Go to Access section and select Enanble Site Local API Access option for the Site Local Access field. This enables local access to K8s cluster.
  • Enter local domain name for the K8s cluster in the <sitename>.<localdomain> format. The local K8s API server will become accessible via this domain name.
  • Optionally, select Custom K8s Port for the Port for K8s API Server field and enter a port value in the Custom K8s Port field. This example uses the default K8s port option.

pk8s access
Figure: Access Section Configuration

  • Select Enable VoltConsole API Access option for the VoltConsole Access field.

Note: Monitoring K8s cluster works only when you enable VoltConsole API access.

Step 3: Configure security section.

The security configuration is enabled with default settings for pod security policies, K8s cluster roles, and K8s cluster role bindings. Optionally, you enable custom settings for these fields. Perform the following steps:

Step 3.1: Configure custom pod security policies.

Select Custom Pod Security Policies option for the POD Security Policies field. Click on the Pod Security Policy List field and add a policy from the list of displayed options or create a new policy and attach. This example shows creating a new policy. Create a new policy as per the following guidelines:

  • Click Create new pod security policy in the Pod Security Policy List field to open new policy form. Enter a name in the metadata section.
  • Click Configure under the Pod Security Policy Specification field and do the following:
Step 3.1.1: Optionally, configure the privileges and capabilities.

Configure the Privilege and Capabilities section as per the following guidelines:

  • Enable the Privileged, Allow Privilege Escalation, Default Allow Privilege Escalation fields.
  • Select Custom Default Capabilities option for the Change Default Capabilities field, Allowed Add Capabilities option for the Allowed Add Capabilities field, and Drop Capabilities for the Drop from K8s Default Capabilities field.
  • For the custom default capabilities, allowed add capabilities, and drop capabilities fields, click on their respective Capability List fields and select the See Common Choices option to expand the choices. Select an option from the list. You can add more choices using the Add item option.
Step 3.1.2: Optionally, configure the volumes and mounts.

Configure the Volumes and Mounts section as per the following guidelines:

  • Click Add item under Volume, Allowed Flex Volumes, Allowed Host Paths, and Allowed Proc Mounts fields. Enter the values for those fields and you can add multiple entries using the Add item option for each of these fields.

Note: Leaving empty value for Volumes disables any volumes. For rest of the fields, the default values are applied. In case of Host Path Prefix, you can select the Read Only checkbox to mount a read-only volume.

  • Enable Read Only Root Filesystem so that containers run with read-only root file system.
Step 3.1.3: Optionally, configure host access and sysctl.

Configure the Host Access and Sysctl section as per the following guidelines:

  • Enable the Host Network, Host IPC, and Host PID fields to allow the use of host network, host IPC, and host PID in the pod spec.
  • Enter port ranges in the Host Ports Ranges field to expose those host ports.
Step 3.1.4: Optionally, configure security context.

Configure the Security Context section as per the following guidelines:

  • Select Run As User, Run As Group, Supplemental Groups Allowed, and FS Groups Allowed for the Select Runs As User, Select Runs As Group, Select Supplemental Groups, and Select FS Groups fields respectively.
  • For each of the above fields, enter the following configuration:

    • Enter ID values in the Starting ID and Ending ID fields. You can add more ranges using the Add item option.
    • Click on the Rules field and select the See Common Choices option to expand the choices. Select one of the MustRunAs, MayRunAs, or RunAsAny choice.

Click Apply and then click Continue to create and apply the pod security policy to the K8s cluster.

Note: You can add more pod security policies using the Add item option.

Step 3.2: Configure K8s cluster role.

Select Custom K8s Cluster Roles option for the K8s Cluster Roles field and click on the Cluster Role List field. Select a role from the displayed list or click Create new cluster role to create and attach it. This example shows creating a new cluster role. Configure the cluster role object as per the following guidelines:

  • Enter a name in the metadata section.
  • Go to Cluster Role section and select Policy Rule List or Aggregate Rule for the Rule Type field.

    • For Policy Rule List option, select List of Resources or List of Non Resource URL(s) options.
    • For the List of Resources option, do the following:

      • Enter list of API groups in the API Groups field. You can add more than one entry using the Add item option.
      • Enter list of resource types in the Resource Types field. You can add more than one entry using the Add item option.
      • Enter list of resource instances in the Resource Instances field. You can add more than one entry using the Add item option.
      • Enter allowed list of operations in the Allowed Verbs field. You can add more than one entry using the Add item option. Alternatively, you can enter * to allow all operations on the resources.
    • For List of Non Resource URL(s) option, do the following:

      • Enter URLs that do not represent K8s resources in the Non Resource URL(s) field. You can add more than one entry using the Add item option.
      • Enter allowed list of operations in the Allowed Verbs field. You can add more than one entry using the Add item option. Alternatively, you can enter * to allow all operations on the resources.

Note: You can add more than one list of resources in case of Policy Rule List option.

  • For Aggregate Rule option, click on the Selector Expression field and set label expression by doing the following:

    • Select a key or type a custom key and click Assign Custom Key option.
    • Select an operator and select a value or type a custom value and click Assign Custom Value option.

Note: You can add more than one label expressions for the aggregate rule. This will aggregare all rules in the roles selected by the label expression.

  • Click Continue to create and assign the K8s cluster role.

Note: You can add more cluster roles using the Add item option.

Step 3.3: Configure K8s cluster role bindings.

Select K8s Cluster Role Bindings option for the K8s Cluster Role Bindings field and click on the Cluster Role Binding List field. Select a role binding from the displayed list or click Create new cluster role binding to create and attach it. This example shows creating a new cluster role binding. Configure the cluster role binding as per the following guidelines:

  • Enter a name in the metadata section.
  • Click on the K8s Cluster Role field and select the role you created in the previous step.
  • Go to Subjects section and select one of the following options for the Select Subject field:
  • Select User and enter a user in the User field.
  • Select Service Account. Enter a namespace and service account name in the Namespace and Name fields respectively.
  • Select Group and enter a group in the Group field.

Note: You can add more subjects using the Add item option.

  • Click Continue to create and assign the K8s cluster role binding.

Note: You can add more cluster role bindings using the Add item option.

Step 4: Complete creating the K8s cluster.

Click Save and Exit to complete creating the K8s cluster object.


Attach K8s Cluster to VoltStack Site

Attaching K8s cluster is possible only at the time of Perform the following steps:

Step 1: Log into VoltConsole and start creating VoltStack site.

Navigate to Manage -> Site Management and select VoltStack Sites. Click Add VoltStack Site.

Step 2: Attach K8s cluster.
  • Go to Advanced Configuration and enable Show Advanced Fields option.
  • Select Enable Site Local K8s API access for the Site Local K8s API access field.
  • Click on the Enable Site Local K8s API access field and select the K8s cluster created in the previous step.

Note: This example does not show all steps required for VoltStack site creation for brevity. For complete set of steps, see Create VoltStack Site.

Step 3: Complete creating VoltStack site.

Click Save and Exit to complete creating VoltStack site. Install nodes and complete registration for the VoltStack site. For more information, see Perform Registration chapter of the Create VoltStack Site document.

Step 4: Download the kubeconfig for the K8s cluster.
  • Navigate to Sites -> Site List. Click ... -> Kubeconfig for your VoltStack site.
  • Save the kubeconfig to your local machine.

You can use this kubeconfig for performing operations on local K8s. This is similar to the regular K8s operations using tools like kubectl.

Note: You may have to manage name resolution for your domain for K8s API access.

Step 5: Deploy applications to the managed K8s cluster.
  • Prepare a deployment manifest for your application and deploy using the kubeconfig downloaded in the previous step.
kubectl apply -f k8s-app-manifest.yaml --kubeconfig k8s-kubecfg.yaml
  • Verify deployment status.
kubectl get pods --kubeconfig k8s-kubecfg.yaml

Concepts


API References