vRA Cloud: Request Kubernetes cluster using self service catalog item

In this article we will see how we can utilise vRealize Automation Cloud to deploy Vanilla Kubernetes Cluster using combinations of Python and PowerCli action flow
Below we get started , Here is quick look how catalog item looks like.


High Level step:

  • Complete the prerequisite of adding constant and Secrets.
  • Create Cloud Template from Yaml in article.
  • Release the Cloud template version.
  • Create PowerCli abx action .
  • Create Python Action.
  • Create ABX Flow and mapped above action .
  • Create Subscription and map the ABX action Flow.
  • Request Kubernetes cluster from Catalog.



Prerequisites :
We will be adding below details as action constant and Secrets in vRA Cloud in secret form.

  • vCenter FQDN or IP address
  • vCenter Username
  • vCenter Password
  • Template username & password
  • Ubuntu 18 template which can download the packages .
  • CSP token
  • API url Region specific- I am in US region so for me api.mgmt.cloud.com
    Below is the screenshot from my lab.
Note: I have attached 2 separate screenshot for secret because i have many
Make sure we use the same name for token,VRAURL and other constant as it is configured in code.

Once we have created above ABX action constant , we will proceed with Setting up the Cloud Template/Blueprint.

formatVersion: 1
inputs:
  masternode_CPU:
    type: integer
    enum:
      - 2
      - 4
    title: Master Node CPU Count
  masternode_memory:
    type: integer
    oneOf:
      - title: 4 GB
        const: 4096
      - title: 8 GB
        const: 8192
    title: Master Node Memory
  workerNode:
    type: integer
    default: 1
    minimum: 1
    maximum: 3
    description: Total Number of Worker Node
    title: Worker node Count
  workernode_cpu:
    type: integer
    enum:
      - 2
      - 4
    title: Worker node CPU Count
  workerNode_memory:
    type: integer
    oneOf:
      - title: 4 GB
        const: 4096
      - title: 8 GB
        const: 8192
    title: Worker Node Memory
resources:
  Cloud_vSphere_Network_1:
    type: Cloud.vSphere.Network
    properties:
      networkType: existing
  K8Worker:
    type: Cloud.vSphere.Machine
    dependsOn:
      - K8Master
    properties:
      image: Ubuntu
      cpuCount: ${input.workernode_cpu}
      count: ${input.workerNode}
      name: ${self.resourceName}
      totalMemoryMB: ${input.workerNode_memory}
      role: K8Worker
      constraints:
        - tag: vCenter155:Cluster
      networks:
        - network: ${resource.Cloud_vSphere_Network_1.id}
          assignment: static
  K8Master:
    type: Cloud.vSphere.Machine
    properties:
      image: Ubuntu
      cpuCount: ${input.masternode_CPU}
      totalMemoryMB: ${input.masternode_memory}
      role: K8Master
      name: ${self.resourceName}
      constraints:
        - tag: vCenter155:Cluster
      networks:
        - network: ${resource.Cloud_vSphere_Network_1.id}
          assignment: static

  • Line number 55 and 68 are import , it is indicating in which vCenter we want to provision.
  • Line number 58 and 71 indicate we are using static IP assignment which is import for this deployment.

Once we have done with above changes for constraint tag with above Yaml , we will release the cloud template.

Next Step is to create the PowerCli abx action which will be doing the heavy lifting of installation of all packages and Adding the worker node to Master.

$m = Get-Module -ListAvailable -Name VMware.VimAutomation.Core
$impot = import-module -ModuleInfo $m
function handler($context, $inputs) {
    #_______________________Debug and Verbose Mode Off____________________________________
    Set-PSDebug -Off
    $DebugPreference="SilentlyContinue"
    $VerbosePreference="SilentlyContinue"
    #_______________________Debug and Verbose Mode Off____________________________________
    
    #_______________________Get Secrets and Inputs____________________________________'
    $vc = $context.getSecret($inputs.vCenterIP)
    $username = $context.getSecret($inputs.vCenterUsername)
    $password = $context.getSecret($inputs.Password)
    $template_username = $context.getSecret($inputs.template_username)
    $template_password = $context.getSecret($inputs.template_password)
    #_______________________Get Secrets and Inputs____________________________________'
    
    $connection = Connect-VIServer -Server $vc -User $username -Password $password -Force
    $data = $inputs['deployment_details']
    foreach($item in $data){
        $asd += @($item|select @{n='VMNAME';E={$item.hostname}})
        
    }
    $k8masterip = $asd|?{$_.VMNAME -match "k8master"}
    $master_vm = Get-VM $k8masterip.VMNAME
 

    # '_________________________________________________________________________________________'
    if($inputs.customProperties.role -match "k8master"){
        $master_script = @'
    sudo apt-get update -y
    sudo apt-get install jq -y
    sudo apt install docker.io -y
    sudo systemctl enable docker
    sudo systemctl start  docker
    sudo ufw disable
    sudo swapoff -a
    sudo apt-get update && sudo apt-get install -y apt-transport-https
    curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
    sudo bash -c 'echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" > /etc/apt/sources.list.d/kubernetes.list'
    sudo apt-get update && sudo apt-get install -y kubelet kubeadm kubectl
    sudo systemctl enable kubelet
    sudo systemctl start kubelet
    masterip=$(hostname -I | cut -d' ' -f1)
    sudo kubeadm init --apiserver-advertise-address=$masterip
    mkdir -p /root/.kube
    sudo cp -i /etc/kubernetes/admin.conf /root/.kube/config
    sudo chown $(id -u):$(id -g) /root/.kube/config
    wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
    kubectl apply -f kube-flannel.yml
'@
        $result =Invoke-VMScript -VM $master_vm -ScriptType bash -ScriptText $master_script -GuestUser $template_username -GuestPassword $template_password
        Write-Host $result.ScriptOutput
    }
    elseif($inputs.customProperties.role -match "k8worker"){
        $worker_vms = Get-VM $inputs.resourceNames
        Write-Host $worker_vms
        $worker_script = @'
    sudo apt-get update -y
    sudo apt-get install jq -y
    sudo apt install docker.io -y
    sudo systemctl enable docker
    sudo systemctl start  docker
    sudo ufw disable
    sudo swapoff -a
    sudo apt-get update && sudo apt-get install -y apt-transport-https
    curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
    sudo bash -c 'echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" > /etc/apt/sources.list.d/kubernetes.list'
    sudo apt-get update && sudo apt-get install -y kubelet kubeadm kubectl
    sudo systemctl enable kubelet
    sudo systemctl start kubelet
'@
        $Cluster_join = @'
    CERT_HASH=$(openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null| openssl dgst -sha256 -hex | sed 's/^.* //')
    TOKEN=$(kubeadm token list -o json | jq -r '.token' | head -1)
    PORT=6443
    IP=$(kubectl get nodes -lnode-role.kubernetes.io/control-plane -o json | jq -r '.items[0].status.addresses[] | select(.type=="InternalIP") | .address')
    echo "sudo kubeadm join $IP:$PORT --token=$TOKEN --discovery-token-ca-cert-hash sha256:$CERT_HASH"
'@
        $result2 =Invoke-VMScript -VM $master_vm -ScriptType bash -ScriptText $Cluster_join -GuestUser $template_username -GuestPassword $template_password
        $hash = @{“k8join” = $result2.ScriptOutput}
        $worker_result = Invoke-VMScript -VM $worker_vms -ScriptType bash -ScriptText $worker_script -GuestUser $template_username -GuestPassword $template_password
        #'____________________________________Worker Node Result_____________________________________________________'
        Write-Host $worker_result.ScriptOutput
        #'____________________________________Worker Node Result_____________________________________________________'
        $worker_script2= $hash.k8join
        $worker_result2 = Invoke-VMScript -VM $worker_vms -ScriptType bash -ScriptText $worker_script2 -GuestUser $template_username -GuestPassword $template_password
        Write-Host $worker_result2.ScriptOutput
        # '____________________________________Worker Node Result_____________________________________________________'
    }
}

I have saved the action with Kubernetes Installation.

Once we have saved the PowerCli ABX action , We will proceed for Python Action.

import json
import requests
import time
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)


def handler(context, inputs):
    cloudurl = context.getSecret(inputs['VRAURL'])
    print(cloudurl)
    token = context.getSecret(inputs['token'])
    Outputs = {}
    deploymentid = inputs['deploymentId']
    print(f'Deployment ID from EBS fetched: {deploymentid}')
    start = time.time()
    api_version_url = f'{cloudurl}/iaas/api/about'
    headers = {
        'accept': "application/json",
        'content-type': "application/json"
    }
    output = requests.get(url=api_version_url, headers=headers, verify=False)
    if output.status_code == 200:
        latest_api_version = output.json()['latestApiVersion']
        apiversion = latest_api_version
        iaasUrl = f"{cloudurl}/iaas/api/login?apiVersion={apiversion}"
        refreshtoken = token
        iaasPayload = f'{{"refreshToken": "{refreshtoken}"}}'
        iaasApiOutput = requests.post(iaasUrl, data=iaasPayload, headers=headers, verify=False)
        if iaasApiOutput.status_code == 200:
            print('Authentication to CLoud instance Completed')
            jsondata = iaasApiOutput.json()['token']
            bearerToken = "Bearer " + jsondata
            bearertoken = bearerToken
            end = time.time()

            headers = {
                'accept': "application/json",
                'content-type': "application/json",
                'authorization': bearertoken
            }
            deployment_url = f'{cloudurl}/deployment/api/deployments/{deploymentid}' \
                             f'/resources?resourceTypes=Cloud.vSphere.Machine&apiVersion={apiversion}'
            deployment_output = requests.get(deployment_url, headers=headers, verify=False)
            if deployment_output.status_code == 200:
                print('Deploymnet details API response is success, starting the payload reading ....')
                data = deployment_output.json()['content']
                print(data)
                data_dict = {}
                data_list = []
                for i in data:
                    data_dict['id'] = i['id']
                    data_dict['name'] = i['name']
                    data_dict['VMIP'] = i['properties']['networks'][0]['address']
                    data_dict['hostname'] = i['properties']['resourceName']
                    data_list.append(data_dict.copy())
                timetaken = end - start
                print(f'Time taken to collect the data: {timetaken}')
                print(data_list)
                Outputs['deployment_details']= data_list
                return Outputs
            else:
                print(f'Deployment API result is : {deployment_output.status_code}')
                if deployment_output.json():
                    print(deployment_output.json())

        else:
            print(f'Authentication API result is: {iaasApiOutput.status_code}')
            if iaasApiOutput.json():
                return iaasApiOutput.json()
    else:
        print(output.status_code)

Once we have created PowerCli and Python Action , we will map into Flow as per below.

we will create new subscription and will map above Created Flow

Subscription topic: Compute Post provision. –compute.provision.post

If you have followed the steps till now , we are ready to provision our Kubernetes Cluster using catalog item .

2 responses to “vRA Cloud: Request Kubernetes cluster using self service catalog item”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: