ISSUE TYPE

• Bug Report

COMPONENT NAME

Kubernetes Service

CLOUDSTACK VERSION

4.15.0

CONFIGURATION

Base install of 4.15.0 (upgraded from 4.11)
Kubernetes Service enabled
CoreOS template
community Kubernetes ISO (tried multiple from v 1.11.4 to 1.16.3)
Using Advanced Networking
User account uses Isolated Network (not L2 or Shared)
SSL is enabled for CS GUI, and System VMs

OS / ENVIRONMENT

CentOS 7 for Management Server

SUMMARY

On a freshly upgraded version to Cloudstack 4.15 (from 4.11), when I create Kubernetes Cluster (regardless of which version), the master and worker VMs are getting created and running successfully, but after the Timeout setting (default 3600 seconds) expires I see the state - "Error". Further more, under the "Access" tab, I see "Kubernetes cluster kubeconfig not available currently". I cannot download the config file/never becomes available.

This happens in Isolated Networks with source NAT enabled. I also tested on a Shared Network on a VLAN directly on the router.

It seems the VMs are getting setup but something is getting blocked when trying to check the status of the service. I have opened all ports for egress in the Isolated Network. ( I can see the data load of 200+MB taking place on the master and worker node via CS GUI)

STEPS TO REPRODUCE

EXPECTED RESULTS

Kubernetes Service should show as Active

ACTUAL RESULTS

Kubernetes Service Shows as Error