Skip to main content
Version: 0.28.0

Deploying CAPA with EKS Enterprise

Weave GitOps Enterprise can leverage Cluster API providers to enable leaf cluster creation. Cluster API provides declarative APIs, controllers, and tooling to manage the lifecycle of Kubernetes clusters across a large number of infrastructure providers. Cluster API custom resource definitions (CRDs) are platform-independent as each provider implementation handles the creation of virtual machines, VPCs, networks, and other required infrastructure parts—enabling consistent and repeatable cluster deployments.

As an AWS advanced technology partner, Weaveworks has been working tirelessly to ensure that deploying EKS anywhere is smooth and removes the barriers to application modernization.

Prerequisites

You'll need to install the following software before continuing with these instructions:

  • github cli >= 2.3.0 (source)
  • kubectl (source)
  • eksctl (source)
  • the AWS Command Line Interface/aws cli (source)
  • clusterctl >= v1.1.3 (source); follow these steps to initialise the cluster and enable feature gates
  • clusterawsadm >= v1.1.0, following Cluster API's instructions
  • Make sure you have a management cluster. If you followed the Weave GitOps Enterprise installation guide, you'll have done this already.
  • Configure your AWS_ACCESS_KEY_IDand AWS_SECRET_ACCESS_KEY with either aws configure or by exporting it in the current shell.
  • Set the GITHUB_TOKEN as an environment variable in the current shell. It should have permissions to create Pull Requests against the cluster config repo.

Multitenancy

Some Cluster API providers allow you to choose the account or identity that the new cluster will be created with. This is often referred to as Multi-tenancy in the CAPI world. Weave GitOps currently supports:

1. Add Common RBAC to Your Repository

When a cluster is provisioned, by default it will reconcile all the manifests in ./clusters/<cluster-namespace>/<cluster-name> and ./clusters/bases.

To display Applications and Sources in the UI we need to give the logged in user permissions to inspect the new cluster.

Adding common RBAC rules to ./clusters/bases/rbac is an easy way to configure this!

Expand to see full template yaml
clusters/bases/rbac/wego-admin.yaml
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: wego-admin-cluster-role-binding
subjects:
- kind: User
name: wego-admin
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: wego-admin-cluster-role
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: wego-admin-cluster-role
rules:
- apiGroups: [""]
resources: ["secrets", "pods"]
verbs: ["get", "list"]
- apiGroups: ["apps"]
resources: ["deployments", "replicasets"]
verbs: ["get", "list"]
- apiGroups: ["kustomize.toolkit.fluxcd.io"]
resources: ["kustomizations"]
verbs: ["get", "list", "patch"]
- apiGroups: ["helm.toolkit.fluxcd.io"]
resources: ["helmreleases"]
verbs: ["get", "list", "patch"]
- apiGroups: ["source.toolkit.fluxcd.io"]
resources: [ "buckets", "helmcharts", "gitrepositories", "helmrepositories", "ocirepositories" ]
verbs: ["get", "list", "patch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["get", "watch", "list"]
- apiGroups: ["pac.weave.works"]
resources: ["policies"]
verbs: ["get", "list"]

2. Build a Kubernetes Platform with Built-in Components Preconfigured for Your Organization

To do this, go to Weaveworks' Profiles Catalog.

See CAPI Templates page for more details on this topic. Once we load a template we can use it in the UI to create clusters!

Download the template below to your config repository path, then commit and push to your Git origin.

clusters/management/apps/capi/templates/capa-template.yaml
apiVersion: templates.weave.works/v1alpha2
kind: GitOpsTemplate
metadata:
name: aws-eks-dev
namespace: default
annotations:
templates.weave.works/inject-prune-annotation: "true"
templates.weave.works/add-common-bases: "true"
labels:
weave.works/template-type: cluster
spec:
description: AWS EKS Development Cluster
params:
- name: CLUSTER_NAME
description: The name for this cluster.
- name: AWS_REGION
description: AWS Region to create cluster
options: ["us-east-1", "eu-central-1", "eu-west-2", "us-west-2"]
- name: KUBERNETES_VERSION
description: EKS Kubernetes version to use
options: ["v1.19.8", "v1.20.7", "v1.21.2"]
- name: WORKER_MACHINE_COUNT
description: Number of worker nodes to create.
resourcetemplates:
- contents:
- apiVersion: gitops.weave.works/v1alpha1
kind: GitopsCluster
metadata:
name: "${CLUSTER_NAME}"
namespace: default
labels:
weave.works/capi: bootstrap
spec:
capiClusterRef:
name: "${CLUSTER_NAME}"

- apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
name: ${CLUSTER_NAME}
namespace: default
labels:
weave.works/capi: bootstrap
spec:
clusterNetwork:
pods:
cidrBlocks:
- 192.168.0.0/16
controlPlaneRef:
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: AWSManagedControlPlane
name: ${CLUSTER_NAME}-control-plane
infrastructureRef:
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: AWSManagedControlPlane
name: ${CLUSTER_NAME}-control-plane

- apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: AWSManagedControlPlane
metadata:
name: ${CLUSTER_NAME}-control-plane
namespace: default
spec:
region: ${AWS_REGION}
sshKeyName: default
version: ${KUBERNETES_VERSION}
eksClusterName: ${CLUSTER_NAME}

- apiVersion: cluster.x-k8s.io/v1beta1
kind: MachinePool
metadata:
name: ${CLUSTER_NAME}-pool-0
namespace: default
spec:
clusterName: ${CLUSTER_NAME}
replicas: ${WORKER_MACHINE_COUNT}
template:
spec:
bootstrap:
dataSecretName: ""
clusterName: ${CLUSTER_NAME}
infrastructureRef:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: AWSManagedMachinePool
name: ${CLUSTER_NAME}-pool-0

- apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: AWSManagedMachinePool
metadata:
name: ${CLUSTER_NAME}-pool-0
namespace: default
spec: {}

3. Add a Cluster Bootstrap Config

This step ensures that Flux gets installed into your cluster. Create a cluster bootstrap config as follows:

 kubectl create secret generic my-pat --from-literal GITHUB_TOKEN=$GITHUB_TOKEN

Download the config with:

Then update the GITOPS_REPO variable to point to your cluster

Expand to see full yaml
clusters/management/capi/boostrap/capi-gitops-cluster-bootstrap-config.yaml
apiVersion: capi.weave.works/v1alpha1
kind: ClusterBootstrapConfig
metadata:
name: capi-gitops
namespace: default
spec:
clusterSelector:
matchLabels:
weave.works/capi: bootstrap
jobTemplate:
generateName: "run-gitops-{{ .ObjectMeta.Name }}"
spec:
containers:
- image: ghcr.io/fluxcd/flux-cli:v0.41.0
name: flux-bootstrap
resources: {}
volumeMounts:
- name: kubeconfig
mountPath: "/etc/gitops"
readOnly: true
args:
[
"bootstrap",
"github",
"--kubeconfig=/etc/gitops/value",
"--owner=$GITHUB_USER",
"--repository=fleet-infra",
"--path=./clusters/{{ .ObjectMeta.Namespace }}/{{ .ObjectMeta.Name }}",
]
envFrom:
- secretRef:
name: my-pat
restartPolicy: Never
volumes:
- name: kubeconfig
secret:
secretName: "{{ .ObjectMeta.Name }}-kubeconfig"

4. Delete a Cluster with the Weave GitOps Enterprise UI

Here are the steps:

  • Select the clusters you want to delete
  • Press the Create a PR to delete clusters button
  • Either update the deletion PR values or leave the default values, depending on your situation
  • Press the Remove clusters button
  • Merge the PR for clusters deletion

Note that you can't apply an empty repository to a cluster. If you have Cluster API clusters and other manifests committed to this repository, and then delete all of them so there are zero manifests left, then the apply will fail and the resources will not be removed from the cluster. A workaround is to add a dummy ConfigMap back to the Git repository after deleting everything else so that there is at least one manifest to apply.

5. Disable CAPI Support

If you do not need CAPI-based cluster management support, you can disable CAPI via the Helm Chart values.

Update your Weave GitOps Enterprise HelmRelease object with the global.capiEnabled value set to false:

clusters/management/weave-gitops-enterprise.yaml
---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
name: weave-gitops-enterprise-charts
namespace: flux-system
spec:
interval: 60m
secretRef:
name: weave-gitops-enterprise-credentials
url: https://charts.dev.wkp.weave.works/releases/charts-v3
---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
name: weave-gitops-enterprise
namespace: flux-system
spec:
chart:
spec:
interval: 65m
chart: mccp
sourceRef:
kind: HelmRepository
name: weave-gitops-enterprise-charts
namespace: flux-system
version: 0.12.0
install:
crds: CreateReplace
upgrade:
crds: CreateReplace
interval: 50m
values:
global:
capiEnabled: false

And that's it!