Skip to main content
Version: 0.38.0

Install Weave GitOps EnterpriseEnterprise

info

To purchase an entitlement to Weave GitOps Enterprise, please contact sales@weave.works.

Follow the instructions on this page to:

tip

There is no need to install the open source version of Weave GitOps before installing Weave GitOps Enterprise.

Prerequisites​

To get up and running with Weave GitOps Enterprise:

  • create a Kubernetes cluster
  • add your cluster to kubeconfig—which you'll get from Kubernetes—so that the kubeconfig correctly points to the management cluster
  • create a Git repository; in the instructions below, we refer to a fleet-infra repository
  • configure your Git client properly (if using GitHub, for example, then review their docs on setting your username and your email address)
  • obtain a valid entitlement secret from Weaveworks and apply it to your cluster
  • install a compatible version of Flux onto your cluster; see below for how-to guidance

Install the Weave GitOps Enterprise CLI Tool​

To do this, you can use either brew or curl.

brew install weaveworks/tap/gitops-ee

Install Flux Onto Your Cluster with the flux bootstrap Command​

The flux bootstrap command enables you to deploy Flux on a cluster the GitOps way. Go here for more information about the command.

flux bootstrap github \
  --owner=<github username> \
  --repository=fleet-infra \
  --branch=main \
  --path=./clusters/management \
  --personal \
  --components-extra image-reflector-controller,image-automation-controller

Your private Git repo should have a clusters/management folder that includes the manifests Flux needs to operate, and that also generates a key value pair for Flux to access the repo.

  • owner: The username (or organization) of the Git repository
  • repository: Git repository name
  • branch: Git branch (default "main")
  • path: Path relative to the repository root; when specified, the cluster sync will be scoped to this path
  • personal: If set, the owner is assumed to be a repo user
  • components-extra: Additional controllers to install

At this point your Flux management cluster should be running. Take a look at the repository you created earlier.

Apply Your Entitlements Secret to Your Cluster​

As noted above, you receive your entitlements secret by contacting sales@weave.works. Use this command to apply it to the cluster:

kubectl apply -f entitlements.yaml

Set up Authentication and RBAC​

Securing Access to the Dashboard​

There are two supported methods for logging in to the dashboard, that work with standard Kubernetes RBAC:

  • Login via an OIDC provider: recommended, as this will allow you to control permissions for existing users and groups that have already been configured to use OIDC. OIDC decouples the need to manage user lists from the application, allowing it to be managed via a central system designed for that purpose (i.e. the OIDC provider). OIDC also enables the creation of groups—either via your provider's own systems or by using a connector like Dex.
  • Login via a cluster user account: which is insecure, and which we only recommend for local and development environments or if you need to activate emergency access to a damaged cluster. However, it is an option if an OIDC provider is not available.

You may decide to give your engineering teams access to the WGE dashboard so they can view and manage their workloads. In this case, you will want to secure dashboard access and restrict who can interact with it. Weave GitOps Enterprise integrates with your OIDC provider and uses standard Kubernetes RBAC to give you fine-grained control of the dashboard users' permissions.

OIDC extends the OAuth2 authorization protocol by including an additional field (ID Token) that contains information (claims) about a user's identity. After a user successfully authenticates with the OIDC provider, Weave GitOps Enterprise uses this information to impersonate the user in any calls to the Kubernetes API. This allows cluster administrators to use RBAC rules to control access to the cluster and the dashboard.

For more specific examples of how to setup OIDC with Weave GitOps, see this guide.

If you run into issues setting up OIDC with Weave GitOps, please refer to the gitops check oidc-config command

To login via your OIDC provider, create a Kubernetes secret to store the OIDC configuration. This configuration consists of the following parameters:

ParameterDescriptionDefault
issuerURLThe URL of the issuer; typically, the discovery URL without a path
clientIDThe client ID set up for Weave GitOps in the issuer
clientSecretThe client secret set up for Weave GitOps in the issuer
redirectURLThe redirect URL set up for Weave GitOps in the issuer—typically the dashboard URL, followed by /oauth2/callback
tokenDurationThe time duration that the ID Token will remain valid after successful authentication"1h0m0s"
tokenDurationThe time duration that the ID Token will remain valid after successful authentication"1h0m0s"
oidcUsernamePrefixThe prefix added to users when impersonating API calls to the Kubernetes API, equivalent to --oidc-username-prefix
oidcGroupsPrefixThe prefix added to groups when impersonating API calls to the Kubernetes API, equivalent to --oidc-groups-prefix

Ensure that your OIDC provider has been set up with a client ID/secret and the dashboard's redirect URL.

Create a secret named oidc-auth in the flux-system namespace with these parameters set:

kubectl create secret generic oidc-auth \
  --namespace flux-system \
  --from-literal=issuerURL=<oidc-issuer-url> \
  --from-literal=clientID=<client-id> \
  --from-literal=clientSecret=<client-secret> \
  --from-literal=redirectURL=<redirect-url> \
  --from-literal=tokenDuration=<token-duration>

Once the HTTP server starts, unauthenticated users will have to click 'Login With OIDC Provider' to log in or use the cluster account (if configured). Upon successful authentication, the users' identities will be impersonated in any calls made to the Kubernetes API, as part of any action they take in the dashboard. By default the Helm chart will configure RBAC correctly, but we recommend reading the service account and user permissions pages to understand which actions are needed for Weave GitOps to function correctly.

danger

Currently, we do not have a persistent session storage, this means that if you scale to multiple replicas, logins will not be persisted.

Customization​

For some OIDC configurations, you may need to customise the requested scopes or claims.

The oidcUsernamePrefix and oidcGroupsPrefix work in the same way as the Kubernetes kube-apiserver command-line options, if you need them for Kubernetes, you will likely need them here.

Scopes​

By default, the following scopes are requested: "openid","offline_access","email","groups".

The "openid" scope is mandatory for OpenID auth and will be added if not provided. The "email" and "groups" scopes are commonly used as unique identifiers in organisations.

"offline_access" allows us to refresh OIDC tokens to keep login sessions alive for as long as a refresh token is valid. You can, however, change the defaults.

kubectl create secret generic oidc-auth \
  --namespace flux-system \
  --from-literal=issuerURL=<oidc-issuer-url> \
  --from-literal=clientID=<client-id> \
  --from-literal=clientSecret=<client-secret> \
  --from-literal=redirectURL=<redirect-url> \
  --from-literal=tokenDuration=<token-duration> \
  --from-literal=customScopes=custom,scopes

The format for the customScopes key is a comma-separated list of scopes to request. In this case, "custom", "scopes", and "openid" would be requested.

Claims​

By default, the following claims are parsed from the OpenID ID Token "email" and "groups". These are presented as the user and groups when WGE communicates with your Kubernetes API server.

This is equivalent to configuring your kube-apiserver with --oidc-username-claim=email --oidc-groups-claim=groups.

Again, you can configure these from the oidc-auth Secret.

kubectl create secret generic oidc-auth \
  --namespace flux-system \
  --from-literal=issuerURL=<oidc-issuer-url> \
  --from-literal=clientID=<client-id> \
  --from-literal=clientSecret=<client-secret> \
  --from-literal=redirectURL=<redirect-url> \
  --from-literal=tokenDuration=<token-duration> \
  --from-literal=claimUsername=sub \
  --from-literal=claimGroups=groups

There are two separate configuration keys. You can override them separately. They should match your kube-apiserver configuration.

GitOps Dashboard Service Account Permissions​

This section covers the service account permissions for the Weave GitOps application, which the WGE UI requires to work. The default permissions will generate a cluster role that includes the permissions:

rules:
- apiGroups: [""]
  resources:  ["users", "groups"] 
  verbs: [ "impersonate" ]
- apiGroups: [""]
  resources: [ "secrets" ]
  verbs: [ "get", "list" ]
- apiGroups: [ "" ]
  resources: [ "namespaces" ]
  verbs: [ "get", "list" ]

These allow the pod to do three things:

  • Impersonate the user and operate in the cluster as them
  • Read the available namespaces; this is required to understand users' permissions
  • Read the cluster-user-auth and oidc-auth secrets, the default secrets to store the emergency cluster user account and OIDC configuration (see securing access to the dashboard)

Impersonation​

The primary way Weave GitOps queries the Kube API is via impersonation. The permissions granted to users and groups that Weave GitOps can impersonate will determine the scope of actions that WGE can take within your cluster.

The application, not the cluster, authenticates the user, either via the emergency cluster user credentials or OIDC. Then it makes Kube API calls on the user's behalf. This is equivalent to making a kubectl call like:

$ kubectl get deployments --as aisha@example.com

Assuming the user aisha@example.com has permissions to get deployments within the cluster, this will return those deployments. The same occurs within the application, so properly configuring application permissions is very important. Without proper restrictions the application can impersonate very powerful users or groups. For example, the system:masters is a group generally bound to the cluster-admin role, which can do anything.

Get Namespaces​

The application itself uses get namespace permissions to pre-cache the list of available namespaces. As the user accesses resources their permissions within various namespaces is also cached to speed up future operations.

Reading the cluster-user-auth and oidc-auth Secrets​

The cluster-user-auth and oidc-auth secrets provide information for authenticating to the application. The former holds the username and bcrypt-hashed password for the emergency user, and the latter holds OIDC configuration.

The application needs to be able to access these secrets in order to authenticate users.

User Permissions​

This section discusses the Kubernetes permissions needed by Weave GitOps application users and groups. At a minimum, a User should be bound to a Role in the flux-system namespace—which is where Flux stores its resources by default—with the following permissions:

rules:
  # Flux Resources
  - apiGroups: ["source.toolkit.fluxcd.io"]
    resources: [ "buckets", "helmcharts", "gitrepositories", "helmrepositories", "ocirepositories" ]
    verbs: [ "get", "list", "watch", "patch" ]

  - apiGroups: ["kustomize.toolkit.fluxcd.io"]
    resources: [ "kustomizations" ]
    verbs: [ "get", "list", "watch", "patch" ]

  - apiGroups: ["helm.toolkit.fluxcd.io"]
    resources: [ "helmreleases" ]
    verbs: [ "get", "list", "watch", "patch" ]

  - apiGroups: [ "notification.toolkit.fluxcd.io" ]
    resources: [ "providers", "alerts" ]
    verbs: [ "get", "list", "watch", "patch" ]

  - apiGroups: ["infra.contrib.fluxcd.io"]
    resources: ["terraforms"]
    verbs: [ "get", "list", "watch", "patch" ]

  # Read access for all other Kubernetes objects
  - apiGroups: ["*"]
    resources: ["*"]
    verbs: [ "get", "list", "watch" ]

For a wider scope, the User can be bound to a ClusterRole with the same set.

On top of this you can add other permissions to view WGE resources like GitOpsSets and Templates.

Flux Resources​

The following table lists resources that Flux works with directly.

API GroupResourcesPermissions
kustomize.toolkit.fluxcd.iokustomizationsget, list, patch
helm.toolkit.fluxcd.ioHelm Releasesget, list, patch
source.toolkit.fluxcd.iobuckets, Helm charts, Git repositories, Helm repositories, OCI repositoriesget, list, patch
notification.toolkit.fluxcd.ioproviders, alertsget, list
infra.contrib.fluxcd.ioTerraformget, list, patch

Weave GitOps needs to be able to query the CRDs that Flux uses before it can accurately display Flux state. The get and list permissions facilitate this.

The patch permissions are used for two features: to suspend and resume reconciliation of a resource by modifying the 'spec' of a resource, and to force reconciliation of a resource by modifying resource annotations. These features work in the same way that flux suspend, flux resume, and flux reconcile does on the CLI.

Resources Managed via Flux​

API GroupResourcesPermissions
""configmaps, secrets, pods, services, persistent volumes, persistent volume claimsget, list, watch
appsdeployments, replica sets, stateful setsget, list, watch
batchjobs, cron jobsget, list, watch
autoscalinghorizontal pod autoscalersget, list, watch
rbac.authorization.k8s.ioroles, cluster roles, rolebindings, cluster role bindingsget, list, watch
networking.k8s.ioingressesget, list, watch

Weave GitOps reads basic resources so that it can monitor the effect that Flux has on what's running.

Reading secrets enables Weave GitOps to monitor the state of Helm releases as that's where it stores the state by default. For clarity this these are the Helm release objects not the Flux HelmRelease resource (which are dealt with by the earlier section).

Feedback from Flux​

Flux communicates the status of itself primarily via events. These events will show when reconciliations start and stop, whether they're successful, and information as to why they're not.

Login UI​

The label of the OIDC button on the login screen is configurable via a feature flag environment variable. This can give your users a more familiar experience when logging in.

Adjust the configuration in the Helm values.yaml file or the spec.values section of the Weave GitOps HelmRelease resource:

extraEnvVars:
  - name: WEAVE_GITOPS_FEATURE_OIDC_BUTTON_LABEL
    value: "Login with ACME"

This section is purposefully vague as we intend to give a broad idea of how to implement such a system. The specifics will dependent on your circumstances and goals.

Our general recommendation is to use OIDC and a small number of groups that Weave GitOps can impersonate.

Configuring Weave GitOps to impersonate Kubernetes groups rather than users has the following benefits:

  • A user's permissions for impersonation by Weave GitOps can be separate from any other permissions that they may or may not have within the cluster.
  • Users do not have to be individually managed within the cluster and can have their permissions managed together.

Example Setup​

Assume that your company has the following people in OIDC:

  • Aisha, a cluster admin, who should have full admin access to Weave GitOps
  • Brian, lead of Team-A, who should have admin permissions to their team's namespace in Weave GitOps and read-only otherwise
  • June and Jo, developers in Team-A who should have read-only access to Weave GitOps

You can then create three groups:

  • wego-admin
    • Bound to the ClusterRole, created by Helm, wego-admin-cluster-role
    • Aisha is the only member
  • wego-team-a-admin
    • Bound to a Role, using the same permissions as wego-admin-role, created in Team-A's namespace
    • Brian and Aisha are members
  • wego-readonly
    • Bound to a ClusterRole that matches wego-admin-cluster-role but with no patch permissions.
    • Aisha, Brian, June and Jo are all members
Using OIDC for cluster and Weave GitOps Authentication

If the same OIDC provider is used to authenticate a user with the cluster itself (e.g. for use with kubectl) and to Weave GitOps then, depending on OIDC configuration, they may end up with the super-set of their permissions from Weave GitOps and any other permissions granted to them.

This can lead to unintended consequences, like viewing secrets. To avoid this, OIDC providers will often let you configure which groups are returned to which clients. The Weave GitOps groups should not be returned to the cluster client (and vice versa).

Code​

The yaml to configure these permissions would look roughly like:

Expand to see example RBAC
# Admin cluster role
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: wego-admin-cluster-role
rules:
  - apiGroups: [""]
    resources: ["secrets", "pods" ]
    verbs: [ "get", "list" ]
  - apiGroups: ["apps"]
    resources: [ "deployments", "replicasets"]
    verbs: [ "get", "list" ]
  - apiGroups: ["kustomize.toolkit.fluxcd.io"]
    resources: [ "kustomizations" ]
    verbs: [ "get", "list", "patch" ]
  - apiGroups: ["helm.toolkit.fluxcd.io"]
    resources: [ "helmreleases" ]
    verbs: [ "get", "list", "patch" ]
  - apiGroups: ["source.toolkit.fluxcd.io"]
    resources: [ "buckets", "helmcharts", "gitrepositories", "helmrepositories", "ocirepositories" ]
    verbs: [ "get", "list", "patch" ]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["get", "watch", "list"]
---
# Read-only cluster role
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: wego-readonly-role
rules:
  # All the 'patch' permissions have been removed
  - apiGroups: [""]
    resources: ["secrets", "pods" ]
    verbs: [ "get", "list" ]
  - apiGroups: ["apps"]
    resources: [ "deployments", "replicasets"]
    verbs: [ "get", "list" ]
  - apiGroups: ["kustomize.toolkit.fluxcd.io"]
    resources: [ "kustomizations" ]
    verbs: [ "get", "list" ]
  - apiGroups: ["helm.toolkit.fluxcd.io"]
    resources: [ "helmreleases" ]
    verbs: [ "get", "list" ]
  - apiGroups: ["source.toolkit.fluxcd.io"]
    resources: [ "buckets", "helmcharts", "gitrepositories", "helmrepositories", "ocirepositories" ]
    verbs: [ "get", "list" ]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["get", "watch", "list"]
---
# Bind the cluster admin role to the wego-admin group
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: wego-cluster-admin
subjects:
- kind: Group
  name: wego-admin # only Aisha is a member
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: wego-admin-cluster-role
  apiGroup: rbac.authorization.k8s.io
---
# Bind the admin role in the team-a namespace for the wego-team-a-admin group
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: wego-team-a-admin-role
  namespace: team-a
subjects:
- kind: Group
  name: wego-team-a-admin # Aisha & Brian are members
  apiGroup: rbac.authorization.k8s.io
roleRef:
  # Use the cluster role to set rules, just bind them in the team-a namespace
  kind: ClusterRole
  name: wego-admin-role
  apiGroup: rbac.authorization.k8s.io
---
# Bind the read-only role to the read-only group
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: wego-readonly-role
subjects:
- kind: Group
  name: wego-readonly # Everyone is a member
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: wego-readonly-role
  apiGroup: rbac.authorization.k8s.io
---

Configure Access for Writing to Git from the Weave GitOps Enterprise UI​

Here we provide guidance for GitHub, GitLab, BitBucket Server, and Azure DevOps.

GitHub requires no additional configuration for OAuth git access

TLS Configuration​

By default, the WGE UI pod will listen on port 8000 with TLS enabled. WGE will generate and use a self-signed certificate for this purpose.

It can then be accessed via port-forwarding:

kubectl port-forward --namespace flux-system svc/clusters-service 8000:8000

If you're using an ingress controller to terminate TLS you can disable it in the Helm release:

  values:
    tls:
      enabled: false

Other ingress conguration changes can be made via the ingress configuration

  values:
    ingress:
      enabled: true
      ... other parameters specific to the ingress type ...

Configure Helm Chart and Commit​

We deploy WGE via a Helm chart. We'll save and adapt the below template before committing it in Git to a Flux-reconciled path.

Clone the newly created repo locally. We're gonna add some things!

git clone git@<provider>:<username>/fleet-infra
cd fleet-infra

Download the helm-release to clusters/management/weave-gitops-enterprise.yaml.

Expand to see file contents
clusters/management/weave-gitops-enterprise.yaml
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
  name: weave-gitops-enterprise-charts
  namespace: flux-system
spec:
  interval: 60m
  secretRef:
    name: weave-gitops-enterprise-credentials
  url: https://charts.dev.wkp.weave.works/releases/charts-v3
---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: weave-gitops-enterprise
  namespace: flux-system
spec:
  chart:
    spec:
      interval: 65m
      chart: mccp
      sourceRef:
        kind: HelmRepository
        name: weave-gitops-enterprise-charts
        namespace: flux-system
      version: 0.x.x
  install:
    crds: CreateReplace
  upgrade:
    crds: CreateReplace
  interval: 50m
  values:
    # -- Configure TLS settings if needed
    # tls:
      # -- Can be disabled if TLS is handled by a user-provided ingress controller
      # enabled: true
      # -- optionally specify a TLS secret
      # secretName: null
    config:
      capi:
        repositoryURL: https://github.com/$GITHUB_USER/fleet-infra
        # -- Can be changed depending on your git repo structure
        # repositoryPath: ./clusters/management/clusters
        # repositoryClustersPath: ./cluster
      git:
        type: github
        # -- Change if using on-prem github/gitlab
        # hostname: https://github.com

Once you have copied the above file, open and adjust the following configuration options:

values.config.capi.repositoryURL​

Ensure this has been set to your repository URL.

values.config.capi.repositoryPath​

By default, WGE will create new clusters in the clusters/management/clusters path. You can configure it with values.config.capi.repositoryPath. You might what to change it to clusters/my-cluster/cluster if you configured Flux to reconcile ./clusters/my-cluster instead.

values.config.capi.repositoryClustersPath​

The other important path to configure is where you'll store applications and workloads run on the new cluster. By default this is ./clusters. When a new cluster is specified, any selected profiles will be written to ./clusters/{.namespace}/{.clusterName}/profiles.yaml. When the new cluster is bootstrapped, Flux will sync the ./clusters/{.namespace}/{.clusterName} path.

Configure Your Password​

To login to the WGE UI, generate a bcrypt hash for your chosen password and store it as a secret in the Kubernetes cluster. There are several different ways to generate a bcrypt hash. Here, we'll use gitops get bcrypt-hash from our CLI.

PASSWORD="<Make up and insert a brand-new password here>"
echo -n $PASSWORD | gitops get bcrypt-hash | kubectl create secret generic cluster-user-auth -n flux-system --from-literal=username=wego-admin --from-file=password=/dev/stdin

A validation to know it’s working:

kubectl get secret -n flux-system cluster-user-auth

(Optional) Install Policy Agent​

Policy agent comes packaged with the WGE chart. To install it, set the following values:

  • values.policy-agent.enabled: set to true to install the agent with WGE
  • values.policy-agent.config.accountId: organization name, used as identifier
  • values.policy-agent.config.clusterId: unique identifier for the cluster

Commit and push all the files

git add clusters/management/weave-gitops-enterprise.yaml
git commit -m "Deploy Weave GitOps Enterprise"
git push

Flux will reconcile the helm-release and WGE will be deployed into the cluster. You can check the flux-system namespace to verify all pods are running.

Next Steps​

Here are a couple of options for you to take your next steps with WGE. Explore one option or all of them, in no particular order.

  • Cluster Management: We'll show you how to join WGE to a cluster and install an application on that cluster without using Cluster API. But if you prefer using Cluster API, our docs cover that too.
  • Install the Terraform Controller to reconcile your Terraform resources in a GitOps way. With Flux and the TF Controller, WGE makes it easy to add Terraform templates to your clusters and continuously reconcile any changes made to the Terraform source manifest.
  • Install Policy agent, which comes packaged with the WGE chart.