AWX Operator

Purpose: Deploying a Rancher RKE2 Cluster-based Ansible AWX Operator server. This can scale to a larger more enterprise environment if needed.

Deploy Rancher RKE2 Cluster¶

You will need to deploy a Rancher RKE2 Cluster on an Ubuntu Server-based virtual machine. After this phase, you can focus on the Ansible AWX-specific deployment. A single ControlPlane node is all you need to set up AWX, additional infrastructure can be added after-the-fact.

Server Configuration¶

The AWX deployment will consist of 3 yaml files that configure the containers for AWX as well as the NGINX ingress networking-side of things. You will need all of them in the same folder for the deployment to be successful. For the purpose of this example, we will put all of them into a folder located at /awx.

# Make the deployment folder
mkdir -p /awx
cd /awx

We need to increase filesystem access limits: Temporarily Set the Limits Now:

sudo sysctl fs.inotify.max_user_watches=524288
sudo sysctl fs.inotify.max_user_instances=512

Permanently Set the Limits for Later:

# <End of File>
fs.inotify.max_user_watches = 524288
fs.inotify.max_user_instances = 512

Apply the Settings:

sudo sysctl -p

Create AWX Deployment Donfiguration Files¶

You will need to create these files all in the same directory using the content of the examples below. Be sure to replace values such as the spec.host=awx.bunny-lab.io in the awx-ingress.yml file to a hostname you can point a DNS server / record to.

apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  service_type: ClusterIP

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress
spec:
  rules:
  - host: awx.bunny-lab.io
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: awx-service
            port:
              number: 80

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - github.com/ansible/awx-operator/config/default?ref=2.10.0
  - awx.yml
  - ingress.yml
images:
  - name: quay.io/ansible/awx-operator
    newTag: 2.10.0
namespace: awx

Ensure the Kubernetes Cluster is Ready¶

Check that the status of the cluster is ready by running the following commands, it should appear similar to the Rancher RKE2 Example:

export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
kubectl get pods --all-namespaces

Ensure the Timezone / Date is Accurate¶

You want to make sure that the Kubernetes environment and Node itself have accurate time for a number of reasons, least of which, is if you are using Ansible with Kubernetes authentication, if the date/time is inaccurate, things will not work correctly.

sudo timedatectl set-timezone America/Denver

Deploy AWX using Kustomize¶

Now it is time to tell Kubernetes to read the configuration files using Kustomize (built-in to newer versions of Kubernetes) to deploy AWX into the cluster.

error: resource mapping not found for name: "awx" namespace: "awx" from ".": no matches for kind "AWX" in version "awx.ansible.com/v1beta1"
ensure CRDs are installed first

NAME                                               READY   STATUS    RESTARTS        AGE
awx-operator-controller-manager-7b9ccf9d4d-cnwhc   2/2     Running   2 (3m41s ago)   9m41s
awx-postgres-13-0                                  1/1     Running   0               6m12s
awx-task-7b5f8cf98c-rhrpd                          4/4     Running   0               4m46s
awx-web-6dbd7df9f7-kn8k2                           3/3     Running   0               93s

cd /awx
kubectl apply -k .

Access the AWX WebUI behind Ingress Controller¶

After you have deployed AWX into the cluster, it will not be immediately accessible to the host's network (such as your personal computer) unless you set up a DNS record pointing to it. In the example above, you would have an A or CNAME DNS record pointing to the internal IP address of the Rancher RKE2 Cluster host.

The RKE2 Cluster will translate awx.bunny-lab.io to the AWX web-service container(s) automatically due to having an internal Reverse Proxy within the Kubernetes Cluster. SSL certificates generated within Kubernetes/Rancher RKE2 are not covered in this documentation, but suffice to say, the AWX server can be configured on behind another reverse proxy such as Traefik or via Cert-Manager / JetStack. The process of setting this up goes outside the scope of this document.

Traefik Implementation¶

If you want to put this behind traefik, you will need a slightly unique traefik configuration file, seen below, to effectively transparently passthrough traffic into the RKE2 Cluster's reverse proxy.

tcp:
  routers:
    awx-tcp-router:
      rule: "HostSNI(`awx.bunny-lab.io`)"
      entryPoints: ["websecure"]
      service: awx-nginx-service
      tls:
        passthrough: true
#      middlewares:
#        - auth-bunny-lab-io  # Referencing the Keycloak Server

  services:
    awx-nginx-service:
      loadBalancer:
        servers:
          - address: "192.168.3.10:443"

Change Admin Password¶

You will want to change the admin password straight-away. Use the following navigation structure to find where to change the password:

graph LR
    A[AWX Dashboard] --> B[Access]
    B --> C[Users]
    C --> D[admin]
    D --> E[Edit]

Troubleshooting¶

You may wish to want to track the deployment process to verify that it is actually doing something. There are a few Kubernetes commands that can assist with this listed below.

AWX-Manager Deployment Logs¶

You may want to track the internal logs of the awx-manager container which is responsible for the majority of the automated deployment of AWX. You can do so by running the command below.

kubectl logs -n awx awx-operator-controller-manager-6c58d59d97-qj2n2 -c awx-manager