Kubernetes — Ingress External Authentication with OAuth2 Proxy and Keycloak

Hi, in this article which is related with Kubernetes Ingress External Authentication with Oauth2 Proxy and Keycloak, I am going to explain below subjects and answer questions.

• What is External Authentication and Benefits of using it
• Overview of External Authentication Architecture
• Implementing External Authentication Mechanisms with Keycloak and Github Auth Provider examples

What is External Authentication and Benefits of Using It

Kubernetes Ingress external authentication is a mechanism that enables authentication for incoming requests to services deployed within a Kubernetes cluster through an Ingress controller. It allows you to enforce authentication before granting access to your applications, providing an additional layer of security and control.

Benefits of using Kubernetes Ingress external authentication:

1. Centralized Authentication: With external authentication, you can centralize the authentication logic for multiple services deployed within your Kubernetes cluster. This eliminates the need for individual authentication mechanisms for each service.
2. Flexible Authentication Providers: Kubernetes Ingress supports various authentication providers, including OAuth2 providers like Google, GitHub, or OpenID Connect providers. This flexibility allows you to leverage existing authentication systems or choose the one that best suits your needs.
3. Simplified User Experience: External authentication allows users to authenticate using their existing credentials from well-known providers. This simplifies the user experience, as users don’t need to create new accounts or remember additional usernames and passwords.
4. Centralized Management and Configuration: Ingress controllers provide a centralized point for managing and configuring external authentication rules for all the services within your cluster. This simplifies the management and ensures consistency across your applications.
5. Integration with Existing Identity Providers: If you already have an identity provider or authentication system in place, Kubernetes Ingress external authentication allows you to integrate seamlessly with it, leveraging your existing user management infrastructure.

Overview Of External Authentication Architecture

Architecture of External Authentication

• Step-1: Client sends a request to the protected app.
• Step-2: The Ingress controller receives the request and examines it to determine if it requires authentication. It checks for any authentication-related headers, cookies, or session information. If the request does not contain any authentication information, the Ingress determines that the client needs to be authenticated. It forwards the request to the OAuth2 proxy with including endpoint of requested URL.
• Step-3: Oauth2-proxy starts the authentication flow using the authentication provider related configuration, in this case Keycloak, it forwards the request to Keycloak for login process.
• Step-4: User enters the credentials and then Keycloak authenticates the user and sends the the authenticated response with headers back to the OAuth2-proxy.
• Step-5: If OAuth2-proxy configured to store session information in the cache, the data will be saved into Redis instance.
• Step-6: OAuth2-proxy redirects to the Ingress with response headers.
• Step-7–8–9: Ingress redirects to client to protected app with response headers. Protected app can perform additional authorization checks using the header info and responds back to the user with the requested resource including the cookie.

Implementing External Authentication Mechanisms with Keycloak and Github Auth Provider Examples

For both examples creating of ingress resource and OAuth2-proxy instance steps are same. I only change the OAuth2-proxy configuration via config map resource respect to each case.

Deploy your ingress resource and protected app with below yaml files.

1-) Ingress Resource

Create an ingress resource with data in below. Be carefull while setting “auth-signin” and “auth-url” annotations. “$host” parameter represents your IP to access OAuth2-proxy instance. “rd=$escaped_request_uri” query is represent your redirect URL after successfully authentication.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/auth-signin: http://$host/oauth2/start?rd=$escaped_request_uri
    nginx.ingress.kubernetes.io/auth-url: http://$host/oauth2/auth
    nginx.ingress.kubernetes.io/proxy-buffer-size: 512k
  name: nginx-ingress
  namespace: nginx-test
spec:
  ingressClassName: nginx
  rules:
  - host: foo.bar.com
    http:
      paths:
      - backend:
          service:
            name: nginx-service
            port:
              number: 80
        path: /index.html
        pathType: Prefix

2-) Example Application Resource

This application deployment represent our protected application in the diagram. You can find deployment resource, its service resource and config map in below. Sample html file injected as config map.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: nginx-test
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80
        volumeMounts:
        - name: html-files
          mountPath: /usr/share/nginx/html
      volumes:
      - name: html-files
        configMap:
          name: html-files-configmap
          items:
          - key: index.html
            path: index.html
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  namespace: nginx-test
spec:
  clusterIP: 10.101.95.46
  clusterIPs:
  - 10.101.95.46
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: ClusterIP
---
apiVersion: v1
data:
  index.html: |
    <!DOCTYPE html>
    <html>
    <head>
        <title>Lets meet the CptTeam</title>
    </head>
    <body>
        <h1>May the Force be with you!</h1>
        <p>Master Yoda greets you padawans!</p>
    </body>
    </html>
kind: ConfigMap
metadata:
  name: html-files-configmap
  namespace: nginx-test

3-) Redis Deployment

Redis needed to store session data.You can deploy redis instance with template in below.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  namespace: oauth2-proxy
spec:
  selector:
    matchLabels:
      app: redis
  replicas: 1
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
        - name: redis
          image: redis:latest
          ports:
            - containerPort: 6379
          resources:
            limits:
              cpu: "0.5"
              memory: "512Mi"
          command: ["redis-server"]
          args: ["--save", "", "--appendonly", "no"]

---
apiVersion: v1
kind: Service
metadata:
  name: redis-service
  namespace: oauth2-proxy
spec:
  selector:
    app: redis
  ports:
    - protocol: TCP
      port: 6379
      targetPort: 6379

4-) OAuth2 Proxy Deployment

To deploy OAuth2-proxy intance to manage OAuth2 workflow, you can use template in below. Before deploy it, firstly create your config map according to your use case which are based on Github Auth Provider or Keycloak Auth Provider(. After successfully creation of config map you can deploy OAuth2 instance.

apiVersion: v1
kind: Service
metadata:
  labels:
    app: oauth-proxy
  name: oauth-proxy
  namespace: oauth2-proxy
spec:
  type: ClusterIP
  selector:
    app: oauth-proxy
  ports:
  - name: http-oauthproxy
    port: 4180
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: oauth-proxy
  name: oauth-proxy
  namespace: oauth2-proxy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: "oauth-proxy"
  template:
    metadata:
      labels:
        app: oauth-proxy
    spec:
      volumes:
        - name: oauth2-proxy-config
          configMap:
            name: oauth2-proxy-config
      containers:
      - name: oauth-proxy
        image: "quay.io/oauth2-proxy/oauth2-proxy:latest"
        ports:
        - containerPort: 4180
        volumeMounts:
        - name: oauth2-proxy-config
          mountPath: /etc/oauth2-proxy.cfg
          subPath: oauth2-proxy.cfg
        args:
          - --config=/etc/oauth2-proxy.cfg
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: oauth2-proxy
  namespace: oauth2-proxy
spec:
  ingressClassName: nginx
  rules:
  - host: foo.bar.com
    http:
      paths:
      - backend:
          service:
            name: oauth-proxy
            port:
              number: 4180
        path: /oauth2
        pathType: Prefix

5-) OAuth2 Proxy Configuration for Keycloak Auth Provider

apiVersion: v1
data:
  oauth2-proxy.cfg: |-
    # Provider config
    provider="keycloak"
    provider_display_name="Keycloak"
    login_url="http://<KeycloakAddress>/realms/<KeycloakRealmName>/protocol/openid-connect/auth"
    redeem_url="http://<KeycloakAddress>/realms/<KeycloakRealmName>/protocol/openid-connect/token"
    validate_url="http://<KeycloakAddress>/realms/<KeycloakRealmName>/protocol/openid-connect/userinfo"
    profile_url="http://<KeycloakAddress>/realms/<KeycloakRealmName>/protocol/openid-connect/userinfo"
    ssl_insecure_skip_verify=true
    # Client config
    client_id="<ClientName>"
    client_secret="<ClientSecret>"
    cookie_secret="ZzBkN000Wm0pQkVkKUhzMk5YPntQRUw_ME1oMTZZTy0="
    cookie_secure="false"
    # Upstream config
    http_address="0.0.0.0:4180"
    upstreams="file:///dev/null"
    email_domains=["*"]
    oidc_issuer_url="http://<KeycloakAddress>/realms/<KeycloakRealmName>"
    cookie_domains=["foo.bar.com"]
    scope="openid"
    whitelist_domains=[".foo.bar.com:*"]
    # Redis session store config
    session_store_type="redis"
    redis_connection_url="redis://<RedisServiceName>:6379"
kind: ConfigMap
metadata:
  name: oauth2-proxy-config
  namespace: oauth2-proxy

6-) OAuth2 Proxy Configuration for Github Auth Provider

Firstly you should create Custom Github Oauth Application.

While creating consider information in below:

Homepage URL is the FQDN(Fully Qualified Domain Name) in the Ingress rule which is http://foo.bar.com

Authorization callback URL is the same as the base FQDN plus /oauth2/callback, like http://foo.bar.com/oauth2/callback

Then replace Oauth2-Proxy configuration with yaml in below.

apiVersion: v1
data:
  oauth2-proxy.cfg: |-
    # Provider config
    provider="github"
    # Client config
    client_id="<ClientName>"
    client_secret="<ClientSecret>"
    cookie_secret="<CookieSecret>"
    # Upstream config
    http_address="0.0.0.0:4180"
    upstreams="file:///dev/null"
    email_domains=["*"]
    # Redis session store config
    session_store_type="redis"
    redis_connection_url="redis://<RedisServiceName>:6379"
kind: ConfigMap
metadata:
  name: oauth2-proxy-config
  namespace: oauth2-proxy

Verification And Checking Result

After successfully deploying all components, lets check it. Call “http://foo.bar.com/index.html” URL from browser and you should redirected to Keycloak Login Page as like below.

Keycloak Login Page

After inserting credentials, you should redirected to “index.html” page which is including html content based on our config map like in below.

Sample HTML Page

You can also redis data with connecting redis pod and inserting “redis-cli -h 127.0.0.1 -p 6379” command. This command connect you to redis-cli and when you use “KEYS *” command you should see the session data key like in below.

Redis Session Data

Note: You must add “foo.bar.com” key into your “/etc/hosts” file with related IP. This IP can be one of your kubernetes nodes if you expose your ingress controller as NodePort or LoadBalancer IP that belongs to your nginx ingress controller service. In my case I am using Bare Metal Load Balancer mechanism so I use LoadBalancer IP.

Once you login, you do not need to again login after your session expired. You can configure expiration time over Keycloak.

Github : https://github.com/menendes

Linkedin : https://www.linkedin.com/in/ibrahim-halil-koyuncu-b1030516a/

References

External OAUTH Authentication - Ingress-Nginx Controller

OAuth Provider Configuration | OAuth2 Proxy

Efficiently Authenticate Your Kubernetes Applications with OAuth2 Proxy — Part 1