Ingress
The Platon PaaS uses HAProxy Ingress as the cluster ingress controller. It is the component that terminates TLS and routes external HTTP traffic into your namespace, based on the Ingress resources you deploy.
If you use the default HTTP_PORT variable and do not define your own Ingress resources, you do not need to know any of this -- the kubernetes-deploy tooling generates a working Ingress for you.
The rest of this page is for projects that define their own Ingress resources.
The basics
A minimal Ingress looks like this:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-app
annotations:
kubernetes.io/tls-acme: "true"
spec:
ingressClassName: haproxy
tls:
- hosts:
- my-app.example.sikt.no
secretName: my-app-tls
rules:
- host: my-app.example.sikt.no
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-app
port:
number: 8080
An Ingress on its own is not enough -- you also need a Service, and a NetworkPolicy that allows traffic from the ingress controller. The hello-world-all-resources example project shows a complete, working set of resources.
One thing is required on the Ingress itself:
ingressClassName: haproxy-- a cluster admission policy rejects Ingress resources with any other ingress class.
Most projects will also want:
kubernetes.io/tls-acme: "true"-- makes cert-manager issue and renew the certificate named inspec.tls[].secretName. Without it, you have to create and renew that secret yourself.
Behaviour worth knowing:
- HTTP is redirected to HTTPS automatically.
- There is no request body size limit.
- Requests are not buffered.
pathTypeshould bePrefixorExact. AvoidImplementationSpecific-- HAProxy Ingress treats it differently from other controllers.
Further documentation
- HAProxy Kubernetes Ingress documentation -- the upstream docs.
- Ingress annotation reference -- the full list of annotations upstream supports. Note that only a subset is allowed in this cluster, see below.
- Backend CRD reference -- for backend behaviour that annotations cannot express.
Allowed annotations
Some annotations have caused bugs where the ingress controller gets stuck with an invalid configuration, which affects every service in the cluster. We therefore use a conservative allowlist. Annotations outside this list are rejected.
| Annotation | What it does | Default |
|---|---|---|
haproxy.org/allow-list | Comma-separated list of IPs or CIDRs allowed to reach the service. Everything else is denied. | none |
haproxy.org/auth-type | Enables basic auth. The only valid value is basic-auth. | none |
haproxy.org/auth-secret | Secret holding the credentials, as secretName. See Basic auth. | none |
haproxy.org/auth-realm | The realm shown in the browser's password prompt. | Protected Content |
haproxy.org/timeout-server | How long HAProxy waits for a response from your pod before giving up. Integer with a unit suffix, e.g. 5s, 1m. Raise this for slow endpoints. | 50s |
haproxy.org/path-rewrite | Rewrites the request path before it is sent to your service. See Path rewrites. | none |
haproxy.org/cr-backend | Attaches a Backend custom resource to the Ingress. See Backend resources. | none |
haproxy.org/rate-limit-period | The time window requests are counted over, e.g. 1s, 1m. | 1s |
haproxy.org/rate-limit-requests | Number of requests a client may make within that window. Rate limiting is off unless you set this. | none |
haproxy.org/rate-limit-size | How many distinct client IPs are tracked in memory. | 100000 |
haproxy.org/rate-limit-status-code | HTTP status returned when a client exceeds the limit. | 403 |
haproxy.org/rate-limit-whitelist | Comma-separated IPs or CIDRs exempt from rate limiting. | none |
If you need an annotation that is not on the list, contact the Platon team. We will test it and add it to the allowlist if it behaves correctly.
Enforced limits
The rate limiting annotations also have their values checked. An Ingress is rejected if a value falls outside these bounds:
| Annotation | Accepted values |
|---|---|
haproxy.org/rate-limit-period | Integer with a ms, s, m or h suffix, at most 1h |
haproxy.org/rate-limit-requests | Plain integer, 1 to 1000000 |
haproxy.org/rate-limit-size | Plain integer, 1 to 500000. Write 100000, not 100k |
haproxy.org/rate-limit-status-code | 403 or 429 |
haproxy.org/rate-limit-whitelist | Comma-separated IPs or CIDRs |
Rate limiting uses a HAProxy stick table, and its entries are allocated up front in the shared ingress controller. A large rate-limit-size therefore costs memory for every service in the cluster, not just your own.
If these bounds do not fit your use case, contact the Platon team.
Annotations on Services
These annotations are only read from Ingress resources. Setting any haproxy.org/* annotation on a Service is rejected, including the ones the upstream documentation lists as service annotations.
Service annotations such as backend-config-snippet write directly into the ingress controller's shared configuration, which is used by every service in the cluster. If you need behaviour that the upstream docs describe as a service annotation, contact the Platon team.
Basic auth
haproxy.org/auth-type: basic-auth
haproxy.org/auth-secret: my-basic-auth
haproxy.org/auth-realm: "Authentication Required"
The secret must contain one key per user, where each value is a password hashed with crypt():
kubectl -n <namespace> create secret generic my-basic-auth \
--from-literal=<username>=$(openssl passwd -5)
-5 selects SHA-256 hashing. You will be prompted for the password.
See the auth-secret documentation for details.
Path rewrites
haproxy.org/path-rewrite takes either a single replacement path, or a regex and a replacement separated by a space:
# Send requests for / to /dashboard
haproxy.org/path-rewrite: /dashboard
# Prefix every path with /new-prefix
haproxy.org/path-rewrite: (.*) /new-prefix\1
Contact the Platon team if you need help translating complex rewrite rules.
Backend resources
Some backend behaviour cannot be expressed with annotations. The most common case is sticky sessions: haproxy.org/cookie-persistence handles simple cookie-based affinity, but it cannot set cookie flags such as secure and SameSite. That requires a HAProxy Backend custom resource, attached to the Ingress with haproxy.org/cr-backend.
Backend resources are managed by the Platon team and must not be included in your deploy code. Allowing developers to deploy them is under evaluation.
To see the resources in your namespace:
kubectl -n <namespace> get backend
kubectl -n <namespace> get backend <name> -o yaml
Contact the Platon team if you need a Backend resource, or need to change the parameters of an existing one.
Known gaps
- Custom error pages / maintenance backends. There is no equivalent to nginx's
custom-http-errorsanddefault-backend. Contact the Platon team if you need to serve a maintenance page when your backends are unavailable.