Guide

Run on Kubernetes

The Helm chart runs TelltaleDNS on any Kubernetes from 1.26, on amd64 and arm64. This guide takes you from one pod to a scaled setup with a Raspberry Pi in the same cluster.

One pod or many

mode: allInOne (the default)mode: scaled
What runsOne StatefulSet pod with a volume for lists, the query log, and users.That pod becomes the controller, plus resolver pods that hold no state and join it by themselves.
Good forA home network: one pod answers tens of thousands of queries a second.Availability: a pod can restart or move while the others keep answering.
Where you manage itThe controller serves the web UI and API, keeps the volume, and publishes the configuration. Resolver pods take its configuration and compiled lists, and ship their query log to it.

Don't raise the replica count of the StatefulSet. Every copy would share one identity, one state.db, and one query log; the second pod refuses to start. Use mode: scaled for more pods.

Install

  1. Get the chart's values, every option commented. Keep only what you change in your own file. The chart's schema rejects a mistyped value with its path.
    helm show values oci://ghcr.io/paimonsoror/charts/telltale --version <version> > values.yaml
  2. Install it with your file (give the DNS Service a fixed address in it: below):
    helm install telltale oci://ghcr.io/paimonsoror/charts/telltale --version <version> \
      -n telltale --create-namespace -f my-values.yaml
    kubectl -n telltale get svc telltale-dns     # EXTERNAL-IP: hand this out in DHCP
  3. Create your admin: open the web UI and use the setup token from the install notes, or create the first admin from a Secret with auth.bootstrapAdmin.existingSecret (keys username, and password or password-hash).

Every value, with its default, is on the Helm values page. Use a release version, or an edge build that follows main.

Real client addresses

Per-device statistics, groups, and rules need each query's real sender. The chart's DNS Service is a LoadBalancer with externalTrafficPolicy: Local, which keeps it; give it a fixed IP with your load balancer's annotation:

MetalLB

service:
  dns:
    annotations:
      metallb.universe.tf/loadBalancerIPs: 192.168.1.53

Cilium

service:
  dns:
    annotations:
      io.cilium/lb-ipam-ips: 192.168.1.53

kube-vip

service:
  dns:
    annotations:
      kube-vip.io/loadbalancerIPs: 192.168.1.53
  • Or answer on the node itself: hostNetwork: true binds port 53 of the node (no LoadBalancer, and MAC addresses are visible). On Ubuntu nodes, turn off systemd-resolved's stub listener first.
  • If every query seems to come from a few addresses, the web UI says Client IPs appear masked. The usual cause is externalTrafficPolicy: Cluster.

One IP for DNS, the web UI, and the cluster port

With externalTrafficPolicy: Local, two Services can share an IP only if they select the same pods. In mode: scaled the DNS Service also selects the resolver pods, so don't give the API Service the DNS Service's IP with a sharing key: DNS would stop on that address. Let the DNS Service carry the other ports instead:

service:
  dns:
    alsoServe:
      api: true        # the web UI and API on the DNS address (only the controller has them)
      cluster: true    # the cluster port, for a node outside Kubernetes

Or keep the web UI on an ingress: ingress.enabled: true with ingress.host, and add your ingress's network to [api] trusted_proxies under config: so the audit log sees real addresses.

Configuration and secrets

  • Your settings go under config:, written after the chart's own (listeners, data directory, ports): upstreams, lists, groups, sign-in, everything in the configuration reference.
  • With Argo CD or Flux, add [cluster] config_source = "gitops". Changes in the web UI then show the TOML to commit: see Git, per node.
  • Secrets as files: secretMounts mounts Secrets at /etc/telltale-secrets/<name>, for settings that take a file, such as an OIDC client_secret_file or a router's api_key_file.
  • With Argo CD, which can't keep a generated Secret stable, create the resolver pods' join Secret yourself and set cluster.bootstrapSecret.existingSecret (at least 32 random characters).
secretMounts:
  - name: oidc
    secretName: telltale-oidc
config: |
  [cluster]
  config_source = "gitops"

  [api]
  trusted_proxies = ["10.0.0.0/8"]

  [auth.oidc]
  public_url = "https://dns.example.com"

  [[auth.oidc.provider]]
  id = "authentik"
  issuer = "https://auth.example.com/application/o/telltale/"
  client_id = "..."
  client_secret_file = "/etc/telltale-secrets/oidc/client_secret"

More pods

mode: scaled
resolvers:
  replicas: 3
  • Spread them: with externalTrafficPolicy: Local, each Kubernetes node sends queries only to its own pods. Use resolvers.affinity (podAntiAffinity on app.kubernetes.io/component: resolver) to put them on different nodes. The Cluster page's Load balance check flags a pod taking more than twice its share.
  • Each pod has its own cache and rate limits. A new pod starts cold, and a client spread over three pods can send three times the per-client limit.
  • Rollouts don't lose lookups: on shutdown a pod reports not ready, then keeps answering for drainDelaySeconds (5) while the Service moves away. A pod that shuts down leaves the cluster's node list at once; one that crashes expires after resolvers.ephemeralTtlSeconds.
  • Resolver pods are trusted less than the controller: they never get the cluster's signing key, can't be promoted, and can't change anything on other nodes. Make changes on the controller.
  • One Kubernetes node is still one machine. For protection against that node going down, run pods on several nodes, or add a Pi as a second DNS server (next).

A Pi in the same cluster

A node outside Kubernetes joins the controller like any cluster member, and makes a good second DNS server for DHCP.

  1. Expose the cluster port (9443 in the chart): service.dns.alsoServe.cluster: true, or cluster.service.type: LoadBalancer. List that address in cluster.advertise, e.g. ["https://192.168.1.53:9443"].
  2. Make a join token inside the controller pod, naming that URL:
    kubectl -n telltale exec sts/telltale -- telltale cluster token create --url https://192.168.1.53:9443 \
      -c /etc/telltale/00-chart.toml -c /etc/telltale/10-values.toml -c /etc/telltale/20-controller.toml
  3. Join from the Pi, then restart it: telltale cluster join tt_join_... --site home-pi.
  4. Optional: staged rollouts. cluster.rollout.canaries: [ephemeral] gives each change to the resolver pods (and the controller) first, and to the Pi after a five-minute bake, pinning the cluster back if the pods start failing.

The Pi keeps its own [cache], listeners, and sign-in settings in its own file: see Shared or per node.

Your pods as devices

TelltaleDNS can name your cluster's pods by the workload that runs them, like any laptop or TV on the network: media/deploy/jellyfin for the Deployment jellyfin in media. A Deployment's replicas are one device across every rollout, so its top domains, anomaly baseline, rhythm, and day on the timeline carry on when its pods are replaced. Namespaces get groups, so lists, quick rules, schedules, and blocked services apply per namespace. See what it looks like.

Pods ask TelltaleDNS directly; it names each one by its workload, answers or forwards, and sends the cluster's own names on to CoreDNS. The controller watches pods (read-only) to know who is who. Pods (their own addresses) jellyfin-7d9f4c8b6-x2k1q10.42.0.15 jellyfin-7d9f4c8b6-m8pqz10.42.1.22 postgres-010.42.0.21 TelltaleDNS media/deploy/jellyfin: one device db/sts/postgres: one device each in its namespace's group Upstreamsthe internet, filtered per group CoreDNScluster.local and reverse zones Kubernetes API: watch pods and namespaces (read-only)
Pods ask TelltaleDNS directly, so their own addresses arrive; Service names still come from CoreDNS.
  1. Turn it on with CoreDNS's address (kubectl -n kube-system get svc kube-dns -o jsonpath='{.spec.clusterIP}') and a fixed address for the DNS Service inside the Service network:
    kubernetes:
      enabled: true
      clusterDns: 10.43.0.10
      podCidr: 10.42.0.0/16       # optional: PTR lookups for pods and Services go to CoreDNS too
      serviceCidr: 10.43.0.0/16
    service:
      dns:
        clusterIP: 10.43.0.53
    The controller gets a token that can only read pods and namespaces; resolver pods get none.
  2. Send the pods' DNS to TelltaleDNS. For the whole cluster, point the kubelet at it. On k3s, in /etc/rancher/k3s/config.yaml on every node, then restart k3s and your workloads:
    kubelet-arg:
      - "cluster-dns=10.43.0.53"
    Use kubelet-arg, not k3s's own cluster-dns option, which moves CoreDNS's Service too. Or, for one workload, set dnsPolicy: None with dnsConfig.nameservers: ["10.43.0.53"] and the cluster search domains.
  3. Group them. Label a namespace (kubectl label namespace media telltaledns.io/group=media) or a pod the same way, list rules in kubernetes.map ([{namespace: "media*", group: "iot"}]), or use Add to group… on a workload in the web UI. Labels win, then rules, then kubernetes.defaultGroup, then the pod's network group.

Through CoreDNS, every pod looks like CoreDNS. If CoreDNS forwards to TelltaleDNS instead, all you see is CoreDNS's address. The API server being unreachable never affects answering: pods keep their last names, and the health level turns degraded after 15 minutes.

Monitoring and encrypted DNS

  • Prometheus and Grafana: serviceMonitor.enabled scrapes the controller and every resolver pod, prometheusRule.enabled adds alert rules (down, all upstreams down, SERVFAIL rate, stale lists, masked client IPs, cluster health), and grafanaDashboard.enabled ships the dashboard for Grafana's sidecar.
  • Who may ask: networkPolicy.enabled limits who may query (dnsFrom), reach the UI (apiFrom), and scrape (metricsFrom).
  • Encrypted DNS for your devices: encrypted.dot.enabled (853), encrypted.doh.enabled (443, plus encrypted.doh.http3), and encrypted.doq.enabled add DoT, DoH, and DoQ to the DNS Service, with the certificate from encrypted.tls.secretName or a cert-manager issuer (encrypted.tls.certManager.issuerRef). It's reloaded when renewed.