Guide
Run on Kubernetes
The Helm chart runs TelltaleDNS on any Kubernetes from 1.26, on amd64 and arm64. This guide takes you from one pod to a scaled setup with a Raspberry Pi in the same cluster.
One pod or many
mode: allInOne (the default) | mode: scaled | |
|---|---|---|
| What runs | One StatefulSet pod with a volume for lists, the query log, and users. | That pod becomes the controller, plus resolver pods that hold no state and join it by themselves. |
| Good for | A home network: one pod answers tens of thousands of queries a second. | Availability: a pod can restart or move while the others keep answering. |
| Where you manage it | The controller serves the web UI and API, keeps the volume, and publishes the configuration. Resolver pods take its configuration and compiled lists, and ship their query log to it. | |
Don't raise the replica count of the StatefulSet. Every copy would share one identity, one state.db, and one query log; the second pod refuses to start. Use mode: scaled for more pods.
Install
- Get the chart's values, every option commented. Keep only what you change in your own file. The chart's schema rejects a mistyped value with its path.
helm show values oci://ghcr.io/paimonsoror/charts/telltale --version <version> > values.yaml - Install it with your file (give the DNS Service a fixed address in it: below):
helm install telltale oci://ghcr.io/paimonsoror/charts/telltale --version <version> \ -n telltale --create-namespace -f my-values.yaml kubectl -n telltale get svc telltale-dns # EXTERNAL-IP: hand this out in DHCP - Create your admin: open the web UI and use the setup token from the install notes, or create the first admin from a Secret with
auth.bootstrapAdmin.existingSecret(keysusername, andpasswordorpassword-hash).
Every value, with its default, is on the Helm values page. Use a release version, or an edge build that follows main.
Real client addresses
Per-device statistics, groups, and rules need each query's real sender. The chart's DNS Service is a LoadBalancer with externalTrafficPolicy: Local, which keeps it; give it a fixed IP with your load balancer's annotation:
MetalLB
service:
dns:
annotations:
metallb.universe.tf/loadBalancerIPs: 192.168.1.53Cilium
service:
dns:
annotations:
io.cilium/lb-ipam-ips: 192.168.1.53kube-vip
service:
dns:
annotations:
kube-vip.io/loadbalancerIPs: 192.168.1.53- Or answer on the node itself:
hostNetwork: truebinds port 53 of the node (no LoadBalancer, and MAC addresses are visible). On Ubuntu nodes, turn off systemd-resolved's stub listener first. - If every query seems to come from a few addresses, the web UI says Client IPs appear masked. The usual cause is
externalTrafficPolicy: Cluster.
One IP for DNS, the web UI, and the cluster port
With externalTrafficPolicy: Local, two Services can share an IP only if they select the same pods. In mode: scaled the DNS Service also selects the resolver pods, so don't give the API Service the DNS Service's IP with a sharing key: DNS would stop on that address. Let the DNS Service carry the other ports instead:
service:
dns:
alsoServe:
api: true # the web UI and API on the DNS address (only the controller has them)
cluster: true # the cluster port, for a node outside Kubernetes
Or keep the web UI on an ingress: ingress.enabled: true with ingress.host, and add your ingress's network to [api] trusted_proxies under config: so the audit log sees real addresses.
Configuration and secrets
- Your settings go under
config:, written after the chart's own (listeners, data directory, ports): upstreams, lists, groups, sign-in, everything in the configuration reference. - With Argo CD or Flux, add
[cluster] config_source = "gitops". Changes in the web UI then show the TOML to commit: see Git, per node. - Secrets as files:
secretMountsmounts Secrets at/etc/telltale-secrets/<name>, for settings that take a file, such as an OIDCclient_secret_fileor a router'sapi_key_file. - With Argo CD, which can't keep a generated Secret stable, create the resolver pods' join Secret yourself and set
cluster.bootstrapSecret.existingSecret(at least 32 random characters).
secretMounts:
- name: oidc
secretName: telltale-oidc
config: |
[cluster]
config_source = "gitops"
[api]
trusted_proxies = ["10.0.0.0/8"]
[auth.oidc]
public_url = "https://dns.example.com"
[[auth.oidc.provider]]
id = "authentik"
issuer = "https://auth.example.com/application/o/telltale/"
client_id = "..."
client_secret_file = "/etc/telltale-secrets/oidc/client_secret"
More pods
mode: scaled
resolvers:
replicas: 3
- Spread them: with
externalTrafficPolicy: Local, each Kubernetes node sends queries only to its own pods. Useresolvers.affinity(podAntiAffinity onapp.kubernetes.io/component: resolver) to put them on different nodes. The Cluster page's Load balance check flags a pod taking more than twice its share. - Each pod has its own cache and rate limits. A new pod starts cold, and a client spread over three pods can send three times the per-client limit.
- Rollouts don't lose lookups: on shutdown a pod reports not ready, then keeps answering for
drainDelaySeconds(5) while the Service moves away. A pod that shuts down leaves the cluster's node list at once; one that crashes expires afterresolvers.ephemeralTtlSeconds. - Resolver pods are trusted less than the controller: they never get the cluster's signing key, can't be promoted, and can't change anything on other nodes. Make changes on the controller.
- One Kubernetes node is still one machine. For protection against that node going down, run pods on several nodes, or add a Pi as a second DNS server (next).
A Pi in the same cluster
A node outside Kubernetes joins the controller like any cluster member, and makes a good second DNS server for DHCP.
- Expose the cluster port (9443 in the chart):
service.dns.alsoServe.cluster: true, orcluster.service.type: LoadBalancer. List that address incluster.advertise, e.g.["https://192.168.1.53:9443"]. - Make a join token inside the controller pod, naming that URL:
kubectl -n telltale exec sts/telltale -- telltale cluster token create --url https://192.168.1.53:9443 \ -c /etc/telltale/00-chart.toml -c /etc/telltale/10-values.toml -c /etc/telltale/20-controller.toml - Join from the Pi, then restart it:
telltale cluster join tt_join_... --site home-pi. - Optional: staged rollouts.
cluster.rollout.canaries: [ephemeral]gives each change to the resolver pods (and the controller) first, and to the Pi after a five-minute bake, pinning the cluster back if the pods start failing.
The Pi keeps its own [cache], listeners, and sign-in settings in its own file: see Shared or per node.
Your pods as devices
TelltaleDNS can name your cluster's pods by the workload that runs them, like any laptop or TV on the network: media/deploy/jellyfin for the Deployment jellyfin in media. A Deployment's replicas are one device across every rollout, so its top domains, anomaly baseline, rhythm, and day on the timeline carry on when its pods are replaced. Namespaces get groups, so lists, quick rules, schedules, and blocked services apply per namespace. See what it looks like.
- Turn it on with CoreDNS's address (
kubectl -n kube-system get svc kube-dns -o jsonpath='{.spec.clusterIP}') and a fixed address for the DNS Service inside the Service network:
The controller gets a token that can only read pods and namespaces; resolver pods get none.kubernetes: enabled: true clusterDns: 10.43.0.10 podCidr: 10.42.0.0/16 # optional: PTR lookups for pods and Services go to CoreDNS too serviceCidr: 10.43.0.0/16 service: dns: clusterIP: 10.43.0.53 - Send the pods' DNS to TelltaleDNS. For the whole cluster, point the kubelet at it. On k3s, in
/etc/rancher/k3s/config.yamlon every node, then restart k3s and your workloads:
Usekubelet-arg: - "cluster-dns=10.43.0.53"kubelet-arg, not k3s's owncluster-dnsoption, which moves CoreDNS's Service too. Or, for one workload, setdnsPolicy: NonewithdnsConfig.nameservers: ["10.43.0.53"]and the cluster search domains. - Group them. Label a namespace (
kubectl label namespace media telltaledns.io/group=media) or a pod the same way, list rules inkubernetes.map([{namespace: "media*", group: "iot"}]), or use Add to group… on a workload in the web UI. Labels win, then rules, thenkubernetes.defaultGroup, then the pod's network group.
Through CoreDNS, every pod looks like CoreDNS. If CoreDNS forwards to TelltaleDNS instead, all you see is CoreDNS's address. The API server being unreachable never affects answering: pods keep their last names, and the health level turns degraded after 15 minutes.
Monitoring and encrypted DNS
- Prometheus and Grafana:
serviceMonitor.enabledscrapes the controller and every resolver pod,prometheusRule.enabledadds alert rules (down, all upstreams down, SERVFAIL rate, stale lists, masked client IPs, cluster health), andgrafanaDashboard.enabledships the dashboard for Grafana's sidecar. - Who may ask:
networkPolicy.enabledlimits who may query (dnsFrom), reach the UI (apiFrom), and scrape (metricsFrom). - Encrypted DNS for your devices:
encrypted.dot.enabled(853),encrypted.doh.enabled(443, plusencrypted.doh.http3), andencrypted.doq.enabledadd DoT, DoH, and DoQ to the DNS Service, with the certificate fromencrypted.tls.secretNameor a cert-manager issuer (encrypted.tls.certManager.issuerRef). It's reloaded when renewed.