PowerDNS Training May 2026

• Authoritative DNS server, such as the PowerDNS server, can be deployed in various different setups

• For security reasons, it sometimes not desirable to have the zone data write-able on a server that is exposed to the Internet
  • In such situations, a hidden primary setup can be useful
  • In an hidden primary setup, the server hosting the primary zone is located inside a protected network (often behind a firewall)
  • The public secondaries are exposed to the Internet, but as secondaries the zone data is read-only.
  • In a hidden primary setup, the delegation and in-zone NS-records do only list the publicly reachable secondaries
    • As an option, the domain name of the hidden primary can be listed in the SOA record MNAME field to support synamic updates to the zone

  

• For DNSSEC signed zones, the hidden master could also act as a DNSSEC signer, adding DNSSEC data to the zones
  • The precious DNSSEC keys are not exposed to the Internet
  • Non-DNSSEC aware DNS management software can be augmented with DNSSEC capabilities using such a hidden signer (also known as a bump-in-the-wire signer.

  

• In the hands-on lab sessions for this training, we will be installing the PowerDNS server from the Debain 13 repositories

apt-get install pdns-server

• PowerDNS supports a range of different backend (SQL Database, BIND 9, LDAP, Lua, Pipe etc). To list the available PowerDNS backends (on Debian 13):

# apt search pdns-backend
Sorting... Done
Full Text Search... Done
pdns-backend-bind/stable,now 4.4.1-1 amd64 [installed,automatic]
  BIND backend for PowerDNS

pdns-backend-geoip/stable 4.4.1-1 amd64
  GeoIP backend for PowerDNS

pdns-backend-ldap/stable 4.4.1-1 amd64
  LDAP backend for PowerDNS

pdns-backend-lmdb/stable 4.4.1-1 amd64
  LMDB backend for PowerDNS

pdns-backend-lua2/stable 4.4.1-1 amd64
  Lua2 backend for PowerDNS

pdns-backend-mysql/stable 4.4.1-1 amd64
  MySQL backend for PowerDNS

pdns-backend-odbc/stable 4.4.1-1 amd64
  UnixODBC backend for PowerDNS

pdns-backend-pgsql/stable 4.4.1-1 amd64
  PostgreSQL backend for PowerDNS

pdns-backend-pipe/stable 4.4.1-1 amd64
  pipe/coprocess backend for PowerDNS

pdns-backend-remote/stable 4.4.1-1 amd64
  remote backend for PowerDNS

pdns-backend-sqlite3/stable 4.4.1-1 amd64
  sqlite 3 backend for PowerDNS

pdns-backend-tinydns/stable 4.4.1-1 amd64
  tinydns compatibility backend for PowerDNS

• Select and install once or more of the available backends. In this example we're installing the MySQL/MariaDB backend:

% apt install pdns-backend-mysql

• After installation, the PowerDNS server should be up and running with a default configuration. To check the status of the PowerDNS service, we use systemctl status pdns:

# systemctl status pdns.service
● pdns.service - PowerDNS Authoritative Server
     Loaded: loaded (/lib/systemd/system/pdns.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2021-10-29 06:38:25 UTC; 14s ago
       Docs: man:pdns_server(1)
	     man:pdns_control(1)
	     https://doc.powerdns.com
   Main PID: 22789 (pdns_server)
      Tasks: 8 (limit: 1132)
     Memory: 43.6M
	CPU: 139ms
     CGroup: /system.slice/pdns.service
	     └─22789 /usr/sbin/pdns_server --guardian=no --daemon=no --disable-syslog --log-timestamp=no --write-pid=no

Oct 29 06:38:25 pdns01 pdns_server[22789]: TCP server bound to [::]:53
Oct 29 06:38:25 pdns01 pdns_server[22789]: PowerDNS Authoritative Server 4.4.1 (C) 2001-2020 PowerDNS.COM BV
Oct 29 06:38:25 pdns01 pdns_server[22789]: Using 64-bits mode. Built using gcc 10.2.1 20210110.
Oct 29 06:38:25 pdns01 pdns_server[22789]: PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it according to the terms >
Oct 29 06:38:25 pdns01 pdns_server[22789]: Creating backend connection for TCP
Oct 29 06:38:25 pdns01 pdns_server[22789]: [bindbackend] Parsing 0 domain(s), will report when done
Oct 29 06:38:25 pdns01 pdns_server[22789]: [bindbackend] Done parsing domains, 0 rejected, 0 new, 0 removed
Oct 29 06:38:25 pdns01 systemd[1]: Started PowerDNS Authoritative Server.
Oct 29 06:38:25 pdns01 pdns_server[22789]: About to create 3 backend threads for UDP
Oct 29 06:38:25 pdns01 pdns_server[22789]: Done launching threads, ready to distribute questions

• The default security settings of the PowerDNS service is already quite good, as can be seen with systemd-analyze:

% systemd-analyze security pdns

• However some additional hardening steps are possible (optional):
  • MemoryDenyWriteExecute - enables a function in the memory management unit (and modern CPUs) that makes memory pages either write-able (data) or executable (program code), but never both. This prevents certain attacks, such as buffer overflow attacks
  • umask - Files created by the PowerDNS processes can only be read or written by the PowerDNS user or root, but not by any other user
  • MemoryMax - Restrict the memory consumption of the PowerDNS process to 2 GB. This is usually more than enough for an authoritative server even with large zone files and prevents system instability in case of a memory leak
  • TasksMax - restricts the number of threads (tasks) of the PowerDNS service to 20. This prevents DoS attacks through a security vulnerability that would allow and external attacker to start new threads. This setting should be monitored and adjusted to the real world scenario where PowerDNS is deloyed (number of CPU cores, network interfaces etc).

# Additional hardening for the PowerDNS systemd unit
MemoryDenyWriteExecute=true
UMask=077
MemoryMax=2G
TasksMax=20

• opcode: query, notify, update, DSO … see IANA registry for DNS OpCodes
• status/rcode:
  • NOERROR - the operation was successful
  • NXDOMAIN - the domain name requested does not exist (or is not delegated)
  • SERVFAIL - some remote DNS server failure or DNSSEC validation failure
  • FORMERR - the query was not correct DNS
  • REFUSED - this server has an access control list that forbids the answer to this client
  • NOTIMPL - a feature used/requested that this server does not implement
  • BADCOOKIE - Bad/missing Server Cookie

  … see IANA registry for DNS RCODEs

• queryid: 16bit value to sort DNS answers to DNS queries
• flags: information on the query and the answer (see IANA registry for DNS Header Flags)
  • AA authoritative answer - answer is coming directly from an authoritative server
  • TC truncated - answer does not fit into the advertised UDP packet size, please re-query over TCP
  • RD recursion desired - this is a query from a client machine, please provide a full complete answer (no referral please)
  • RA recursion available - this answer comes from a DNS resolver that is willing to accept queries with RD flag set
• DNSSEC flags: AD- and CD-Flag
  • AD authentic data - the DNS resolver sending this answer has performed a successful DNSSEC validation on the data. If we trust the resolver, we can trust the data
  • CD checking disabled - a client asking a DNSSEC validating DNS resolver to not perform DNSSEC validation but to pass all DNSSEC data unaltered (even if the data is invalid). Used for troubleshooting DNSSEC issues.
• QUERY: number of query resource records (usually one)
• ANSWER: count of DNS resource records in the answer. Can be more than one. Can be zero if no data is available for the query.
• AUTHORITY: number of authority records in the answer. Can be a SOA-Record (for negative answers) or NS-Records for referrals or positive answers. Many modern DNS server only fill the authority section if required by the protocol to keep answer packets small
• ADDITIONAL: additional resource records that not have been requested but might help with the name resolution, and the EDNS (Extended DNS) OPT-Record (see RFC 6891 Extension Mechanisms for DNS (EDNS(0))

• EDNS Version 0 is the current version
  • new Versions are being discussed in the IETF
• additional EDNS flags. DO - DNSSEC OK, this DNS client supports DNSSEC records
• maximum UDP answer packet size in byte as negotiated between dig and the DNS server/resolver. Current default is 4096, must be between 512 and 4096. The default changed to 1232 on DNS flag day 2020

• The answer section contains zero (if there is no data available), one or more DNS resource records that match the query

• if present, the authority section contains the authoritative name server that hosts the content delivered in the answer. For negative (NXDOMAIN/NOERROR-NODATA) answers, the authority section contains the SOA record of the zone that is authoritative for the negative answer.

• if present, the additional section contains additional DNS records that have not been explicitly requested, but might help in the name resolution process. Because the additional section can be misused in attacks, modern DNS server software minimizes the additional section data.
• If EDNS data is available, it is also in the additional section, but dig displays this data in the OPT pseudo-section

• the footer contains the size (in bytes) of the answer packet, the DNS server that has send the answer, the time it took to receive the answer and the time and date of the DNS communication

• usually dig sends the query to the DNS resolver configured in the operating system (file /etc/resolv.conf on Unix/Linux)
• with the @ syntax the query can be sent to other DNS servers (resolver or authoritative server)

• command line switches can alter the query sent by dig.
  • +multi formats the output in human readable form (wrapped for 80 column terminal)
  • +norec sends a query without RD flag (non-recursive or iterative query)

• the switch +dnssec requests DNSSEC data from the upstream server
• the switch +cd sends the CD (checking disabled) flag to disable upstream DNSSEC validation (if the target of the query is a DNS resolver)

• dig can initiate a zone transfer from an authoritative server that contains the zone database (primary or secondary server)
• the data is printed in the Master Zone Format and can be used to start a new zone database file
• Example: list top level domains with DNSSEC delegation:

  $ dig @f.root-servers.net AXFR . +noidnout | \
    grep DS | \
    grep -v RRSIG | \
    cut -d "." -f 1 | \
    uniq
  

• pdns_server is the actual PowerDNS server binary. Besides starting the PowerDNS service, this command can be used to check the configuration file

% pdns_server --config=check
Apr 28 08:31:46 Loading '/usr/lib/x86_64-linux-gnu/pdns/libgmysqlbackend.so'

• The parameter --version shows some help information, but also the compile time configuration of the PowerDNS server binary. This is helpful to recompile a newer PowerDNS version with the same configuration, or to check for the existence of a compile-time feature

% pdns_server --version
Apr 28 08:32:29 PowerDNS Authoritative Server 4.9.7 (C) PowerDNS.COM BV
Apr 28 08:32:29 Using 64-bits mode. Built using gcc 14.2.0.
Apr 28 08:32:29 PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it according to the terms of the GPL version 2.
Apr 28 08:32:29 Features: libcrypto-ecdsa libcrypto-ed25519 libcrypto-ed448 libcrypto-eddsa libgeoip libmaxminddb lua lua-records PKCS#11 protobuf sodium curl scrypt 
Apr 28 08:32:29 Built-in modules: 
Apr 28 08:32:29 Loading '/usr/lib/x86_64-linux-gnu/pdns/libbindbackend.so'
Apr 28 08:32:29 Loading '/usr/lib/x86_64-linux-gnu/pdns/libgmysqlbackend.so'
Apr 28 08:32:29 Loaded modules: bind gmysql
Apr 28 08:32:29 Configured with: " '--build=x86_64-linux-gnu'
                   '--prefix=/usr'
		   '--includedir=${prefix}/include'
		   '--mandir=${prefix}/share/man'
		   '--infodir=${prefix}/share/info'
		   '--sysconfdir=/etc'
		   '--localstatedir=/var'
		   '--disable-option-checking'
		   '--libdir=${prefix}/lib/x86_64-linux-gnu'
		   '--runstatedir=/run'
		   '--disable-maintainer-mode'
		   '--disable-dependency-tracking'
		   '--sysconfdir=/etc/powerdns'
		   '--enable-systemd'
		   '--with-systemd=/usr/lib/systemd/system'
		   '--with-dynmodules=bind ldap lmdb lua2 pipe gmysql godbc gpgsql gsqlite3 geoip remote tinydns'
		   '--with-modules='
		   '--enable-ixfrdist'
		   '--enable-tools'
		   '--with-protobuf'
		   '--enable-unit-tests'
		   '--enable-lua-records'
		   '--enable-experimental-pkcs11'
		   '--enable-reproducible'
		   '--disable-silent-rules'
		   '--with-libcrypto=/usr'
		   'build_alias=x86_64-linux-gnu'
		   'CFLAGS=-g -O2 -Werror=implicit-function-declaration -ffile-prefix-map=/build/reproducible-path/pdns-4.9.7=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection' 'LDFLAGS=-Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -ffile-prefix-map=/build/reproducible-path/pdns-4.9.7=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -DPACKAGEVERSION='\''"4.9.7-1.Debian"'\'''"

• pdns_control is the older of the two PowerDNS command line utilities. It contains many functions to maintain the running PowerDNS server

• pdnsutil is the newer command line management tools. It contains functions to work with DNSSEC and TSIG security protocols (which will be covered on day 3 of this training).
  • It also contains functions to manage zones and zone content

• Cookies protect a querier against bogus answers.
  • If the querier is a DNS resolver, cache poisoning is thwarted.

• A bogus answer additionally injures the owner of the fraudulently answered domain name.

• Cookie protect an innocent victim against spoofed IP address attacks.

[sidebar] image::./images/Reflection-Attack-00.png[scaledwidth=65%]

[sidebar] image::./images/Reflection-Attack-01.png[scaledwidth=65%]

• Cookie protects DNS servers from being DoS victims & from being misused in reflection attacks.
  • This works for both authoritative and resolver servers.

Cookies allow DNS servers to quickly respond to spoofed IPs, avoiding the cost of handling the query, avoiding applying other secure mechanisms (e.g response rate limiting), and reducing CPU usage. (This protects the server from being a victim of a DoS attack) With cookies, the server limits its misuse in a reflection, making the Internet a better place.

• Client cookies automatically protect a querier from bogus answers and from cache poisoning.
• This further protects the domain owner!
  • An attacker can not provide bogus RRs for her domain.

• Server cookies allow a server to authenticate that queries are not from a spoofed IP.
• This protects a victim whose IP has been spoofed.
• This protects the server against DoS.
  • It can quickly respond to the query before executing costly work (e.g. DNSSEC crypto for any server, recursion for DNS resolver)
• Server Cookies require BIND configuration.

• Cookie security is against off-path attacks.
  • If an attacker has the packets between the querier & server, cookies add no security.

• Cookies require support from querier and server.
  • If implemented by only one side, cookies are ignored.
• With cookies, port randomization is not required.

• Cookies are 64-bit pseudorandom values.
• They are an OPT RR option, and thus require EDNS.
• A server doesn't store per-client state.
  • It doesn't store any cookie state.
• DNS cookies are not related to browser cookies.
• RFC 7873. 2016-05. Domain Name System (DNS) Cookies

• What happens if the server cookie is valid?

• This server cookie is valid (good).

$ dig @ns12a.dnslab.org zone12.dnslab.org +nocmd +noqu +norec

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el8 <<>> @ns12a.dnslab.org zone12.dnslab.org +nocmd +noqu +norec
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15954
;; flags: qr aa; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: 77eb710a2d489ed5010000005ecdff33b816374a28e8fde5 (good)
;; ANSWER SECTION:
zone12.dnslab.org.	60	IN	A	161.35.224.95

;; Query time: 155 msec
;; SERVER: 161.35.224.95#53(161.35.224.95)
;; WHEN: Wed May 27 05:48:35 UTC 2020
;; MSG SIZE  rcvd: 88

• Identical queries, but dig doesn't save server cookies, or even its own client cookie. Each time dig queries it use a new client cookie, and thus get a new server cookie.

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: 41cb7f09d4942eb3010000005ecdff829ab46f795ab83c4c (good)

• and another

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: bf7e888966399c5b010000005ecdff985ecbdb747d5b7955 (good)

• The command line switch +qr makes dig print the outgoing query in addition to the received answer. We see the client cookie in the outgoing query.

$ dig @ns12a.dnslab.org zone12.dnslab.org +nocmd +qr +norec

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el8 <<>> @ns12a.dnslab.org zone12.dnslab.org +nocmd +qr +norec
; (1 server found)
;; global options: +cmd
;; Sending:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10846
;; flags: ad; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 919e4f49e09f6d5d
;; QUESTION SECTION:
;zone12.dnslab.org.		IN	A

;; QUERY SIZE: 56

;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10846
;; flags: qr aa; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: 919e4f49e09f6d5d010000005ecdffcd693be2fca534ac93 (good)
;; QUESTION SECTION:
;zone12.dnslab.org.		IN	A

;; ANSWER SECTION:
zone12.dnslab.org.	60	IN	A	161.35.224.95

;; Query time: 155 msec
;; SERVER: 161.35.224.95#53(161.35.224.95)
;; WHEN: Wed May 27 05:51:09 UTC 2020
;; MSG SIZE  rcvd: 88

• DNS cookies can be enabled in PowerDNS:

  This setting MUST be 32 hexadecimal characters, as the siphash algorithm’s key used to create the cookie requires a 128-bit key.

• Create a random Cookie using openssl

  % openssl rand -hex 16 # 16 hex numbers are 32 hex digits
  

• Configure the cookie in /etc/powerdns/pdns.conf, check the configuration and restart the PowerDNS server

  edns-cookie-secret=<hex-string>
  

• Test with a recent version of dig that will display the cookie.
  • On the first query, the dig command does not know the server cookie
  • The server responds with a BADCOOKIE error (EDNS-Error) and sends its configured cookie to the client (the dig command)
  • The dig command will read the cookie, will create a new cookie value out of the PowerDNS servers cookie and its own cookie value and send this to the PowerDNS server
  • Now the PowerDNS server will see the configured cookie, a session is established, and the PowerDNS server will respond to the query

       $ dig @pdns009a.dnslab.org zone009.dnslab.org soa
    ;; BADCOOKIE, retrying.
    
    ; <<>> DiG 9.20.21-1~deb13u1-Debian <<>> @pdns009a.dnslab.org zone009.dnslab.org soa
    ; (2 servers found)
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35681
    ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
    ;; WARNING: recursion requested but not available
    
    ;; OPT PSEUDOSECTION:
    ; EDNS: version: 0, flags:; udp: 1232
    ; COOKIE: 9ffd13e9513a9c0d0100000069fc242b234886251a41310e (good)
    ;; QUESTION SECTION:
    ;zone009.dnslab.org.            IN      SOA
    
    ;; ANSWER SECTION:
    zone009.dnslab.org.     60      IN      SOA     pdns009a.dnslab.org. hostmaster.zone009.dnslab.org. 1002 3600 1800 3542400 60
    
    ;; Query time: 0 msec
    ;; SERVER: 2a03:b0c0:3:f0:0:2:5dce:8000#53(pdns009a.dnslab.org) (UDP)
    ;; WHEN: Thu May 07 05:33:31 UTC 2026
    ;; MSG SIZE  rcvd: 131
    

• DNSIND is shorthand for a group of extensions to the original DNS protocol.
  • I is for Incremental Zone Transfer
  • N is for NOTIFY
  • D is for Dynamic Update
• Together, the DNSIND extensions allow DNS to handle dynamic networks.
• A dynamic update allows a node to change the contents of a zone. DNS dynamic update is the RFC standard API to change DNS content over the network. As it is a RFC standard, it works across different DNS server products. Use cases:
  • DHCP servers, for example, can add A, AAAA and PTR records to reflect the leases they give out.
  • Microsoft Active Directory can update the SRV records required for service discovery via dynamic updates
  • DNS management systems such as VinylDNS (https://www.vinyldns.io), NetBox (https://netboxlabs.com) or Men & Mice Micetro (https://menandmice.com) can manage zone content through dynamic updates
  • ACME compatible x509 certificate management systems (e.g. Let's encrypt, RFC 8555 https://datatracker.ietf.org/doc/html/rfc8555) can authenticate the domain owner with the help of dynamic DNS updates, allowing centralized automated certificate management
• A dynamic update can:
  • Add resource records to a zone
  • Delete records from a zone:
    • One record
    • All records of a certain type with one domain name (an RRSet)
    • All records with one domain name
• A server can be configured to only accept updates from senders with the proper PSK (Pre-Shared TSIG Key).
• A sender can define update prerequisites, such as the existence/non-existence of:
  • A particular record
  • Resource Records with a specific domain name and type
• A sender first tries to transmit an update to a zone's primary master.
  • Most compare the SOA's mname field to the NS RRSet.
  • If the mname matches an NS RR, the update is sent there.
  • Otherwise the update is sent to one of the domain names in the NS RRSet (from the sender's perspective, a secondary).

    [sidebar]

image::./images/dynamic-update-00.png[scaledwidth=65%]

[sidebar] image::./images/dynamic-update-01.png[scaledwidth=65%]

[sidebar] image::./images/dynamic-update-02.png[scaledwidth=65%]

[sidebar] image::./images/dynamic-update-03.png[scaledwidth=65%]

[sidebar] image::./images/dynamic-update-04.png[scaledwidth=65%]

[sidebar] image::./images/dynamic-update-05.png[scaledwidth=65%]

[sidebar] image::./images/dynamic-update-06.png[scaledwidth=65%]

[sidebar] image::./images/dynamic-update-07.png[scaledwidth=65%]

• PowerDNS supports dynamic updates on a server that hosts primary (master) zones. Dynamic updates are disabled in the default configuration and can be enabled in the configuration file (global)

dnsupdate=yes

• In addition to the global configuration, dynamic updates need to be enabled on a zone

% pdnsutil set-meta example.org ALLOW-DNSUPDATE-FROM 127.0.0.0/8
Oct 30 08:03:57 gmysql Connection successful. Connected to database 'powerdns' on '/run/mysqld/mysqld.sock'.
Oct 30 08:03:57 gmysql Connection successful. Connected to database 'powerdns' on '/run/mysqld/mysqld.sock'.
Set 'example.org' meta ALLOW-DNSUPDATE-FROM = 127.0.0.0/8

• The tool nsupdate (Debian packet dnsutils) is the standard tool to send dynamic update messages. Here an example on how to add a new record to a zone:

$ nsupdate
> server 127.0.0.1
> ttl 60
> add ddns.example.org in TXT "This is a dynamically added DNS record"
> send
> quit

• The dynamic update in the PowerDNS logfile

Oct 30 08:05:45 pdns01 pdns_server[30874]: Remote 127.0.0.1 wants 'ddns.example.org|SOA', do = 0, bufsize = 512: packetcache MISS
Oct 30 08:05:45 pdns01 pdns_server[30874]: Remote 127.0.0.1 wants 'example.org|SOA', do = 0, bufsize = 512: packetcache MISS
Oct 30 08:05:45 pdns01 pdns_server[30874]: UPDATE (9824) from 127.0.0.1 for example.org: Processing started.
Oct 30 08:05:45 pdns01 pdns_server[30874]: UPDATE (9824) from 127.0.0.1 for example.org: starting transaction.
Oct 30 08:05:45 pdns01 pdns_server[30874]: UPDATE (9824) from 127.0.0.1 for example.org: Adding record ddns.example.org|TXT
Oct 30 08:05:45 pdns01 pdns_server[30874]: UPDATE (9824) from 127.0.0.1 for example.org: Increasing SOA serial (1111 -> 2021103001)
Oct 30 08:05:45 pdns01 pdns_server[30874]: UPDATE (9824) from 127.0.0.1 for example.org: Update completed, 2 changed records committed.
Oct 30 08:06:26 pdns01 pdns_server[30874]: 1 domain for which we are master needs notifications
Oct 30 08:06:26 pdns01 pdns_server[30874]: Queued notification of domain 'example.org' to 165.22.233.102:53

• The record will be published immediately and can be queried over DNS:

% dig @localhost ddns.example.org txt

; <<>> DiG 9.16.22-Debian <<>> @localhost ddns.example.org txt
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 2483
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;ddns.example.org.       IN      TXT

;; ANSWER SECTION:
ddns.example.org. 60     IN      TXT     "This is a dynamically added DNS record"

;; Query time: 3 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sat Oct 30 08:06:28 UTC 2021
;; MSG SIZE  rcvd: 103

• In the default configuration PowerDNS will wait 60 seconds after a dynamic update before sending out NOTIFY messages to the secondary. This configuration changes PowerDNS to send a notification to all secondary servers after every update. This will speed up the propagation of changes and is very useful for ACME TLS certificate verification

% pdnsutil set-meta example.org NOTIFY-DNSUPDATE 1

• Each dynamic update will increment the SOA serial number. PowerDNS has a SOA serial number edit policy (SOA-EDIT-DNSUPDATE) that defines how the SOA serial number should be incremented
  • DEFAULT: Generate a soa serial of YYYYMMDDNN. If the current serial is lower than the generated serial, use the generated serial. If the current serial is higher or equal to the generated serial, increase the current serial by 1.
  • INCREASE: Increase (increment) the current serial by 1.
  • EPOCH: Change the serial to the number of seconds since the EPOCH (seconds since 1.1.1970, also known as unixtime)
  • SOA-EDIT: Change the serial to whatever SOA-EDIT would provide. Used for DNSSEC zones to ensure freshness of signatures on secondary (slave) server
  • SOA-EDIT-INCREASE: Change the serial to whatever SOA-EDIT would provide. If what SOA-EDIT provides is lower than the current serial, increase the current serial by 1. Exception: with SOA-EDIT=INCEPTION-EPOCH, the serial is bumped to at least the current EPOCH time.

% pdnsutil set-meta example.org SOA-EDIT-DNSUPDATE INCREASE

• nsupdate is a BIND application for sending dynamic changes to a server.
  • It works both interactively and non-interactively.
  • In both modes, commands are one-per-line.
• The three main commands are:
  • update add
  • update delete
  • send
• update prepares RRs changes, send executes them.
• Note: ENTER by itself is the same as send
• Non-interactively, commands come from a file or STDIN.

$ nsupdate nsupdate.input.file

$ nsupdate < nsupdate.input.file

• Interactive mode:

$ nsupdate
>

• The interactive command show allows changes to be reviewed before being sent.
  • Here no updates have been prepared.

$ nsupdate
> show
Outgoing update query:
;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id:      0
;; flags:; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0
>

• update add prepares a RR to be added to a zone.
  • All fields, except the Class must be provided.
  • TTL can be provided in a separate statement.

> update add a.example.com IN A 192.0.2.1
ttl 'IN': not a valid number
> update add a.example.com 3600 IN A 192.0.2.1
> ttl 3600
> update add a.example.com AAAA 2001:db8:0:deaf::1
> show
Outgoing update query:
;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id:      0
;; flags:; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0
;; UPDATE SECTION:
a.example.com.		3600	IN	A	192.0.2.1
a.example.com.		3600	IN	AAAA	2001:db8:0:deaf::1
> send

• update delete prepares RRs to be deleted.
  • Class is optional, and TTL, if provided, is ignored.
  • Providing all fields except TTL and Class deletes one RR.

> update delete b.example.com. 1800 A 192.0.2.99
> update delete example.net. MX 15 mailserver.example.net.
> show
Outgoing update query:
;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id:      0
;; flags:; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0
;; UPDATE SECTION:
b.example.com.		0	NONE	A	192.0.2.99
example.net.		    0	NONE	MX	15 mailserver.example.net.

• update delete Providing the RTYPE without RDATA, deletes a RRSet.

> update delete b.example.com. AAAA
> update delete c.example.com IN SRV
> show
Outgoing update query:
;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id:      0
;; flags:; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0
;; UPDATE SECTION:
b.example.com.		0	NONE	A	192.0.2.99
example.net.		  0	NONE	MX	15 mailserver.example.net.
b.example.com.		0	ANY	AAAA
c.example.com.		0	ANY	SRV
>

• update delete Omitting RTYPE and RDATA deletes all RRSets.

> update delete d.example.com.
> show
Outgoing update query:
;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id:      0
;; flags:; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0
;; UPDATE SECTION:
b.example.com.		0	NONE	A	192.0.2.99
example.net.	    0	NONE	MX	15 mailserver.example.net.
b.example.com.		0	ANY	AAAA
c.example.com.		0	ANY	SRV
d.example.com.		0	ANY	ANY

• Since BIND 9.9.0, the key word update is optional.

> delete  www.example.com. A
> add www.example.com. 600 A 192.0.2.80
> add www.example.com. 600 A 192.0.2.88

• send - Transmit the prepared updates to the authoritative server.
  • WARNING - hitting ENTER is the same as send!

> send

• answer - shows the results received from the server after a send.

> send
update failed: REFUSED
> answer
Answer:
;; ->>HEADER<<- opcode: UPDATE, status: REFUSED, id:  13117
;; flags: qr; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0
>

• prereq nxdomain <domain name> - requires that "domain name" doesn't exist (no RRs of any type).

> prereq nxdomain www.example.com
> update add www.example.com 3600 CNAME example.com.
> send

• prereq yxdomain <domain name> - requires that "domain name" does exist.

> prereq yxdomain www.example.com
> update add web.example.com 3600 CNAME www.example.com.
> send

• server <servername or IP> [port] - specifies the authoritative server that will receive the update.
  • If not set, nsupdate uses MNAME from the SOA RR and port 53.
  • Generally, updates should be sent to the primary authoritative server.

> server ns0.example.com

• nsupdate -l sends all updates to localhost.
  • The MNAME is not checked, and server commands are ignored.
  • This is easy use, and is recommended if you are on the primary server
• local <IP address> [port] - specifies the outgoing IP address. This is useful for multi-homed machines, especially when IPv6 is used, and when the server limits updates to specific IPs.

> local 2001:db8:100:0:ff:aa01:42:12:feaa

• zone <zonename> - specify the zone to be changed.
  • This is necessary when the update is an NS or glue RR to be made in the parent zone.

$ nsupdate
> zone example.com.
> update add ns1.subdomain.example.com. 3600 IN A 192.0.2.53
> update add ns2.subdomain.example.com. 3600 IN A 203.0.113.53
> send

• the classic DNS from 1983 has not been designed with security in mind
• Attack vector: DNS cache poisoning
• Attack vector: Men-in-the-Middle data spoofing
• Attack vector: changes to the client DNS resolver configuration
• Attacks of authoritative DNS servers
• DNSSEC can help

• The DNS security extension DNSSEC secures DNS data by augmenting the data with cryptographic signatures
  • The owner (administrator) creates a pair of private and public keys for each DNS zone (asymmetric crypto)
  • The owner/administrator signs all DNS data with the private/secret key
  • The recipient of the data (DNS resolver or client operating system or application) will verify (validate) the data
    • That the data has not been changed on the server nor during transit
    • That the data comes from the owner (the owner of the private key)

• DNSSEC creates a chain of trust between the parent zone and the child zone

• Applications and DNS resolvers can follow the chain of trust to a configured trust anchor to validate the DNS data
• We will look into DNSSEC in more details later in this course

DNSSEC keys can be generated with different crypto algorithms. Some of these algorithms are obsolete and deprecated, others are not (yet) widely supported by deployed DNS software to be useful

• Every additional bit increases the strength of a DNSSEC against brute-force attacks to break the key
• Double the RSA key size results in
  • Generating signatures is up to eight times more work
  • Validation of DNSSEC signatures inside an DNS resolver up to 4 times more work
  • Size of key and signature records increases and can create operational issues

• For organizational reasons, DNSSEC splits the chain of trust inside a DNS zone onto two different keys: the Key-Signing-Key (KSK) and the Zone-Signing-Key (ZSK)

• DNSSEC keys have an organisational lifetime (there is no technical limit on the lifetime, unlike with X.509 TLS certificates)
  • the administrator responsible for a DNSSEC-signed zone can decide when to change the DNSSEC keys
• Renewing the DNSSEC key material of a zone is called a key rollover
• Keys with weak algorithms or short key lengths should be changed (rolled) in shorter intervals
  • 1024bit RSASHA256 -> 30 days (ZSK)
  • 1536bit RSASHA256 -> 120 days (ZSK)
  • 2048bit RSASHA256 -> 360 days (KSK)
  • 2560bit RSASHA256 -> 720 days+ / 2 years+ (KSK)

• Zones with high security requirements should keep the DNSSEC keys "offline"
  • this requires manual DNSSEC signing
• Keys can be stored securely in Hardware Security Modules (HSM)
• HSM selection criteria for DNSSEC signing
  • Number of key storage slots
  • Timely support for new operating system releases (Linux distribution)
  • Timely support for new OpenSSL releases
  • Stability of the HSM-Driver
• Zones with medium to low security requirements should use a hidden-primary DNSSEC signer configuration
  • The DNSSEC signing authoritative server is not exposed to the Internet, it will not receive DNS queries
  • DNS secondary authoritative servers in the Internet will receive the DNSSEC signed zone from the hidden DNSSEC signer through DNS zone transfer
  • The DNSSEC key material is secured with operating system level permissions
  • With full automation of DNSSEC key-rollovers, the DNS server administrators don't need access to the DNSSEC keys
  • Login and access to the DNSSEC key files should be audited (Logging online and towards a remote log server)
• Zone content can be split across multiple zones or DNS server with DNS delegation
  • Every zone can have a different DNSSEC security level
  • Example: main zone example.com signed via a hidden-signer (keys not exposed to the Internet) has a sub-zone with e-mail address hashes (OPENPGPKEY and SMIMEA) in the zone securemail.example.com) which is secured with NSEC3-NARROW scheme and keys that are online on every authoritative server

• The RRSIG records in DNSSEC secure the positive answers from DNS (answers to queries where DNS records exist)
• But also negative answers need to be protected
  • without protection for negative answers, attackers could spoof negative answers to launch denial-of-service attacks. The attacker can make DNS clients believe that a DNS resource does not exist
• DNS knows two kinds of negative answers: NXDOMAIN and NODATA/NXRRSET
  • NXDOMAIN = the requested domain name does not exist
  • NODATA/NXRRSET = the requested domain name does exist, but the requested record type does not exist for this name
• DNS error messages such as SERVFAIL, FORMERR or REFUSED cannot be secured with DNSSEC (these errors indicate errors in the protocol or in the DNS server infrastructure). If the protocol is broken, DNSSEC can't work either.
• The solution: the NSEC records in the DNSSEC signed zone create a list of all existing data in the zone (domain names and records types for the domain names)
  • When returning a negative answer, the DNS server returns the negative answer containing the NSEC record (plus a signature for the NSEC record) which proves that the requested data does not exist in DNS

• The NSEC record is an elegant solution to the problem, but it has drawbacks:
  • It is now possible for outsiders to list the complete zone contents by following the NSEC record chain. This is called zone walking.
  • For most zones this is not a big problem, as the DNS zone content is public anyway (SOA, NS records, WWW-A, MX records)
  • But it can be an issue for zones with sensitive content (email addresses, hostnames of critical infrastructure, new product names that should no go public yet)
  • Operators of TLDs zones stay away from DNSSEC with NSEC, as it allows outsiders to record all changes to the TLD zone (new delegations and delegation removals)
  • The new compact authenticated denial of existance standard (RFC 9824) solves this issue.
• Example of zone walking:

$ ldns-walk paypal.com

• RFC 5155 (2008) defines the NSEC3 record as an alternative to NSEC
  • All modern DNS servers support NSEC3
  • The NSEC3 record works similarly to NSEC, with the difference that the link between the owner names is done using SHA1 hashes of the domain names instead of the clear text names
  • NSEC3 makes it harder, but not impossible, to list the zone content

• The NSEC3 record contains a number of parameters used for the NSEC3 operation
  • The hashing algorithm used (currently only SHA1 is defined)
  • Flags: allows for DNSSEC opt-out in delegations.
  • Number of hash iterations
  • A salt value

• Every zone with NSEC3 records contains an NSEC3PARAM record. This record holds information needed by authoritative DNS servers to generate NSEC3 records for negative answers
• Example: (SHA1, no flags, 20 iterations, salt value "ABBACAFE")

nsec3.dnslab.org.    0   IN  NSEC3PARAM 1 0 20 ABBACAFE

• The idea of the hash iterations in the NSEC3PARAM record was to allow the operator of the zone to fine tune the work required for calculating the hash
  • A higher number of iterations should make it harder for attackers to brute force break an NSEC3 domain name
  • A higher number also creates more CPU load for DNS resolvers that validate the NSEC3 records, making DNS name resolution slower
  • The iteration parameter of an NSEC3 signed zone should be re-evaluated from time to time (yearly) to adjust the value for the technical progress
• The salt should make it impossible for an attacker to pre-calculate rainbow tables for the zone
• The salt is a hexadecimal number, every hex digit contains 4 bit of information

• The iterations had more an negative impact on the CPU load of the authoritative DNS servers than preventing attacks
• The salt is not really needed, as each zone naturally has unique hashes so that a rainbow table for multiple zones is not possible

• RFC 9276 - Guidance for NSEC3 Parameter Settings (Best current practice) gives updates guidelines for NSEC and NSEC3 deployments
• The iterations value should be set to 0 (meaning 1 iteration of SHA1 hashing)
  • DNSSEC responses containing NSEC3 records with iteration counts greater than 150 are now treated as insecure by major DNS resolvers!
• As NSEC3 zones are inherently salted, the salt parameter should be set to -
• The current recommendation for NSEC3PARAM is 1 0 0 - (SHA1 Hash, no flags, 1 iteration, no salt)

• Most DNS zone can work with NSEC instead of NSEC3
  • Don't store names in DNS that should be secret (internal names, product names etc)
  • DNS is a service to publish data, not for hiding data

• RFC 7129 (also RFC 4470 and RFC 4471) describe a special variant of NSEC3 usage, called the narrow mode
  • With narrow mode, the NSEC3 records are not pre-calculated when the zone is signed, but they are created on the fly whenever a negative response is needed
    • To be able to calculate the signatures for such answers, every authoritative DNS server for the zone must have access to the private DNSSEC keys (at least the ZSK)!
• With NSEC3 narrow mode, the DNS server does not return an NSEC3 chain of existing records, but a synthetic NSEC3 record that proves that the requested name does not exist
  • This prevents zone walking, as the number of possible NSEC3 records returned is near infinite. It will exhaust the memory of the attacker that will try to store this NSEC3 chain
• Known implementations:
  • Phreebird (Dan Kaminski, Proof-of-Concept, not maintained)
  • PowerDNS Authoritative
  • Cloudflare CDN NSEC3 Narrow-Mode (closed source implementation)

• To sign DNS zone inside PowerDNS, a basic DNSSEC configuration must be added to the configuration
  • For the MySQL/MariaDB backend, the backend needs to have DNSSEC enabled with gmysql-dnssec
  • It is a good idea to have a basic DNSSEC signing policy (default key algorithms and sizes, NSEC3 iterations and how the SOA serial is incremented) in the PowerDNS configuration file. Increment the SOA serial is important in a primary/secondary setup with classic DNS zone transfer.

# DNSSEC
gmysql-dnssec
default-ksk-algorithm=rsasha256
default-ksk-size=2048
default-zsk-algorithm=rsasha256
default-zsk-size=1024
max-nsec3-iterations=1
default-soa-edit-signed=INCREMENT-WEEKS

• First step of signing a zone is to create a zone signing key (ZSK). For RSA keys, the ZSK usually is smaller than the KSK.

% pdnsutil add-zone-key zoneNNN.dnslab.org ZSK active published 1536 rsasha256
Added a ZSK with algorithm = 8, active=1
Requested specific key size of 1536 bits
1

• Next step is the creation of the key signing key (KSK)

% pdnsutil add-zone-key zoneNNN.dnslab.org KSK active published 2048 rsasha256
Added a KSK with algorithm = 8, active=1
Requested specific key size of 2048 bits
2

• The command pdnsutil show-zone <zone-name> will show the zone keys, including the delegation signer (DS) record that need to go into the parent zone

% pdnsutil show-zone zoneNNN.dnslab.org
This is a Master zone
Last SOA serial number we notified: 1012 == 1012 (serial in the database)
Metadata items: None
Zone has NSEC semantics
keys:
ID = 1 (ZSK), flags = 256, tag = 37441, algo = 8, bits = 1536     Active         Published  ( RSASHA256 )
ID = 2 (KSK), flags = 257, tag = 26177, algo = 8, bits = 2048     Active         Published  ( RSASHA256 )
KSK DNSKEY = zoneNNN.dnslab.org. IN DNSKEY 257 3 8 AwEAAdmaHnizfuBb89XXhYBoQBnsJ/d+K7zkelu8rAvbsk3pzgwWMEufNXEqBxyQxxKYmqk/m+Wcq2UtuGnRugMfk2iaCod9DHp2iJj9qZ6C2Gqv9jVzwK1OE3yHq9xTuzq408ibgrxQvhZwYcH4mbqTlopFz/qACNKYdr52zCFnmK3cjHSphNqo3pGlBBGFLrXdLS8oHW6Ec+G7DObTqk+rGUuKHCStG0qPKoKj20L0j2frmcG5h5X0aH+7vHyBeikwuJoyqiwmEg0PYmknAnicuh3ubcPB5P+5ZcItpe6xY52eKqWIjsjcZI0aUmlOqI/ISuwP5b7UhYPgEc8JWlu6T9E= ; ( RSASHA256 )
DS = zoneNNN.dnslab.org. IN DS 26177 8 1 2416dc421e6debc0c0fd75b8742086df862f054f ; ( SHA1 digest )
DS = zoneNNN.dnslab.org. IN DS 26177 8 2 5993bff9c0f53af0162495f9b5c72a8fc5d05b24602e3905d3f4641da55a6f2c ; ( SHA256 digest )
DS = zoneNNN.dnslab.org. IN DS 26177 8 4 2810a931cf90eb4977eafdaaa3cfc8d90bb439ab836b3e05b23aef88ea0368f200a1b145472d36efc1b40c3567af3e71 ; ( SHA-384 digest )

• As the keys are active, PowerDNS will start signing the responses with RRSIG records

# dig @localhost zoneNNN.dnslab.org soa +dnssec +multi

; <<>> DiG 9.16.22-Debian <<>> @localhost zoneNNN.dnslab.org soa +dnssec +multi
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44805
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1232
;; QUESTION SECTION:
;zoneNNN.dnslab.org.    IN SOA

;; ANSWER SECTION:
zoneNNN.dnslab.org.     60 IN SOA pdns010a.dnslab.org. hostmaster.zoneNNN.dnslab.org. (
                                1012       ; serial
                                3600       ; refresh (1 hour)
                                1800       ; retry (30 minutes)
                                3542400    ; expire (5 weeks 6 days)
                                60         ; minimum (1 minute)
                                )
zoneNNN.dnslab.org.     60 IN RRSIG SOA 8 3 60 (
                                20211111000000 20211021000000 37441 zoneNNN.dnslab.org.
                                InqX6esX3pl2GdGratuae544DO41egiVRLaLIPa41c9H
                                JqJjr2uMH5IcUtRtfDX4q296SSq5nNL9aouDtrvw+zNN
                                xGc/0qJ/aVr/uTqEXJBLGVbdZnCjpfvYNrDh3IIMBcSm
                                2TVifrT6JuGvIRUB/GskXvTYIDZW5DQ3YVP131sp4u9t
                                SgKWDvdt93N43maqy0imqJKwU7mV+t9Rx+El37c/CQZD
                                V+ERB5enFiqUr8e08wjc7rvDLAMFzBYbnc+3 )

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Nov 03 19:48:02 UTC 2021
;; MSG SIZE  rcvd: 345

• The default authenticated denial of existence is NSEC, so there are NSEC records

% dig @localhost zoneNNN.dnslab.org srv +dnssec +multi

; <<>> DiG 9.16.22-Debian <<>> @localhost zoneNNN.dnslab.org srv +dnssec +multi
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39295
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 4, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1232
;; QUESTION SECTION:
;zoneNNN.dnslab.org.    IN SRV

;; AUTHORITY SECTION:
zoneNNN.dnslab.org.     60 IN SOA pdns010a.dnslab.org. hostmaster.zoneNNN.dnslab.org. (
                                1012       ; serial
                                3600       ; refresh (1 hour)
                                1800       ; retry (30 minutes)
                                3542400    ; expire (5 weeks 6 days)
                                60         ; minimum (1 minute)
                                )
zoneNNN.dnslab.org.     60 IN RRSIG SOA 8 3 60 (
                                20211111000000 20211021000000 37441 zoneNNN.dnslab.org.
                                InqX6esX3pl2GdGratuae544DO41egiVRLaLIPa41c9H
                                JqJjr2uMH5IcUtRtfDX4q296SSq5nNL9aouDtrvw+zNN
                                xGc/0qJ/aVr/uTqEXJBLGVbdZnCjpfvYNrDh3IIMBcSm
                                2TVifrT6JuGvIRUB/GskXvTYIDZW5DQ3YVP131sp4u9t
                                SgKWDvdt93N43maqy0imqJKwU7mV+t9Rx+El37c/CQZD
                                V+ERB5enFiqUr8e08wjc7rvDLAMFzBYbnc+3 )
zoneNNN.dnslab.org.     60 IN NSEC zoneNNN.dnslab.org. A NS SOA RRSIG NSEC DNSKEY
zoneNNN.dnslab.org.     60 IN RRSIG NSEC 8 3 60 (
                                20211111000000 20211021000000 37441 zoneNNN.dnslab.org.
                                QYedsQarfZicH7Pbawra6TAUH2XPK6zuH+jaVjKVffOe
                                GFyCDNwTxIx8U8MT7ntR1mJxUjuFf+byftkS3WkNxxNa
                                Fs+73xLeWgro+J3zY2t+M4rozZBdHKX0zsU0b/XMDYl+
                                PW5w+0zgcIoTs7aTexwsHg587TwaluBZI6sQhxLJ0tsB
                                wPOEu7BoIBhCaTqrgI4aQiK6uO3R9k2f5pieXrxtDHNC
                                lDf/3cdgIhMPQkpIeB7Pph1f+5bHfZ01JBPd )

;; Query time: 7 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Nov 03 19:48:52 UTC 2021
;; MSG SIZE  rcvd: 628

• After making changes to a DNSSEC signed zone, it is always advised to execute pdnsutil rectify-zone <zone-name>. rectify-zone will recreate missing signatures and NSEC/NSEC3 records. rectify-zone is important when making changes to the zone on the database level or through the API. For zones that are managed through the API, set the API-RECTIFY meta-data on the zone, or make sure that default-api-rectify is set to yes in the configuration

% pdnsutil rectify-zone zoneNNN.dnslab.org
Adding NSEC ordering information

• Send the DS-Record or the DNSKEY record of the KSK to the operator of the parent zone. The DS-Record(s) and the DNSKEY-Record can be listed with pdnsutil show-zone <zonename>
• Add a new record

% pdnsutil add-record zoneNNN.dnslab.org test TXT 60 '"Hello PowerDNS"'
New rrset:
test.zoneNNN.dnslab.org. 60 IN TXT "Hello PowerDNS"
test.zoneNNN.dnslab.org. 60 IN TXT "Hello DNSSEC"

• The new record (set) is immediately signed

% dig @localhost test.zoneNNN.dnslab.org.  txt +dnssec

; <<>> DiG 9.16.22-Debian <<>> @localhost test.zoneNNN.dnslab.org. txt +dnssec
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10652
;; flags: qr aa rd; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1232
;; QUESTION SECTION:
;test.zoneNNN.dnslab.org.       IN      TXT

;; ANSWER SECTION:
test.zoneNNN.dnslab.org. 60     IN      RRSIG   TXT 8 4 60 20211111000000 20211021000000 37441 zoneNNN.dnslab.org. H0MNWJ/xP9xfLYxaTaqEyhe+jLszDxZ4MP6AR6pz1S3GsHhMktR3/P6u DrBihkZlNMi+7ZhTWHFmndCwd9vK/nnaHZXNV2JL72oMPS4vhXWgw6AP T+mUsOkRyHudJRoASpyOOPhqGxjDVUXanDdpbVAfoEjhWnZ6a+NWC47C 04zT/4NSaIpZPtCIYKftbiPB0CjEFPsViloRNAUFwbYHZ4o9TZYdDekQ nOXAapLooAh0DFk21ijjrVV1jbigS6oV
test.zoneNNN.dnslab.org. 60     IN      TXT     "Hello PowerDNS"
test.zoneNNN.dnslab.org. 60     IN      TXT     "Hello DNSSEC"

;; Query time: 3 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Nov 03 20:03:34 UTC 2021
;; MSG SIZE  rcvd: 346

• After making changes to a DNSSEC signed zone, it is a good idea to increment the SOA serial a last time and fix the DNSSEC NSEC(3) chain in the zone with rectify-zone

% pdnsutil increase-serial zoneNNN.dnslab.org
% pdnsutil rectify-zone zoneNNN.dnslab.org

• Check for a successful zone transfer to all secondary DNS server of the zone

• To start a ZSK key rollover for a zone on PowerDNS, create and publish (but don't activate) a new ZSK. The new key should have the same algorithm then the previous key. The size can be different.

% pdnsutil add-zone-key zoneNNN.dnslab.org ZSK inactive published 1536 rsasha256
Added a ZSK with algorithm = 8, active=1
Requested specific key size of 1536 bits
3

• The zone should now have 2 ZSK DNSKEY records published

% pdnsutil show-zone zoneNNN.dnslab.org
This is a Master zone
Last SOA serial number we notified: 1635970166 == 1635970166 (serial in the database)
Metadata items: None
Zone has NSEC semantics
keys:
ID = 1 (ZSK), flags = 256, tag = 37441, algo = 8, bits = 1536     Active         Published  ( RSASHA256 )
ID = 4 (ZSK), flags = 256, tag = 14634, algo = 8, bits = 1536   Inactive         Published  ( RSASHA256 )
ID = 2 (KSK), flags = 257, tag = 26177, algo = 8, bits = 2048     Active         Published  ( RSASHA256 )
KSK DNSKEY = zoneNNN.dnslab.org. IN DNSKEY 257 3 8 AwEAAdmaHnizfuBb89XXhYBoQBnsJ/d+K7zkelu8rAvbsk3pzgwWMEufNXEqBxyQxxKYmqk/m+Wcq2UtuGnRugMfk2iaCod9DHp2iJj9qZ6C2Gqv9jVzwK1OE3yHq9xTuzq408ibgrxQvhZwYcH4mbqTlopFz/qACNKYdr52zCFnmK3cjHSphNqo3pGlBBGFLrXdLS8oHW6Ec+G7DObTqk+rGUuKHCStG0qPKoKj20L0j2frmcG5h5X0aH+7vHyBeikwuJoyqiwmEg0PYmknAnicuh3ubcPB5P+5ZcItpe6xY52eKqWIjsjcZI0aUmlOqI/ISuwP5b7UhYPgEc8JWlu6T9E= ; ( RSASHA256 )
DS = zoneNNN.dnslab.org. IN DS 26177 8 1 2416dc421e6debc0c0fd75b8742086df862f054f ; ( SHA1 digest )
DS = zoneNNN.dnslab.org. IN DS 26177 8 2 5993bff9c0f53af0162495f9b5c72a8fc5d05b24602e3905d3f4641da55a6f2c ; ( SHA256 digest )
DS = zoneNNN.dnslab.org. IN DS 26177 8 4 2810a931cf90eb4977eafdaaa3cfc8d90bb439ab836b3e05b23aef88ea0368f200a1b145472d36efc1b40c3567af3e71 ; ( SHA-384 digest )

• Rectify the zone so that all signatures and NSEC/NSEC3 records are present

% pdnsutil rectify-zone zoneNNN.dnslab.org

• Increment the SOA serial number to trigger NOTIFY for zone-transfer

% pdnsutil increase-serial zoneNNN.dnslab.org

• The new ZSK is now visible in the DNS zone

% dig dnskey zoneNNN.dnslab.org +multi

; <<>> DiG 9.16.22-Debian <<>> dnskey zoneNNN.dnslab.org +multi
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 59184
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;zoneNNN.dnslab.org.    IN DNSKEY

;; ANSWER SECTION:
zoneNNN.dnslab.org.     38 IN DNSKEY 256 3 8 (
                                AwEAAa56Jv3sjwd2yKNbbdcoTNMViBFDv3StAD1Kif7+
                                G/X993ac8cwevgp77dgGx/gmdLTu9/sV3df5/Y31l9j/
                                55QzquNU4Svxnkbf+4onN9dB0Vzh+ASUzrCTfu592tso
                                xpM6YxQUbjgYDI8308u/dgcBSsl5uaJjIcxUNUaP/oC0
                                qn2l4x7TtmHsoMXlrjJB5/oChdvyuU10i0ts3GOpJmka
                                vuynXawWuE5LQAXxNS+POXbKCHf7dzbtUzAQjFBXAw==
                                ) ; ZSK; alg = RSASHA256 ; key id = 14634
zoneNNN.dnslab.org.     38 IN DNSKEY 256 3 8 (
                                AwEAAdSEeKfBK/EXKfKtwB1rpSForHhwVBekz2ZyOd+F
                                uHdNXlCRyhRlok3zUq9BOQLPpqPFm+GGdFttACu3sAeO
                                aq7NIrpFHVijWwXMX0zBeak7ujwFKlC4cKszfEVB6kaq
                                m62JyAGQF6Z6mzDz4q/iW8gq1LmtdMwG+ked3lp5zQk/
                                0MZUBfJcqrya1biDnfOteCv/InoTrydYbFvUNVBe2VlA
                                FZG7pUhQcZs+CQn8y71GU+pepogCM8IJOiE7Fw+7zQ==
                                ) ; ZSK; alg = RSASHA256 ; key id = 37441
zoneNNN.dnslab.org.     38 IN DNSKEY 257 3 8 (
                                AwEAAdmaHnizfuBb89XXhYBoQBnsJ/d+K7zkelu8rAvb
                                sk3pzgwWMEufNXEqBxyQxxKYmqk/m+Wcq2UtuGnRugMf
                                k2iaCod9DHp2iJj9qZ6C2Gqv9jVzwK1OE3yHq9xTuzq4
                                08ibgrxQvhZwYcH4mbqTlopFz/qACNKYdr52zCFnmK3c
                                jHSphNqo3pGlBBGFLrXdLS8oHW6Ec+G7DObTqk+rGUuK
                                HCStG0qPKoKj20L0j2frmcG5h5X0aH+7vHyBeikwuJoy
                                qiwmEg0PYmknAnicuh3ubcPB5P+5ZcItpe6xY52eKqWI
                                jsjcZI0aUmlOqI/ISuwP5b7UhYPgEc8JWlu6T9E=
                                ) ; KSK; alg = RSASHA256 ; key id = 26177

;; Query time: 3 msec
;; SERVER: 67.207.67.3#53(67.207.67.3)
;; WHEN: Wed Nov 03 20:28:06 UTC 2021
;; MSG SIZE  rcvd: 747

• Wait for the TTL of the DNSKEY record set (60 seconds in our example)
• Activate the new key, deactivate the old key, rectify zone and increment the SOA serial. The new ZSK is now active

% pdnsutil activate-zone-key zoneNNN.dnslab.org 4
% pdnsutil deactivate-zone-key zoneNNN.dnslab.org 1
% pdnsutil rectify-zone zoneNNN.dnslab.org
Adding NSEC ordering information
% pdnsutil increase-serial zoneNNN.dnslab.org
SOA serial for zone zoneNNN.dnslab.org set to 1635971832

• Check that the signatures on the SOA record are now created by the new ZSK (Keytag 14634 in this example)

% dig soa zoneNNN.dnslab.org +multi +dnssec

; <<>> DiG 9.16.22-Debian <<>> soa zoneNNN.dnslab.org +multi +dnssec
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27737
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;zoneNNN.dnslab.org.    IN SOA

;; ANSWER SECTION:
zoneNNN.dnslab.org.     60 IN SOA pdns010a.dnslab.org. hostmaster.zoneNNN.dnslab.org. (
                                1635971879 ; serial
                                3600       ; refresh (1 hour)
                                1800       ; retry (30 minutes)
                                3542400    ; expire (5 weeks 6 days)
                                60         ; minimum (1 minute)
                                )
zoneNNN.dnslab.org.     60 IN RRSIG SOA 8 3 60 (
                                20211111000000 20211021000000 14634 zoneNNN.dnslab.org.
                                NwewpF4JnrwWOj6TInXN4mcczGySQ21uVZJXkCzQcLnH
                                GrA8sZwG16yM97YjYSIUf2HqRSlAs1e7sxWwI+QACLg5
                                BG9W3qelTjeczV3aQex5r2dsjOofPuAPTth8FmOJslPB
                                9klh3RDUtWLTmeDzr82lucc4NtvU0zFUpvL1UOwLWg0x
                                J/N/EshkZBwN9odpGcZU+FIyaci7hs/FYXSnJS0FD3Tp
                                MJcsSjt9u4aNtwQbBzWe/jIteyDlZxcmk0lH )

;; Query time: 27 msec
;; SERVER: 67.207.67.3#53(67.207.67.3)
;; WHEN: Wed Nov 03 20:37:59 UTC 2021
;; MSG SIZE  rcvd: 345

• Wait for the maximum TTL in the zone
• Remove the old ZSK from the zone

% pdnsutil remove-zone-key zoneNNN.dnslab.org 1
% pdnsutil rectify-zone zoneNNN.dnslab.org
Adding NSEC ordering information
% pdnsutil increase-serial zoneNNN.dnslab.org
SOA serial for zone zoneNNN.dnslab.org set to 1635972173

• The zone should now only have one ZSK left in the DNSKEY record set

% pdnsutil show-zone zoneNNN.dnslab.org
Nov 03 20:41:31 gmysql Connection successful. Connected to database 'powerdns' on '/run/mysqld/mysqld.sock'.
Nov 03 20:41:31 gmysql Connection successful. Connected to database 'powerdns' on '/run/mysqld/mysqld.sock'.
This is a Master zone
Last SOA serial number we notified: 1635971832 == 1635971832 (serial in the database)
Metadata items: None
Zone has NSEC semantics
keys:
ID = 4 (ZSK), flags = 256, tag = 14634, algo = 8, bits = 1536     Active         Published  ( RSASHA256 )
ID = 2 (KSK), flags = 257, tag = 26177, algo = 8, bits = 2048     Active         Published  ( RSASHA256 )
KSK DNSKEY = zoneNNN.dnslab.org. IN DNSKEY 257 3 8 AwEAAdmaHnizfuBb89XXhYBoQBnsJ/d+K7zkelu8rAvbsk3pzgwWMEufNXEqBxyQxxKYmqk/m+Wcq2UtuGnRugMfk2iaCod9DHp2iJj9qZ6C2Gqv9jVzwK1OE3yHq9xTuzq408ibgrxQvhZwYcH4mbqTlopFz/qACNKYdr52zCFnmK3cjHSphNqo3pGlBBGFLrXdLS8oHW6Ec+G7DObTqk+rGUuKHCStG0qPKoKj20L0j2frmcG5h5X0aH+7vHyBeikwuJoyqiwmEg0PYmknAnicuh3ubcPB5P+5ZcItpe6xY52eKqWIjsjcZI0aUmlOqI/ISuwP5b7UhYPgEc8JWlu6T9E= ; ( RSASHA256 )
DS = zoneNNN.dnslab.org. IN DS 26177 8 1 2416dc421e6debc0c0fd75b8742086df862f054f ; ( SHA1 digest )
DS = zoneNNN.dnslab.org. IN DS 26177 8 2 5993bff9c0f53af0162495f9b5c72a8fc5d05b24602e3905d3f4641da55a6f2c ; ( SHA256 digest )
DS = zoneNNN.dnslab.org. IN DS 26177 8 4 2810a931cf90eb4977eafdaaa3cfc8d90bb439ab836b3e05b23aef88ea0368f200a1b145472d36efc1b40c3567af3e71 ; ( SHA-384 digest )

• The new DNSKEY record set should also be seen in the DNS zone

% dig dnskey zoneNNN.dnslab.org +multi

; <<>> DiG 9.16.22-Debian <<>> dnskey zoneNNN.dnslab.org +multi
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16831
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;zoneNNN.dnslab.org.    IN DNSKEY

;; ANSWER SECTION:
zoneNNN.dnslab.org.     60 IN DNSKEY 256 3 8 (
                                AwEAAa56Jv3sjwd2yKNbbdcoTNMViBFDv3StAD1Kif7+
                                G/X993ac8cwevgp77dgGx/gmdLTu9/sV3df5/Y31l9j/
                                55QzquNU4Svxnkbf+4onN9dB0Vzh+ASUzrCTfu592tso
                                xpM6YxQUbjgYDI8308u/dgcBSsl5uaJjIcxUNUaP/oC0
                                qn2l4x7TtmHsoMXlrjJB5/oChdvyuU10i0ts3GOpJmka
                                vuynXawWuE5LQAXxNS+POXbKCHf7dzbtUzAQjFBXAw==
                                ) ; ZSK; alg = RSASHA256 ; key id = 14634
zoneNNN.dnslab.org.     60 IN DNSKEY 257 3 8 (
                                AwEAAdmaHnizfuBb89XXhYBoQBnsJ/d+K7zkelu8rAvb
                                sk3pzgwWMEufNXEqBxyQxxKYmqk/m+Wcq2UtuGnRugMf
                                k2iaCod9DHp2iJj9qZ6C2Gqv9jVzwK1OE3yHq9xTuzq4
                                08ibgrxQvhZwYcH4mbqTlopFz/qACNKYdr52zCFnmK3c
                                jHSphNqo3pGlBBGFLrXdLS8oHW6Ec+G7DObTqk+rGUuK
                                HCStG0qPKoKj20L0j2frmcG5h5X0aH+7vHyBeikwuJoy
                                qiwmEg0PYmknAnicuh3ubcPB5P+5ZcItpe6xY52eKqWI
                                jsjcZI0aUmlOqI/ISuwP5b7UhYPgEc8JWlu6T9E=
                                ) ; KSK; alg = RSASHA256 ; key id = 26177

;; Query time: 135 msec
;; SERVER: 67.207.67.3#53(67.207.67.3)
;; WHEN: Wed Nov 03 20:43:36 UTC 2021
;; MSG SIZE  rcvd: 535

• To start a KSK rollover with PowerDNS, create, publish and activate a new KSK (same algorithm as existing KSK, size can differ)

% pdnsutil add-zone-key zoneNNN.dnslab.org KSK active published 2048 rsasha256
Added a KSK with algorithm = 8, active=1
Requested specific key size of 2048 bits
5
% pdnsutil rectify-zone zoneNNN.dnslab.org
% pdnsutil increase-serial zoneNNN.dnslab.org

• The zone should now have 2 KSK DNSKEY records

% pdnsutil show-zone zoneNNN.dnslab.org
This is a Master zone
Last SOA serial number we notified: 1635972173 != 1635972638 (serial in the database)
Metadata items: None
Zone has NSEC semantics
keys:
ID = 4 (ZSK), flags = 256, tag = 14634, algo = 8, bits = 1536     Active         Published  ( RSASHA256 )
ID = 2 (KSK), flags = 257, tag = 26177, algo = 8, bits = 2048     Active         Published  ( RSASHA256 )
KSK DNSKEY = zoneNNN.dnslab.org. IN DNSKEY 257 3 8 AwEAAdmaHnizfuBb89XXhYBoQBnsJ/d+K7zkelu8rAvbsk3pzgwWMEufNXEqBxyQxxKYmqk/m+Wcq2UtuGnRugMfk2iaCod9DHp2iJj9qZ6C2Gqv9jVzwK1OE3yHq9xTuzq408ibgrxQvhZwYcH4mbqTlopFz/qACNKYdr52zCFnmK3cjHSphNqo3pGlBBGFLrXdLS8oHW6Ec+G7DObTqk+rGUuKHCStG0qPKoKj20L0j2frmcG5h5X0aH+7vHyBeikwuJoyqiwmEg0PYmknAnicuh3ubcPB5P+5ZcItpe6xY52eKqWIjsjcZI0aUmlOqI/ISuwP5b7UhYPgEc8JWlu6T9E= ; ( RSASHA256 )
DS = zoneNNN.dnslab.org. IN DS 26177 8 1 2416dc421e6debc0c0fd75b8742086df862f054f ; ( SHA1 digest )
DS = zoneNNN.dnslab.org. IN DS 26177 8 2 5993bff9c0f53af0162495f9b5c72a8fc5d05b24602e3905d3f4641da55a6f2c ; ( SHA256 digest )
DS = zoneNNN.dnslab.org. IN DS 26177 8 4 2810a931cf90eb4977eafdaaa3cfc8d90bb439ab836b3e05b23aef88ea0368f200a1b145472d36efc1b40c3567af3e71 ; ( SHA-384 digest )
ID = 5 (KSK), flags = 257, tag = 45824, algo = 8, bits = 2048     Active         Published  ( RSASHA256 )
KSK DNSKEY = zoneNNN.dnslab.org. IN DNSKEY 257 3 8 AwEAAaicwPSEdFMZAfofnKAVbAb6nzZyRqYW+1GLNIMKFRw/YZ97Gi+ZfAjkJWOkeVLbNmeKgZSMz185NkoSwlWFdrJv8hR43Hu1UJgMdKr3EzKti7W8U3YwEnBl7W1yv518LhOPFata3H5i082/qiP2c9j4JXmH1fC1TqtwTDGDpS9myPgidRgMPqeqtZ+5aQYfcEo+Kw0EJT0xD1lUURGwUztEdM4nfjIcaMf6o3//Te5KJZEhhesBgUvpbYN13ZQjA3EmzcQqoFj8HW9QZ8lC7JlYxHucA+SluGgg4BmkefAlUhuPAuF9cLvri720DVATqSQwezTIgJNromdhfjnpvec= ; ( RSASHA256 )
DS = zoneNNN.dnslab.org. IN DS 45824 8 1 c2eff1df58bf152036fd8d556553c1096136f0d0 ; ( SHA1 digest )
DS = zoneNNN.dnslab.org. IN DS 45824 8 2 33509b6659026377b337b7009442f45ae0d2a08b21e3ba726094bbac0f121635 ; ( SHA256 digest )
DS = zoneNNN.dnslab.org. IN DS 45824 8 4 213c2eba14d47e7418feea186ff0c565accd81cc65353524bff2c9894fb203995fea8b868046932a7d3bedb20351ee99 ; ( SHA-384 digest )

• Two KSK DNSKEY records are in the zone

% dig dnskey zoneNNN.dnslab.org +multi

; <<>> DiG 9.16.22-Debian <<>> dnskey zoneNNN.dnslab.org +multi
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 3417
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;zoneNNN.dnslab.org.    IN DNSKEY

;; ANSWER SECTION:
zoneNNN.dnslab.org.     60 IN DNSKEY 256 3 8 (
                                AwEAAa56Jv3sjwd2yKNbbdcoTNMViBFDv3StAD1Kif7+
                                G/X993ac8cwevgp77dgGx/gmdLTu9/sV3df5/Y31l9j/
                                55QzquNU4Svxnkbf+4onN9dB0Vzh+ASUzrCTfu592tso
                                xpM6YxQUbjgYDI8308u/dgcBSsl5uaJjIcxUNUaP/oC0
                                qn2l4x7TtmHsoMXlrjJB5/oChdvyuU10i0ts3GOpJmka
                                vuynXawWuE5LQAXxNS+POXbKCHf7dzbtUzAQjFBXAw==
                                ) ; ZSK; alg = RSASHA256 ; key id = 14634
zoneNNN.dnslab.org.     60 IN DNSKEY 257 3 8 (
                                AwEAAaicwPSEdFMZAfofnKAVbAb6nzZyRqYW+1GLNIMK
                                FRw/YZ97Gi+ZfAjkJWOkeVLbNmeKgZSMz185NkoSwlWF
                                drJv8hR43Hu1UJgMdKr3EzKti7W8U3YwEnBl7W1yv518
                                LhOPFata3H5i082/qiP2c9j4JXmH1fC1TqtwTDGDpS9m
                                yPgidRgMPqeqtZ+5aQYfcEo+Kw0EJT0xD1lUURGwUztE
                                dM4nfjIcaMf6o3//Te5KJZEhhesBgUvpbYN13ZQjA3Em
                                zcQqoFj8HW9QZ8lC7JlYxHucA+SluGgg4BmkefAlUhuP
                                AuF9cLvri720DVATqSQwezTIgJNromdhfjnpvec=
                                ) ; KSK; alg = RSASHA256 ; key id = 45824
zoneNNN.dnslab.org.     60 IN DNSKEY 257 3 8 (
                                AwEAAdmaHnizfuBb89XXhYBoQBnsJ/d+K7zkelu8rAvb
                                sk3pzgwWMEufNXEqBxyQxxKYmqk/m+Wcq2UtuGnRugMf
                                k2iaCod9DHp2iJj9qZ6C2Gqv9jVzwK1OE3yHq9xTuzq4
                                08ibgrxQvhZwYcH4mbqTlopFz/qACNKYdr52zCFnmK3c
                                jHSphNqo3pGlBBGFLrXdLS8oHW6Ec+G7DObTqk+rGUuK
                                HCStG0qPKoKj20L0j2frmcG5h5X0aH+7vHyBeikwuJoy
                                qiwmEg0PYmknAnicuh3ubcPB5P+5ZcItpe6xY52eKqWI
                                jsjcZI0aUmlOqI/ISuwP5b7UhYPgEc8JWlu6T9E=
                                ) ; KSK; alg = RSASHA256 ; key id = 26177

;; Query time: 215 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Wed Nov 03 20:54:11 UTC 2021
;; MSG SIZE  rcvd: 811

• Send the DS-Record or the DNSKEY record of the new KSK to the operator of the parent zone
• Wait for the new DS record to appear in the parent zone
• Remove the old KSK from the zone

% pdnsutil remove-zone-key zoneNNN.dnslab.org 2
% pdnsutil rectify-zone zoneNNN.dnslab.org
Adding NSEC ordering information
% pdnsutil increase-serial zoneNNN.dnslab.org
SOA serial for zone zoneNNN.dnslab.org set to 1635973094

• KSK rollover completed

• PowerDNS supports a special NSEC3 mode called the narrow mode
  • In this mode, it will create NSEC3 records on the fly that cover just the minimal gap in the zone from the requested domain name
  • NSEC3 narrow prevents zone walking
  • NSEC3 narrow mode requires life-signing, which demands that all authoritative server have access to the private DNSSEC keys (a potential security risk)
    • All PowerDNS server authoritative for a NSEC3 narrow signed zone should operate in native mode (not primary / secondary mode)
    • narrow mode is recommended for zones that contain sensitive data (such as email addresses -> SMIMEA or OPENPGPKEY data)
• To switch to NSEC3 narrow mode

% pdnsutil set-nsec3 zoneNNN.dnslab.org "1 0 10 -" narrow
% pdnsutil rectify-zone zoneNNN.dnslab.org
% pdnsutil increase-serial zoneNNN.dnslab.org

• Narrow-Mode NSEC3 records

% dig srv zoneNNN.dnslab.org +dnssec @1.1.1.1 +multi

; <<>> DiG 9.16.22-Debian <<>> srv zoneNNN.dnslab.org +dnssec @1.1.1.1 +multi
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19702
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 4, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1232
;; QUESTION SECTION:
;zoneNNN.dnslab.org.    IN SRV

;; AUTHORITY SECTION:
zoneNNN.dnslab.org.     60 IN SOA pdns010a.dnslab.org. hostmaster.zoneNNN.dnslab.org. (
                                1635973813 ; serial
                                3600       ; refresh (1 hour)
                                1800       ; retry (30 minutes)
                                3542400    ; expire (5 weeks 6 days)
                                60         ; minimum (1 minute)
                                )
zoneNNN.dnslab.org.     60 IN RRSIG SOA 8 3 60 (
                                20211111000000 20211021000000 14634 zoneNNN.dnslab.org.
                                SYfqwgtasVpUhKpBPAutbNTeXdGuY3A2F8Nv0cYFwqVI
                                uJDeJiRpUVk9HbqPqh6b7Q7QNFt7c5lD9onT+kegkqHR
                                5DEKAzPMPcNTdqx9ZjQrYXZrDpin7Zo/Mf+kEPCMArgC
                                opDlfppFhEcUSZL3mHU/WPJlj1Zr9JwIq+Ll+TCiaJ02
                                tY4a59LTEcB10HNrD5KHdURU7EE9uwg6BhEFXzvNAnW7
                                uHTYbd6v8K7yj2bukDAn3qfHTwBpgcgRKE3u )
6egermavujrtij09ssmf3r4kg1bhbihl.zoneNNN.dnslab.org. 60 IN NSEC3 1 0 10 - (
                                6EGERMAVUJRTIJ09SSMF3R4KG1BHBIHM
                                A NS SOA RRSIG DNSKEY NSEC3PARAM )
6egermavujrtij09ssmf3r4kg1bhbihl.zoneNNN.dnslab.org. 60 IN RRSIG NSEC3 8 4 60 (
                                20211111000000 20211021000000 14634 zoneNNN.dnslab.org.
                                MHO22RkxtdJmcXsDeij7JuKVHMLVF1SwvreztmCVAebK
                                /dGv98g+843LoCXi2yDRmYWeKtVodsATmr5+SI3JV4a5
                                TGG9JMThzcJ5ytHHYXp+17DoSygx0ppI8y8mbAYFBhLM
                                1HLpSBWHw9YzTzJhRqkeG7R0E6hTSX1y0fSJt58ov+hU
                                MyiwZwN9HMw+ulDLDa5DxEUXfNar+PS9WC08EAemiitZ
                                6a2xgH0fHCHWky+go2DddRrQvM6BkTIMsvc9 )

;; Query time: 39 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Wed Nov 03 21:10:23 UTC 2021
;; MSG SIZE  rcvd: 667

• often DNSSEC validation issues are caused by operational issues (expired DNSSEC signatures, DS record and DNSKEY KSK mismatch etc)
  • Negative Trust Anchors can be used to disable DNSSEC validation for mis-configured domains
• Negative Trust Anchors are defined in RFC 7646 Definition and Use of DNSSEC Negative Trust Anchors
• Negative trust anchors (nta) disable DNSSEC validation for a specific domain for a certain amount of time
  • NTA can be used by operators in case a misconfiguration for a remote DNSSEC signed zone is detected. Care should be take to check that the DNSSEC validation failure is indeed a misconfiguration and not attack
• Domains with an NTA are processed as if there is no trust-anchor for that domain
• NTAs should always have an operational lifetime
  • Don't add a NTA and forget about it
  • Re-evaluate the NTA every few days
  • Having an NTA configured should flag a warning in the monitoring system
  • Remove the NTA as soon as the DNSSEC issue is fixed
• Example: adding an NTA:

% rec_control add-nta example.com 'Signatures expired'

• Example: list active NTAs in the PowerDNS recursor

% rec_control get-ntas

• Example: removing an NTA

% rec_control clear-nta example.com

• Removing all NTAs (be careful!)

% rec_control clear-nta '*'

• Peter Eckel: NTA als Hilfsmittel bei DNSSEC Störungen

• On a primary, a catalog zone is created and maintained as a normal primary zone. SOA record and one NS record must exist, but their values are not used except for NOTIFY messages

# pdnsutil create-zone catalog.zone
Creating empty zone 'catalog.zone'

# pdnsutil add-record catalog.zone . NS pdnsNNNa.dnslab.org
New rrset:
catalog.zone. 3600 IN NS pdnsNNNa.dnslab.org

# pdnsutil add-record catalog.zone . NS pdnsNNNb.dnslab.org
New rrset:
catalog.zone. 3600 IN NS pdnsNNNa.dnslab.org
catalog.zone. 3600 IN NS pdnsNNNb.dnslab.org

# pdnsutil replace-rrset catalog.zone . soa 60 "pdnsNNNa.dnslab.org . 1001 3600 300 360000 3600"
Current records for catalog.zone IN SOA will be replaced
New rrset:
catalog.zone. 60 IN SOA pdnsNNNa.dnslab.org . 1001 3600 300 360000 3600

• A catalog zone-file contains the SOA record and the NS records for the zone
• It must also contain a TXT record with the version number of the catalog zone protocol implementation
  • Version 1: the catalog zone protocol as it has been implemented in BIND 9.11
  • Version 2: the catalog zone protocol as described in the Internet RFC and implemented in BIND 9.16
  • DNS server software will ignore catalog zones with a version number it does not support

# pdnsutil add-record catalog.zone version TXT '"2"'
New rrset:
version.catalog.zone. 3600 IN TXT "2"

• A catalog needs to be marked as a producer (source of catalog zone data) in the zones meta-data

# pdnsutil set-kind catalog.zone producer

• On a secondary DNS server, a catalog zone is created and maintained as a secondary zone.

pdnsutil create-secondary-zone catalog.zone 178.128.198.145

• The new zone needs to be marked as a catalog zone consumer (A catalog zone that will be received and new zones will be created based on the zones content)

# pdnsutil set-kind catalog.zone consumer

• New zones will be created on the primary server

# pdnsutil create-zone example.org
Creating empty zone 'example.org'

# pdnsutil add-record example.org . NS pdnsNNNa.dnslab.org
New rrset:
example.org. 3600 IN NS pdnsNNNa.dnslab.org

# pdnsutil add-record example.org . NS pdnsNNNb.dnslab.org
New rrset:
example.org. 3600 IN NS pdnsNNNa.dnslab.org
example.org. 3600 IN NS pdnsNNNb.dnslab.org

# pdnsutil replace-rrset example.org . soa 60 "pdnsNNNa.dnslab.org . 1001 3600 300 360000 3600"
Current records for example.org IN SOA will be replaced
New rrset:
example.org. 60 IN SOA pdnsNNNa.dnslab.org . 1001 3600 300 360000 3600

• Register new zone inside the catalog zone

# pdnsutil set-catalog example.org catalog.zone

• Mark the new zone as a primary zone

# pdnsutil set-kind example.org primary

• Log output on the seconday

May 07 19:47:51 pdnsb009 pdns_server[45527]: AXFR-in zone: 'catalog.zone', primary: '178.128.198.145', Catalog-Zone backend transaction started
May 07 19:47:51 pdnsb009 pdns_server[45527]: AXFR-in zone: 'catalog.zone', primary: '178.128.198.145', Catalog-Zone create zone 'example.org'
May 07 19:47:51 pdnsb009 pdns_server[45527]: AXFR-in zone: 'catalog.zone', primary: '178.128.198.145', Catalog-Zone backend transaction committed
May 07 19:47:51 pdnsb009 pdns_server[45527]: AXFR-in zone: 'catalog.zone', primary: '178.128.198.145', zone committed with serial 1778183270
May 07 19:47:52 pdnsb009 pdns_server[45527]: AXFR-in zone: 'example.org', primary: '178.128.198.145', zone committed with serial 1001

• View the new content of the catalog zone

# dig @pdns009a.dnslab.org axfr catalog.zone +noall +answer +onesoa
catalog.zone.           60      IN      SOA     pdns009a.dnslab.org. . 1778183270 3600 300 360000 3600
catalog.zone.           3600    IN      NS      pdns009a.dnslab.org.
catalog.zone.           3600    IN      NS      pdns009b.dnslab.org.
version.catalog.zone.   0       IN      TXT     "2"
u455jfvg7dcpidisupprc6hho5u5d5ni.zones.catalog.zone. 0 IN PTR example.org.