DNS doesn't resolve in codespace

[imre-kerr-sb1]

Seems like tailscaled is unable to update the dns resolver settings.

Tailscaled log:

logtail started
Program starting: v1.34.1-t328b49c4d-g921b59a2e, Go 1.19.2-ts3fd24dee31: []string{"tailscaled", "--state=mem:"}
LogID: xxxxx
logpolicy: using system state directory "/var/lib/tailscale"
logpolicy.ConfigFromFile /var/lib/tailscale/tailscaled.log.conf: open /var/lib/tailscale/tailscaled.log.conf: no such file or directory
logpolicy.Config.Validate for /var/lib/tailscale/tailscaled.log.conf: config is nil
wgengine.NewUserspaceEngine(tun "tailscale0") ...
setting link attributes: netlink receive: no such file or directory
router: v6nat = true
dns: resolvedIsActuallyResolver error: resolv.conf doesn't point to systemd-resolved; points to [127.0.0.53 168.63.129.16]
dns: [rc=resolved resolved=not-in-use ret=direct]
dns: using "direct" mode
dns: using *dns.directManager
link state: interfaces.State{defaultRoute=eth0 ifs={docker0:[172.17.0.1/16] eth0:[172.16.5.4/24]} v4=true v6=false}
magicsock: disco key = d:xxxxx
Creating WireGuard device...
Bringing WireGuard device up...
external route: up
Bringing router up...
Clearing router settings...
Starting link monitor...
Engine created.
pm: migrating "_daemon" profile to new format
got LocalBackend in 2.104s
Start
Backend: logs: be:xxxxx fe:
Switching ipn state NoState -> NeedsLogin (WantRunning=false, nm=false)
blockEngineUpdates(true)
wgengine: Reconfig: configuring userspace WireGuard config (with 0/0 peers)
wgengine: Reconfig: configuring router
wgengine: Reconfig: configuring DNS
dns: Set: {DefaultResolvers:[] Routes:{} SearchDomains:[] Hosts:0}
dns: Resolvercfg: {Routes:{} Hosts:0 LocalDomains:[]}
dns: OScfg: {Nameservers:[] SearchDomains:[] MatchDomains:[] Hosts:[]}
health("overall"): error: state=NeedsLogin, wantRunning=false
Start
generating new machine key
machine key written to store
Backend: logs: be:xxxxx fe:
Switching ipn state NoState -> NeedsLogin (WantRunning=true, nm=false)
blockEngineUpdates(true)
control: client.Shutdown()
control: client.Shutdown: inSendStatus=0
control: mapRoutine: quit
control: Client.Shutdown done.
StartLoginInteractive: url=false
control: client.Login(false, 6)
control: LoginInteractive -> regen=true
control: doLogin(regen=true, hasUrl=false)
control: control server key from https://controlplane.tailscale.com: ts2021=[fSeS+], legacy=[nlFWp]
control: Generating a new nodekey.
control: RegisterReq: onode= node=[Zi5HA] fup=false nks=false
control: creating new noise client
control: RegisterReq: got response; nodeKeyExpired=false, machineAuthorized=false; authURL=true
control: AuthURL is https://login.tailscale.com/a/xxxxxxxx
Received auth URL: https://login.tailsc...
popBrowserAuthNow: url=true
blockEngineUpdates(true)
stopEngineAndWait...
requestEngineStatusAndWait
requestEngineStatusAndWait: waiting...
requestEngineStatusAndWait: got status update.
stopEngineAndWait: done.
control: doLogin(regen=false, hasUrl=true)
control: RegisterReq: onode= node=[Zi5HA] fup=true nks=false
control: RegisterReq: got response; nodeKeyExpired=false, machineAuthorized=true; authURL=false
blockEngineUpdates(false)
active login: xxxx
Switching ipn state NeedsLogin -> Starting (WantRunning=true, nm=true)
magicsock: SetPrivateKey called (init)
wgengine: Reconfig: configuring userspace WireGuard config (with 1/5 peers)
wgengine: Reconfig: configuring router
monitor: RTM_NEWROUTE: src=, dst=10.xx.0.0/16, gw=, outif=10, table=52
monitor: RTM_NEWROUTE: src=, dst=10.xx.xx.0/24, gw=, outif=10, table=52
Taildrop disabled; no state directory
peerapi starting without Taildrop directory configured
peerapi: serving on http://100.64.222.158:33280
peerapi: serving on http://[fd7a:115c:a1e0:efe3::6440:de9e]:33280
Switching ipn state Starting -> Running (WantRunning=true, nm=true)
health("router"): error: setting up filter/ts-input: running [/usr/sbin/ip6tables -t filter -N ts-input --wait]: exit status 3: ip6tables v1.8.4 (legacy): can't initialize ip6tables table `filter': Table does not exist (do you need to insmod?)
Perhaps ip6tables or your kernel needs to be upgraded.
magicsock: home is now derp-14 (ams)
magicsock: endpoints changed: 20.234.135.20:1025 (stun), 172.16.5.4:60438 (local), 172.17.0.1:60438 (local)
control: NetInfo: NetInfo{varies=false hairpin=false ipv6=false ipv6os=true udp=true icmpv4=false derp=#14 portmap= link=""}
magicsock: adding connection to derp-14 for home-keep-alive
magicsock: 1 active derp conns: derp-14=cr0s,wr0s
derphttp.Client.Connect: connecting to derp-14 (ams)
magicsock: derp-14 connected; connGen=1
network-lock unavailable; no state directory

/etc/resolv.conf:

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "systemd-resolve --status" to see details about the actual nameservers.

nameserver 127.0.0.53
search k3lhcgm3d11urhp2rjmnl5p2jd.ax.internal.cloudapp.net
options timeout:1 attempts:5
nameserver 168.63.129.16

devcontainer.json

{
  "runArgs": ["--device=/dev/net/tun"],
  "features": {
      // ...
      "ghcr.io/tailscale/codespace/tailscale": {}
      // ...
  }
}

dig output

$ dig xxx.tailxxxx.ts.net

; <<>> DiG 9.16.1-Ubuntu <<>> xxx.tailxxxx.ts.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 15864
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;xxx.tailxxxx.ts.net.         IN      A

;; Query time: 120 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Thu Jan 12 09:57:29 UTC 2023
;; MSG SIZE  rcvd: 50

$ dig @100.100.100.100 xxx.tailxxxx.ts.net

; <<>> DiG 9.16.1-Ubuntu <<>> @100.100.100.100 xxx.tailxxxx.ts.net
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 19364
;; flags: qr aa rd ad; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;xxx.tailxxxx.ts.net.         IN      A

;; Query time: 0 msec
;; SERVER: 100.100.100.100#53(100.100.100.100)
;; WHEN: Thu Jan 12 09:57:34 UTC 2023
;; MSG SIZE  rcvd: 39

When doing that last one, I get the following in tailscaled log:

dns: resolver: forward: no upstream resolvers set, returning SERVFAIL

[mausch]

Having the same issue here.
tailscale status says:

# Health check:
#     - router: setting up filter/ts-input: running [/usr/sbin/ip6tables -t filter -N ts-input --wait]: exit status 3: ip6tables v1.8.7 (legacy): can't initialize ip6tables table `filter': Table does not exist (do you need to insmod?)
Perhaps ip6tables or your kernel needs to be upgraded.

[mausch]

This seems to work fine now 🤷‍♂️
To get it to work I've had to set "privileged": true

[andrewoke]

Having the same issue here. tailscale status says:

# Health check:
#     - router: setting up filter/ts-input: running [/usr/sbin/ip6tables -t filter -N ts-input --wait]: exit status 3: ip6tables v1.8.7 (legacy): can't initialize ip6tables table `filter': Table does not exist (do you need to insmod?)
Perhaps ip6tables or your kernel needs to be upgraded.

There are other issues (tailscale/tailscale#3996) that reference this. For me: running ubuntu-2204 image this worked:

rm -f /sbin/ip6tables && ln -s /sbin/ip6tables-nft /sbin/ip6tables
sudo tailscale down # if it's already running
sudo tailscale up --accept-routes

Using this, I didn't need privileged, root or anything else.

[fonewiz]

Having the same issue here. tailscale status says:

# Health check:
#     - router: setting up filter/ts-input: running [/usr/sbin/ip6tables -t filter -N ts-input --wait]: exit status 3: ip6tables v1.8.7 (legacy): can't initialize ip6tables table `filter': Table does not exist (do you need to insmod?)
Perhaps ip6tables or your kernel needs to be upgraded.

There are other issues (tailscale/tailscale#3996) that reference this. For me: running ubuntu-2204 image this worked:

rm -f /sbin/ip6tables && ln -s /sbin/ip6tables-nft /sbin/ip6tables
sudo tailscale down # if it's already running
sudo tailscale up --accept-routes

Using this, I didn't need privileged, root or anything else.

Thanks for this, fixed my issue completely. I wanted to add that I had to issue sudo su root first though.