feat: forward guest DNS to Docker's embedded resolver#793
Conversation
✅ Deploy Preview for urunc ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
25a6350 to
a743f29
Compare
a743f29 to
6723c67
Compare
|
Hey @cmainas, I have implemented the changes we discussed during our sync. Could you please take a look and review them? This will allow us to make any necessary changes before we extend this logic to other unikernels. |
22a5784 to
89b24e6
Compare
cmainas
left a comment
There was a problem hiding this comment.
Hello @alimx07 ,
thank you for all the changes and the fix. A few comments:
- In the e2e testing the docker network does not exist and therefore creating the contianer will fail. We need to setup the network. Maybe in https://github.com/urunc-dev/urunc/blob/main/tests/e2e/docker_test.go#L25 and also remove later.
- Let;s find a better name for loclahost.go. Maybe something like "dns_at_localhost.go"?
- I think I got confused with the
genericRulesanddockerRules. Have we found a case where we should apply thegenericRules(e.g. a CNI) rather thandockerRules? - We can wrap the execution of iptables in a function, instead of having cmd and cmd.Run in various places.
- We should check if we can obtain the information we gather form iptables from other sources (e.g. netlink).
- We can restructure the code and this logic in the network_dynamic.go (the only mode where these changes apply) and avoid re-opening the tap device etc. We can me use of https://github.com/urunc-dev/urunc/blob/main/pkg/network/network_dynamic.go#L23C6-L23C20 for passing information.
- We should also clean up all the rules we apply.
| StaticNet: false, | ||
| SideContainers: []string{}, | ||
| Skippable: false, | ||
| TestFunc: dnsResolveExternalTest, |
There was a problem hiding this comment.
We can make this matchTest. Look at https://github.com/urunc-dev/urunc/blob/main/tests/e2e/test_cases.go#L19 for example.
| if isDocker(resolvConf) { | ||
| f.custom = dockerRules | ||
| } |
There was a problem hiding this comment.
The decision if we are going to apply the "docerRules" should be based on the localhost detection as a nameserver and not if there is a "docker" string in /etc/reolv.conf
There was a problem hiding this comment.
Do you mean like if we detect some localhost nameserver (e.g. 127.0.011), we should go with dockerRules or in general if there is localhost one, lets go with them ?
There was a problem hiding this comment.
Also if you mean the second, this will be the answer for this question and dockerRules vs genericRules one:
I think if we detect a localhost nameserver, then we know that we should apply some specific rules. by default it will be genericRules one (nat into nameserver:53, which is the default port for dns resolvers, currently I do not have specific case for this, but in a naive world this would be the answer).
If we detect other custom environment like docker in our case, then other rules should be applied (nat into some upper ports in docker case). in the future, having more environments with other custom rules if any just means add another detector in the list.
for docker detection, having docker in the resolv.conf path maybe a way, reading if there is a .dockerenv in the filesystem is another one
There was a problem hiding this comment.
I think this comes from our effort to try and generalize the approach, but without any other case rather than docker, this will be difficult. I can understand the difference between dockerRules and genericRules and this is the correct approach. However, there are two things:
- Relying on finding the keyword "docker" in
/etc/resolv/confdoes not sound a good idea. It would be better to discover a "docker topology" and decide based on that. - The
genericRulescan not be tested (at least for the time being), since we do not have a case yet.
So it would be better to assume a "docker topology" whenever we see a localhost-based DNS and let's fail when this does not apply. We can easily fix it or change our approach later.
Also, it would be useful to add comments in the code explaining the rational and possible implications.
Regarding the first question: We should focus on a single nameserver entry which points to a localhost address (like docker). If there more than ones, maybe some other entry can work.
| // Rule: DNSServer stays empty unless the rules actually got installed. | ||
| if fwd != nil && networkType == "dynamic" { | ||
| if err := fwd.Apply(networkInfo.TapDevice, networkInfo.EthDevice.Interface); err != nil { | ||
| uniklog.Warnf("failed to apply localhost forwarding rules: %v", err) |
There was a problem hiding this comment.
nit: We can return here instead of using else.
| ipt, err := exec.LookPath("iptables") | ||
| if err != nil { | ||
| return err | ||
| } |
There was a problem hiding this comment.
nit: This should be placed in dnat.
| return err | ||
| } | ||
| dst := net.JoinHostPort(f.LoIP.String(), "53") | ||
| for _, proto := range []string{"udp", "tcp"} { |
There was a problem hiding this comment.
We are not going to have more than udp and tcp as protocols here, so it would be cleaner if we unfold the loop. As it happens in docker.go
There was a problem hiding this comment.
We can merge this file with "loclahost.go" or its new name.
There was a problem hiding this comment.
what about something like local_dns.go or loDns.go as shorter one ?
There was a problem hiding this comment.
Sure, both are better than localhost.go.
89b24e6 to
39be2e9
Compare
On Docker user custom networks the DNS resolver (127.0.0.11) is loopback only and unreachable by the unikernel guest. Detect this case per container, expose the resolver via a virtual resolver IP using tc redirects between the tap and lo + a PREROUTING DNAT, and rewrite the guest's resolv.conf to point at that IP. Signed-off-by: Ali Mohamed <amx746@gmail.com>
39be2e9 to
7e73541
Compare
| info.DNSServer = n.DNSForwarder.VirtIP.String() | ||
| netlog.Debugf("localhost forwarding applied: virtual resolvIP %s", info.DNSServer) | ||
| } | ||
|
|
There was a problem hiding this comment.
Since users are already familiar with the urunc localhost resolver issue, making this a hard failure is probably the wrong move. I'll fail open here for now. We can always flag it later if needed.
| _, err := commonCmdExec(cmdBase) | ||
| return err | ||
| } | ||
|
|
There was a problem hiding this comment.
@cmainas , Regarding network. I think using the same pattern as container (rm , create) is the cleanest way here.
I know this is out of the scope of the PR now, and it just being larger and larger, but what touched here is minimal as possible. Also , a future PR will be opened (by Me) adding more test cases (instead of docker) touching other CLI tools. so if you are good, we can move with this right now.
| } | ||
| return tcpPort, udpPort, nil | ||
| } | ||
|
|
There was a problem hiding this comment.
Unfortunately, this doesn't affect the performance much. It's basically the same (plus a few microseconds).
I still think we can optimize this, but I would rather move forward, finalize support for the other unikernels, and come back to this later.
Description
On Docker user custom networks the DNS resolver (127.0.0.11) is loopback only and unreachable by the unikernel guest. Detect this case per container, expose the resolver via a virtual resolver IP
configured by userusing tc redirects between the tap and lo , and rewrite the guest'sresolv.confto point at that IP in our unikernel islinuxone. Also we add a custom rules DNAT rules according to the enviroment (e.g. Docker -> PREROUTING DNAT(for DNS server ports))Two TC rules added on tap and lo:
src=ResolvIPinto TapRelated issues
How was this tested?
Tested in user custom docker network with urunc container
u1and normal oneu2:u1lookup ongithub.comu1lookup onu2u2lookup onu1LLM usage
Checklist
make lint).make test_ctr,make test_nerdctl,make test_docker,make test_crictl).