Featured image of post Network Namespaces and Docker

Network Namespaces and Docker

the first post in the series Container Networking Explained


The video Network Namespaces Basics Explained in 15 Minutes is a good introduction to how Linux network namespaces work. However, this video has several mistakes, so getting a basic idea of the command line is sufficient.

Set up the environment

To experiment with the content in this article, you need a Linux machine. If you don’t have Linux installed, a virtual machine also works well. You can also use my Github repository network-namespace-experiment, which provides a minimal Vagrantfile and the scripts to do the experiment.

We can set up two network namespaces connected to a bridge with the following commands:

# two namespaces
ip netns add red
ip netns add blue
# the bridge connecting the two namespaces
ip link add bridge0 type bridge
# the veth pair connecting red namespace to the bridge
ip link add veth-red type veth peer name veth-red-br
ip link set veth-red netns red
ip link set veth-red-br master bridge0
# turn the interfaces up
ip -n red link set veth-red up
ip link set veth-red-br up
# do the same for blue namespace
ip link add veth-blue type veth peer name veth-blue-br
ip link set veth-blue netns blue
ip link set veth-blue-br master bridge0
ip -n blue link set veth-blue up
ip link set veth-blue-br up
# turn the bridge up
ip link set bridge0 up

The namespaces red and blue are connected to a bridge, bridge0, like the diagram below.


Network namespaces communication

How can the two network namespaces, red and blue, communicate with each other? The answer is: we need to assign IP addresses to them. The following commands assign to red and to blue.

ip -n red addr add dev veth-red
ip -n blue addr add dev veth-blue

ping to blue from red to verify that it works.

ip netns exec red ping

How does it work? It turns out that a routing rule is added to the route table of red automatically when assigning it the IP address dev veth-red scope link

The routing rule has no gateway address, and the scope is set to link. This is exactly what the network prefix /24 in the IP address implies: All the addresses in the range could be reached directly from interface veth-red, without going through any routers.

The same goes for network namespace blue, where the following routing rule is added automatically. dev veth-blue scope link

This is exactly how Docker bridge network works. The difference is that the default Docker bridge is called docker0 instead of bridge0, which is created when the daemon dockerd starts.

Network namespaces and host communication

To communicate with the host, we have to assign an IP address to the bridge0 interface.

ip addr add dev bridge0

Now we can reach the network namespaces from the host


and the host can be reached from the network namespace using the IP address of the bridge.

ip netns exec red ping

This is exactly how Docker enables container-to-host communication for bridge networks. See this question for more details.

What are separated by network namespaces?

Run ip link and then ip -n red link, we see that network interfaces are separated. Moreover, other networking configurations, like route table (shown by ip route) and iptables are also separated. In other words, network namespaces are like separate machines to the host.

Notice that Docker (and other container tools) use several namespaces in combination. In addition to network namespaces, they use PID namespaces, mount namespaces, UTS namespaces, etc. However, for this article, we only care about network namespaces. To learn about other namespaces, you can find resources in Containers Deep Dive.

What is a virtual bridge?

The configuration becomes the following after IP addresses are added.

setup with ip addresses

You might wonder what a bridge is and why a bridge is added by the ip link command, which is normally used to manage network interfaces. The word “bridge” is just a synonym for “switch”, but a Linux virtual bridge is more than a switch. It’s a switch + an interface. The setup above is equivalent to the following physical network setup.

physical setup

The name bridge0 appears twice: It’s not only the bridge connecting red, blue, and the host machine together, but also the interface that connects to the bridge on the host machine. In other words, it’s a bridge from the perspective of the network namespaces, but it’s an interface from the perspective of the host machine. When we add an IP address to bridge0 using ip addr add dev bridge0, we are actually adding an IP address to the interface. It’s crucial to understand the “double roles” of a virtual bridge.

In Home network configuration, I said that a typical home router has a switch built-in. Actually the switch is usually a virtual bridge managed by software instead of a physical switch.

Access the Internet from a network namespace

To access the Internet from red, first we add a default routing rule which sends non-local traffic to the host machine.

ip -n red route add default via dev veth-red

We should also set up NAT in the host machine. According to SNAT in iptables, the rule should look like the following (! stands for negation).

iptables -t nat -A POSTROUTING -s ! -o bridge0 -j MASQUERADE

Instead of specifying a particular output interface, we use the “!” sign to specify “any interfaces that are not bridge0”. This is because the name of the output interface on different machines might be different, and a machine might even have more than one output interface (for example, a laptop might have both wired and Wi-Fi interfaces).

Then, run the following ping command to verify red has Internet access.

ip netns exec red ping

If it doesn’t work, turn on the kernel parameter net.ipv4.ip_forwarding and try again.

sysctl -w net.ipv4.ip_forward=1

This is exactly how Docker containers access the Internet. In summary, we need

  • a default routing rule sending traffic to the host machine
  • a MASQUERADE rule in the host machine setting up source NAT
  • net.ipv4.ip_forward=1

Docker sets up the MASQUERADE rule and the parameter net.ipv4.ip_forward=1 when dockerd starts, and the default routing rule is configured when a container starts.

Hosts as routers

Why do we need the kernel parameter net.ipv4.ip_forwarding=1? It turns out that it’s because the host machine should act as a router. Remember that routers are devices that accept traffic that does not go directly to themselves. In our scenario, the host machine takes traffic from destining, but its own IP address is, neither the source address nor the destination address. Hence it acts as a router. The difference between the host and a regular router is that instead of routing traffic for physical machines, the host routes traffic for network namespaces running on itself.

net.ipv4.ip_forward is the parameter controlling whether the host could act as a router. To set the parameter, use the following command.

sysctl -w net.ipv4.ip_forward=1

sysctl is the command to read and write Linux kernel parameters. Do not confuse it with systemctl, which is used to interact with systemd.

sysctl doesn’t preserve the configuration if you reboot your machine. To set it permanently, add a file under /etc/sysctl.d/ (like /etc/sysctl.d/99-docker.conf), and add the following line to this file.


In this way, systemd will set this parameter at boot time.

Publish a port

Let’s run a web server in red on port 80

ip netns exec red python3 -m http.server 80

How to access it from port 8080 on the host?

Different machine access

We need the following DNAT rule for port 8080 to be accessible from a different machine on the same network as the host, according to DNAT in iptables.

iptables -t nat -A PREROUTING -p tcp --dport 8080 -j DNAT --to-destination

Host access

To be able to access the published port locally from the host, we need the following according to Router access:

iptables -t nat -A OUTPUT -p tcp --dport 8080 -j DNAT --to-destination
iptables -t nat -A POSTROUTING -m addrtype --src-type LOCAL -o bridge0 -j MASQUERADE
sysctl -w net.ipv4.conf.bridge0.route_localnet=1

Suppose your machine is connected to the Internet using an interface whose IP address is (this is the default for virtual machines on VirtualBox), now access the published port by curl, curl and curl to make sure everything works well.

What we have added are

  • a DNAT rule in the OUTPUT chain
  • a SNAT rule to correct the source address of packets sending to the network namespaces so that we can access the published port through addresses other than bridge0, such as or
  • route_localnet, which allows localhost addresses to pass reroute-check

See Router access for why we need them.

Access from blue namespace

Try running ip netns exec blue curl and we get no response. Why?

The answer is simple: According to Hairpin NAT, what happens are:

  1. blue sends a request packet to, whose source address is
  2. The destination address gets translated to, and the packet gets sent to red
  3. red replies to blue, but the reply packet goes to blue directly through bridge0 without going through iptables. As a result, the source address of the reply packet is, without being corrected by reverse NAT
  4. blue gets the reply packet, but will not put the request and the response into the same connection since the source address of the response does not match the destination address of the request

How to solve the problem? It turns out that we do not need to turn on hairpin NAT for virtual bridges. Instead, what we need is to set a kernel parameter to 1.

sysctl -w net.bridge.bridge-nf-call-iptables=1

If the command above fails, your system might not have the kernel module br_netfilter loaded. Load the module and retry.

modprobe br_netfilter

Similar to sysctl, we can create a file under /etc/modules-load.d and write the following to the file for systemd to load the module at boot time:


By the way, this is exactly how we configure prerequisites for containerd. The standard configuration also adds net.bridge.bridge-nf-call-ip6tables=1 for ipv6 support, and an additional kernel module overlay for overlay filesystem.

When this parameter is set, iptables rules will be applied to packets going through virtual bridges. In this way, the reply packet will also go through the reverse operation of DNAT, and its source address will be corrected to match the destination of the request.

bridge NAT

I also want to call out that packets from the network namespaces like blue or red go through the PREROUTING chain instead of the OUTPUT chain, even though the namespaces run on the host machine. This is because the host machine treats the network namespaces as if they are different machines. That is why the PREROUTING chain rule in Different machine access is required even when we do not enable other machines on the same network as the host to access the published port.

Same namespace access

What if we access the exposed port from the namespace red?

ip netns exec red curl

We get no response. The option net.bridge.bridge-nf-call-iptables does not work for access from the same namespace since the reply does not even go through the bridge, so the hairpin NAT rule is needed (See Hairpin NAT):

iptables -t nat -A POSTROUTING -s -d -p tcp --dport 80 -j MASQUERADE

Try curl again, and it still doesn’t work. It turns out we have to set hairpin mode for the bridge, bridge0. When we connect to port 8080 on the host from red, the packets arrive bridge0 through interface veth-red-br and get routed back through the same interface by our DNAT rule. The bridge will reject such “going back” routing if veth-red-br’s hairpin mode is not turned on. To turn it on:

ip link set veth-red-br type bridge_slave hairpin on

Now, try ip netns exec red curl and it works!

Experiment with Docker

We described how Docker publishes ports above when the option userland-proxy is set to false. However, the default value is true. I will explain the difference later, and let’s see how to configure this option for Docker.

First, set userland-proxy to false in file /etc/docker/daemon.json (create one if not exist).

    "userland-proxy": false

Then, restart dockerd (the easiest way is to restart your computer).

Finally, run a container to experiment using

docker run --rm -it -p 8080:80 nginx

You can try everything we explained above, including kernel parameters, virtual bridge settings, route tables, and iptables rules. For example, we can inspect the iptables rules using

iptables -vL -t nat

What is the userland-proxy option? It’s an alternative way to publish ports. When its value is true, Docker will start a proxy listening on published ports and redirect the traffic to the desired container. For example, for the nginx container above, it will listen on and redirect the traffic to (Suppose nginx runs in such IP address). Setting userland-proxy to false triggers some bugs and compatibility issues in older kernels, so Docker keeps the default option to be true. You can see the discussion in this issue. When the value is true, the iptables rules are slightly different from what we have shown above since we describe an iptables-only approach.

Container Networking From Scratch