Contact

Subscribe via Email

Subscribe via RSS

Categories

Creative Commons Attribution 4.0 International License
© Rakhesh Sasidharan

Elsewhere

Notes on Teredo (part 4)

Continuing my posts on Teredo, consider the following network:

IPv6 - Teredo

There are two Windows clients behind two different NAT routers. There’s a Teredo server and a Teredo relay. And there are IPv6 only and IPv6/IPv4 resources on the Internet that these clients would like to access. The Teredo server and relay are on different networks and the Teredo relay is in a different network from the clients and IPv6 resources. They don’t need to be on different networks or even different servers; I just put them so to illustrate that they are independent devices and not necessarily on the same network as the client and resources.

The clients communicate with the Teredo server and relay through IPv6 packets in UDP messages in IPv4 packets. The Teredo server and relay communicate with the IPv6 parts of the Internet over IPv6, except when the host has both IPv4 and IPv6 in which case the host acts as its own relay and talks to the clients directly over IPv4 (using IPv6 in UDP messages as usual).

Once I configure both clients with the Teredo server address they get a Teredo IPv6 addresses as below. These are both link-local as well as 2001::/32 prefix addresses. The Teredo relay too gets Teredo IPv6 addresses while the Teredo server has link-local Teredo IPv6 addresses only.

IPv6-Teredo2

Decoding the Teredo IPv6 address

Let’s decode the Teredo IPv6 address of one of the clients to see what it means.

Client 192.168.99.200 has address 2001:0:1117:34fa:1027:374b:e1fc:f635. So the network prefix is 2001:0:1117:34fa, host identifier is 1027:374b:e1fc:f635.

Dealing with the network prefix:

  • 2001::/32 is the Teredo network prefix.
  • 1117:34fa is 32 bits of the Teredo server IPv4 address in hex. 0x11 = 17, 0x17 = 23, 0x34 = 52, 0xFA = 250. Which converts to 17.23.52.250.

And the host identifier:

  • 1027 is 16  bits of flags and random bits.
  • 374b is 16 bits of the obfuscated external port of the NAT. (Obfuscation happens by XOR-ing the digits with 0xFFFF).
    • 0x374b = 0xc8b4 unobfuscated =51380.
  • e1fc:f635 is 32 bits of obfuscated external IPv4 address of the NAT.
    • 0xe1 = 0x1e unobfuscated = 30
    • 0xfc = 0x3 unobfuscated = 3
    • 0xf6 = 0x9 unobfuscated = 9
    • 0x35 = 0xca unobfuscated = 202
    • So the NAT’s external IPv4 address is 30.3.9.202.

Packet flow

Let’s look at some Wireshark captures to see what happens. From one of the clients (192.168.20.200) I am pinging an IPv6 host and here’s what happens.

(The Teredo IPv6 addresses in these captures are different from the ones above because I made the captures a while later and the NAT IPv4 address and port changed by then).

First the client makes a DNS request (over IPv4) to resolve the name, one of the name servers replies with the IPv6 address (the AAAA records).

Capture1

Next the client contacts the Teredo server to configure itself. It sends an ICMPv6 Router Solicitation message to the ff02::2 multicast IPv6 address (link-local scope, all routers). The source is set to a link-local IPv6 address fe80::ffff:ffff:fffe which as far as I can tell seems to be an anycast address but I am not too sure. (Typically Router Solicitation messages set the source address as fe80::/64 followed by a random or EUI-64 based host identifier).

Capture2

It’s worth noting that this ICMPv6 message is actually sent within a UDP message in an IPv4 packet addresses to the Teredo server IPv4 address. Pay attention to the destination IPv4 address of this packet – it’s the first IPv4 address of the Teredo server. From the capture above you’ll see there are two Router Solicitation messages sent and two replies got. That’s because the second Router Solicitation message is to the second IPv4 address of the Teredo server. Both addresses send a reply and that’s how the Teredo client identifies whether it’s behind a symmetric NAT or not.

Capture3

In the capture above it’s also worth noting the port number used by the client’s NAT: 52643. There is now a mapping on the NAT device from this port to the client. The mapping could be a cone or restricted, but that does not matter to us (as long as it’s not symmetric).

Moving on, the Teredo client sends an IPv6 Connectivity Test message from it’s Teredo IPv6 address to the IPv6 address of the destination. The client has auto configured an IPv6 address for itself. And it knows the destination IPv6 address from the initial DNS query. This test message is really an ICMPv6 ping request to the destination, sent within an UDP message over IPv4 to the Teredo server. I had mentioned this previously in the section on Teredo relays. This is how a Teredo client finds the Teredo relay closest to the destination.

Capture4The Teredo server now does a Neighbor Solicitation to find the MAC address of the destination. From the IPv6 address it knows the destination is on the same network as itself. If it were on different networks, the Teredo server would have sent this packet to its gateway for forwarding.

Notice the destination address ff02::1:ff00:254. As mentioned in an earlier post this is a solicited node multicast address.

  • ff02 indicates the scope is link-local.
  • ::1:ffxx:xxxx is the solicited node address a host with last 24bits IPv6 address xx:xxxx would have subscribed to. For the destination host IPv6 address 2dcc:7c4e:3651:52::254, this would be 00:254.

Capture5

The destination host now sends a Neighbor Solicitation message to find its default gateway’s MAC address, gets a reply, and sends the ping reply to the Teredo client. Notice the ping reply is a pure IPv6 ICMPv6 packet.

Capture6

This packet reaches the Teredo relay as it is advertising the 2001::/32 prefix. This packet is discarded.

The Teredo relay sends a UDP message to the NAT router of client, on the port that’s mapped to it. The NAT router replies with a prohibited message. This tells the Teredo relay that any previous mappings the client NAT might have had for the relay are gone and so a new mapping must be made. Since it can’t contact the client directly it must take help from the Teredo server.

Capture7

The Teredo relay now sends a bubble packet from itself to the IPv6 Teredo address of the client. This bubble packet is sent via UDP over IPv4 to the Teredo server. (Remember this is how the need for hole punching is communicated to the server).

Capture8The Teredo server sends this packet via UDP to the port that’s mapped on the client’s NAT router. Note the IPv6 portion of the packet it sends is same as the one it receives. This way the client is able to identify the Teredo relay’s IPv4 and Teredo IPv6 address.

Capture9I was under the impression the Teredo relay address is sent to the client as an “Origin Indication” by the Teredo server but apparently the client picks it up from the IPv6 packet the relay sends it. The Teredo server doesn’t modify the IPv6 payload. The payload is a bubble packet (next header is 59) and contains details of the relay.

Now the Teredo client contacts the Teredo relay and sends it bubble packets. The relay responds and thus the client and relay know of each other.

Capture12

All this was just prep work done behind the scenes when the client wants to talk to an IPv6 host. I generated this traffic by trying to ping the IPv6 host from the client so below is the traffic for that:

Capture14

The Teredo client contacts the relay via UDP/ IPv4 and passes on the ICMPv6 ping request packet. That’s sent on the IPv6 network to the destination. A reply arrives. The reply is passed via UDP/IPv4 to the client. No more additional steps.

Below is the capture for all four ping requests and replies.

Capture15

And that’s it!

Microsoft has a good page on Teredo that’s worth looking at. Goes into more details with the processes and scenarios that I don’t go into above.

Notes on Teredo (part 3)

In the previous two parts I talked about Teredo in general and also about NAT &  Teredo. In this post I hope to talk more about how Teredo works.

Teredo Clients

Microsoft has made available Teredo servers on the Internet. These are reachable at win8.ipv6.microsoft.com and teredo.ipv6.microsoft.com and Windows clients have this address already set as their Teredo server.

If the Teredo server address is not reachable, the client is in an offline state:

If the Teredo server address is reachable, the client is in a dormant state. As the name indicates this is a state in which the Teredo client is not active, but when required it can contact the server and auto-configure an IPv6 address and send/ receive packets.

Send some IPv6 traffic and the state automatically changes to qualified. (Note how the first ping reply took a lot more time than the rest as the Teredo interface was being configured. Sometimes the first reply can timeout too).

Now the Teredo state also shows the type of NAT the client is behind and also the local and external mappings.

Another thing to note in the output above is the “Network” which is currently set to “unmanaged”. Since Teredo allows a client to be reached across a firewall/ NAT and this is something an organisation might not want for its managed machines, the Teredo client tries to accommodate that and before initializing itself it checks whether the computer is on a managed network. If the computer is domain joined and on a network where it’s domain controllers are reachable – i.e. within an organisation – the Teredo client detects that it’s on a managed network and disables itself.

This setting can be changed to set the Teredo client as qualified even in a managed network. This can be done via GPOs, PowerShell, or netsh. The netsh command for this is:

This command must be run as an administrator. When a Teredo client is in a managed network and qualified, it is known as an Enterprise Client. Hence the name.

It is also possible to configure clients with a manually specified Teredo server. This can be done via PowerShell …

… or netsh

Teredo Servers

Setting up your own Windows Teredo Server is easy. Windows 7, Windows 8, Windows Server 2008 R2, Windows Server 2012, and later can function as a Teredo server.

Here’s how I enable one of these as a Teredo server:

And that’s it really! The computer is now working as a Teredo server.

Running the above command again shows updated stats.

One thing to keep in mind that even a computer functioning as a Teredo server must be setup with a Teredo server. If the pre-configured Teredo server (e.g. win8.ipv6.microsoft.com) is unreachable, the Teredo interlace will be disabled and the computer will not work as a Teredo server. The Teredo state will show as offline even though this computer itself is a server.

To fix this, set this computer as its own Teredo server.

Teredo Relays

Previously I wrote about how a Teredo relay sends packets to a Teredo client. How do clients know which Teredo relay to use though? So far we haven’t set a Teredo relay anywhere in our client and server configuration, so where does it enter the picture?

While Teredo servers are specific to a client – i.e. the client is assigned a Teredo server and each client uses only one Teredo server – Teredo relays are specific to the remote end and a particular client will use different relays for different destinations. Here’s how the process works:

  1. When a Teredo client needs to contact a remote IPv6 host, it first sends an ICMPv6 packet to the remote host.
  2. Since it doesn’t know how to contact this host, and this is an initial setup connection, the client sends this packet to the Teredo server as an UDP message in IPv4.
  3. The Teredo server receives this message, decapsulates the IPv6 packet, and sends it on the IPv6 network. Note: this IPv6 packet has the destination address set as the IPv6 address of the remote host, and source address set as the Teredo IPv6 address of the Teredo client.
  4. Now for the fun part! The IPv6 packet reaches the destination host, the host creates a reply IPv6 packet with itself as the source and the Teredo client IPv6 address as the destination. This packet is sent on the IPv6 network. On the IPv6 network are many Teredo relays, all of them advertising the 2001:0:/32 prefix. The packet will reach the relay nearest to the destination host who will then send it to the Teredo client. Once the Teredo client receives the ICMPv6 reply, it knows which relay was used and thus knows the IPv4 address of the relay closest to the destination.
  5. The Teredo client then sends the actual IPv6 packet as a UDP message in an IPv4 packet to this Teredo relay. And since a hole punching is done for this relay address, further packets to and from this relay can travel through.

Similarly when an IPv6 host has a packet for a Teredo client, the packet makes its way to the relay closest to that host. The relay then checks whether it already has a communication set up with the client, in which case it sends the packet over via IPv4. If there’s no on-going communication, or it’s been a while, the relay goes through the hole punching process again and sends the packet.

Similar to the Teredo server, Windows 7, Windows 8, Windows Server 2008 R2, Windows Server 2012, and later can function as a Teredo relay. Setting up one of these as a Teredo relay is quite straight-forward. All one has to do is:

  1. Ensure the Teredo interface is ready – i.e. the relay can reach a Teredo server and the interface is not offline.
  2. Enable forwarding on the Teredo interface. Enable forwarding on the interface(s) to the IPv6 network.
  3. Publish a route for the 2001::.32 prefix.
  4. Enable IPv6 router advertisements on the IPv6 network so other routers pick up the published route.

And that’s it! Here are the commands:

That’s all for now!

Notes on Teredo (part 2)

Before going more into Teredo it’s worth talking about the types of NAT.

Types of NAT

When an internal device sends a packet to an Internet device, the source address and port (i.e. the private IPv4 address and port number of the internal device) are translated by the NAT. The outgoing packet from the NAT device will have the source IPv4 address set as the public IPv4 address of the NAT box and a newly assigned port on the NAT box as the source port address of the packet. And a mapping for this will be stored on the NAT box (internal IPv4 address, port number <-> Public IPv4 address, port number).

What happens next is what defines the various types of NAT:

Cone NAT

In a cone NAT, once a mapping is thus stored, packets from any external device to the public IPv4 address and port number of the mapping will be forwarded to the internal IPv4 address and port number. The key word here is any. So my machine behind the NAT can send a packet to (say) Google.com from port 1234 of my machine -> this will create a mapping on the NAT box from my internal IPv4 address and port 1234 to the public IPv4 address and (say) port 5467. Now a totally different server (say) Yahoo.com can send a packet to port 5678 of the public IPv4 address and it will be forwarded to my machine’s port 1234.

Essentially my internal machine has a port mapped on the NAT now which is always forwarded to my machine. Of course, the mapping could go after a while of disuse and then I’ll get a new external port number and mapping, but suppose I kept sending traffic to keep the mapping alive the port is forever forwarded to me.

You can imagine this looks like a cone. The tip of the cone is the public IPv4 address and port. The base – the open end – is everything in the external world. Everything in the external world can contact me on that port. It’s like a cone, hence the name.

Restricted NAT

In a restricted NAT, once a mapping is stored, only packets from the particular external device which the packets were originally sent to can connect to the mapping and be forwarded. That is, in the example above once there’s a mapping for my internal IPv4 address and port 1234 to the external public IPv4 and port 5467 for Google.com, only Google.com can send a packet to the external IPv4 address and port 5678 and it will be forwarded to my machine, if Yahoo.com sends a packet to the same IPv4 address and port it will be discarded.

There is a stricter form of restricted NAT where even the port number of the sender is checked. That is, if initial packet was to pot 9999 of Google.com, only packets from port 9999 of Google.com are allowed to enter via that mapping. Packets from port 9998 of Google.com will be silently discarded!

Symmetric NAT

Symmetric NAT takes things one step further!

In the two NAT types above the mapping is stored for all traffic from the internal device and internal IP. That is if I send packets from port 1234 and my internal IPv4, a mapping is created to port 5678 on the NAT box and that is used for all traffic from port 1234 of my internal machine. So – I contact Google.com from internal port 1234, the same mapping is used. I contact Yahoo.com from the same internal port, the same mapping is used. I contact Bing.com from the same internal port, the same mapping is used! The only difference between the two types above was to do with the incoming packets – whether they were from an IPv4 address (and port, in the stricter version) that was previously contacted. But the mapping was always the same for all traffic.

However, in Symmetric mapping a new mapping is created for each destination. So if I contact Google.com from port 1234 of my internal machine, it could be mapped to port 5678 of the NAT box and a mapping created for that. Next, if I contact Yahoo.com from port 1234 of the internal machine (same internal port as before), it could be mapped to port 5889 of the NAT box and a new mapping created for that. And later if I contact Bing.com from again the same port 1234 of the internal machine, yet another mapping will be created on the NAT box, say to port 6798. And worse, each of these mappings behaves like a Restricted NAT mapping – i.e. only the IP address to which the packets were initially sent when creating that mapping can use the mapping to send back packets.

You can see why this is called Symmetric NAT. It’s because each traffic from the internal side has its own mapping on the external side. There is a one-to-one mapping from the internal port & IPv4 address to the external port for each traffic.

Why do the types matter?

It’s important to know what type of NAT a particular is in when using Teredo. Because remember my quick overview of Teredo – at step 2 the host behind the NAT asks the server for an IPv6 address and it gets it. What this also does is that a mapping is created on the NAT box for the host’s internal IPv4 address and port number to the NAT box’s public IPv4 address and external port. Now …

If the NAT were Cone, then at step 8 a Teredo relay can forward IPv6 over IPv4 packets to the internal host by sending to the external port and NAT public IPv4 address.

But if the NAT were Restricted, then step 8 would fail as the internal host hasn’t contacted the relay yet and so there’s no mapping for the relay’s IPv4 address and/ port port  (it could be that the internal host has contacted the relay to send some packets and this is a response from the relay, in which case it will be let through, but it could also be a fresh connection from the relay forwarding packets from an IPv6 host that are not in response to anything from the internal host – these will be discarded). So in the case of a restricted NAT Teredo has some additional stuff to do first – namely create a mapping in the NAT table for the Teredo relay’s IPv4 address.

First the Teredo relay checks if the IPv4 address and port of the Teredo client – which it extracts from the Teredo IPv6 address of the client – are known to it. If it is known (as in the relay and client have communicated recently) it means a mapping exists and the relay can send the packet to the client.

If a mapping does not exist the Teredo relay takes help of the Teredo server to punch a hole in the NAT. This is akin to the Romeo & Juliet example I described yesterday. The relay needs to contact the client but the client’s NAT box will discard any packets that are not in response to something that the client has sent out, so the relay needs a third party server “friend” to help out. Here’s what happens:

  1. The Teredo relay queues the incoming packet.
  2. From the Teredo IPv6 address of the client it extracts the IPv4 address of the Teredo server.
  3. The relay then creates a “bubble packet” with the source address as the relay’s IPv6 address, destination as the client’s Teredo IPv6 address, and sends it to the Teredo server’s IPv4 address. A bubble packet is essentially an empty IPv6 packet. Since it is sent to the IPv4 address of the Teredo server, it will be encapsulated in an IPv4 packet.
  4. The Teredo server extracts the IPv6 bubble packet. From the bubble packet’s destination IPv6 address the Teredo server notes that it itself if the Teredo server for that client. This tells the server that its help is required for hole punching. It notes the IPv4 address of the relay from the source address of the packet (this is used in step 7).
  5. The Teredo server extracts the NAT IPv4 address and port from the host portion of the client’s Teredo address. It puts the bubble packet within a UDP message and sends it over IPv4 to the IPv4 address and port of the NAT box. The NAT box forwards this packet to the internal host (the Teredo client) as a mapping already exists for the Teredo server IPv4 address.
    • The UDP packet has to contain the Teredo server IPv4 address and port as the source address and port. It has to else the packet won’t pass through NAT. But the client also needs to know the IPv4 address of the Teredo relay. So the Teredo server sets an “Origin Indication” within this UDP packet that specifies the IPv4 address of the Teredo relay.
  6. The Teredo client receives the bubble packet in UDP. From the “Origin Indication” it knows the IPv4 address of the Teredo relay. From the bubble packet it knows the IPv6 address of the Teredo relay. And since it doesn’t know the Teredo relay’s IPv4 address and this packet came from its Teredo server – indicating that the client has to do it’s part of the hole punching – the client will now create a new IPv6 bubble packet, setting its Teredo IPv6 address as the source and the Teredo relay’s IPv6 address as the destination, put this within a UDP message, set the IPv4 address of the Teredo relay as the destination, and send it out.
  7. The packet passes through NAT and reaches the Teredo relay. Since this is a response to the bubble it previously sent, the Teredo relay knows the mapping is ready. Now the Teredo relay sends the previously queued incoming packet to the Teredo client and it gets through …!

Phew! Now we know why Teredo is a tunnel of last resort. There’s so much behind the scenes stuff that has to happen to keep it working. And that’s not to mention additional stuff like regular bubble packets from the Teredo client to server to keep the NAT mapping alive, checks to ensure there’s no spoofing done, and many more. Added to that, for security reasons an update to the Teredo RFC (Teredo RFC is RFC 4380, update is RFC 5991) specifies that Teredo should always assume it’s behind a Restricted NAT and so the above steps must always be done, even for clients behind Core NATs.

Back to NATs – if the NAT were Symmetric, Teredo does not work at all unless you make some changes on the NAT to exempt the Teredo clients. (Teredo in Windows Vista and above can work between Teredo clients if only one Teredo client is behind a Symmetric NAT and the other is behind a Cone/ Restricted NAT).

Identifying the type of NAT

Here’s how a Teredo client identifies the type of NAT it is behind.

Two things are in play here:

  1. The Teredo client IPv6 address has a flag that specifies whether it is behind a cone NAT or not. This flag is in the host bits of the address – remember I had previously mentioned the host bit has some flags, random bits, and the NAT IPv4 address and port number? One of these flags specifies whether the client is behind a cone NAT or not.
  2. The Teredo server has two public IPv4 addresses. The Teredo RFC does not expect these to be consecutive IPv4 addresses, but Windows Teredo clients expect these to be.

When the Teredo client contacts the Teredo server initially for an IPv6 address, it sends a Router Solicitation message as all IPv6 clients do (the difference being this is a unicast message, sent within a UDP message over IPv4 to a specific Teredo server address). The Router Solicitation message requires a link-local address – all Router Solicitation messages do – so the Teredo Client generates one, following the same format as a regular Teredo IPv6 address. The network prefix of this link-local address is set to fe80::/64, with the host bits having flags and random bits as usual but with the embedded IPv4 address being the private IPv4 address and port of the internal host rather than the public IPv4 address and port of the NAT box (because the Teredo client doesn’t know what this is).

This link-local address sets the cone flag is 1 – meaning the client believes it is behind a cone NAT.

When the Teredo server receives this it sends a Router Advertisement message as usual (as a UDP message within IPv4 unlike usual). The server does a trick here though. Instead of responding from the public IPv4 address on which it received the UDP message from the client, it responds from the second public IPv4 address. If this reply from a different IPv4 address gets through the NAT, then the client knows it is indeed behind a cone NAT. But if no replies come through (after an appropriate time-out (default: 4s) and number of retries (3 times)), the client realizes it may not be behind a cone NAT and so it sends a new Router Solicitation message to the Teredo server only this time it sets the cone flag to 0 (i.e. not behind a cone NAT).

Again the Teredo server receives the message and sends a Router Advertisement message, but now since the cone flag is 0 it sends the reply from the same IPv4 address as it received the message on. This will get through, confirming to the client that it is behind a non-cone NAT. (Note: If this reply too does not get through, after an appropriate time-out and number of retries the client realizes that UDP messages are blocked/ not getting through NAT/ firewall and so it sets the Teredo interface as disconnected/ off-line. Teredo cannot be used in this situation).

Next the client needs to know if it is behind a symmetric NAT. It now contacts the Teredo server on the second IPv4 address with a Router Solicitation message, setting the cone flag to 0 so the server uses the same IPv4 address when replying, and when it gets a reply from the Teredo server it compares the NAT port in the reply with the NAT port in the previous reply. If the ports are same the client determines that it is behind a restricted NAT; if the ports are different the client determines it is behind a symmetric NAT and that Teredo might not work.

(Note: I oversimplified a bit above to keep things easy. When the Teredo server sends a Router Advertisement, this includes the network prefix only. The host bits are set by the Teredo client once it identifies the type of NAT it is behind. The host bits require knowledge of the NAT IPv4 address and port, but how does the Teredo client know these? It knows these because the Router Advertisements from the Teredo server contains an “Origin Indication” field specifying the public IPv4 address and port. This is how the client gets the port number used for both Router Advertisements and determines if it is behind a symmetric NAT or not. Once that determination is done, the client has all the info required to self-assign a Teredo IPv6 address).

Notes on Teredo (part 1)

Previously I had talked about ISATAP. Today I want to blog about Teredo.

Teredo is another IPv6 transition mechanism. It is meant to be used as a transition strategy of last resort – i.e. only if other mechanisms such as ISATAP and 6to4 (which I haven’t blogged about yet) fail. This is because Teredo needs support from other servers on the Internet to do its work; and also because Teredo doesn’t encapsulate IPv6 packets within IPv4 directly, rather it puts them within UDP packets that are then carried by IPv4 packets. The latter means there’s extra overhead for using Teredo but it has the advantage that Teredo can work through NAT (with the exception of one type of NAT called Symmetric NAT) and so is more likely to work than ISATAP or 6to4.

Unlike ISATAP Teredo is meant for use over the Internet. And unlike 6to4 Teredo does not require a public IPv4 address. Teredo can work over the Internet from hosts with a private IPv4 address behind a NAT.

Before I go into the details of Teredo here’s a quick overview:

  1. If you have a host with private IPv4 address, you need some way of assigning it a global IPv6 address. But how do you do that? 6to4 takes the approach of creating an IPv6 address from the IPv4 address and that works because it requires public IPv4 addresses – which are unique in the first place, resulting in a unique IPv6 address. Teredo doesn’t have that luxury so it needs an IPv6 address generated through some other means.
  2. Here’s what Teredo does. It asks a server on the Internet (called a Teredo Server) for an IPv6 address. The server assigns it an IPv6 whose network prefix has first 32 bits as 2001:0000 and next 32 bits as the IPv4 address of the Teredo Server in hexadecimal. Thus all Teredo clients connecting to that server have the same network prefix.
    1. Say the Teredo Server IPv4 address is 17.23.52.1. You can use the in-built Windows calculator (in Programmer mode) to convert decimal to hex. 17 = 0x11, 23 = 0x17, 52 = 0x34, 1 = 0x1. So 17.23.53.1 in hex would be 1117:3401, resulting in a Teredo network prefix of 2001:0:1117:3401.
  3. The Teredo server also sets the host portion of the IPv6 address. This consists of some flags and random bits followed by the UDP port the client’s request came from along with the public IPv4 address of the NAT box the client is behind. Thus the host portion is also unique – the uniqueness being provided by the random bits as well as the UDP port of the client request, with some level of uniqueness also being provided by the public IPv4 address of the NAT box (though this is not unique among all clients within that same NAT).
  4. This way an IPv4 only host behind a NAT can get for itself a global unicast IPv6 address. The next question is how will it send and receive packets to the IPv6 Internet?
  5. For this Teredo clients need a Teredo Relay (this is usually a separate server, but one could have the Teredo server doubling as a Teredo relay too).
  6. A Teredo relay is a server set up by an ISP or organization that is happy to act as a “relay” between Teredo clients and IPv6 hosts. The relay advertises to the IPv6 Internet that it can route to the Teredo network prefix 2001:0:/32 (note that it advertises the entire Teredo network prefix, not just a specific network like 2001:0:1117:3401/64).
  7. So Teredo clients send IPv6 packets encapsulated in IPv4 to the IPv4 address of the Teredo relay. The relay passes it on to the IPv6 Internet as pure IPv6 packets with the source address set to the global unicast Teredo address of the client.
  8. The relay also receives packets to the Teredo prefix 2001:0:/32 from the IPv6 portion of the Internet and passes it on to the IPv4 clients. It knows which clients to pass these on to so because the host portion of the Teredo client address contains the IPv4 public IPv4 address of the NAT box and UDP port which will be forwarded to the private IPv4 address of the client. So all the relay needs to do is send IPv4 packets (containing IPv6 packets) to this public IPv4 address & UDP port.
    • It’s worth emphasizing here that the Teredo relay does not have an IPv6 routing table entry to the Teredo client. Rather, the packet is sent via IPv4. That’s why a relay is able to broadcast the entire 2001:0/32 and get packets for any Teredo client.
  9. If an IPv6 host on the Internet has both IPv6 and IPv4 addresses it can skip the Teredo relay altogether to send packets to the Teredo client. How? Because when this host receives a packet with source address as that of the Teredo client, it knows from the network prefix that this is a Teredo address, and it knows from the host bits how to contact this client via IPv4. So why go a roundabout way through Teredo relays and such, when it can directly send IPv4 packets (containing IPv6 packets) to the Teredo client? That’s precisely what the host does. This functionality is called Teredo Host Specific Relay – the host acts as its own relay. It does not matter if the IPv4 address of such a host is public or private. The IPv6 address must be a native global unicast address or a 6to4 address (obviously because that’s how it’s reachable on the IPv6 network initially).

This, in a nutshell, is how Teredo works. I think it’s a very cool piece of technology! It’s cool not only in terms of how it allows clients behind NATs to have a global unicast IPv6 address and be able to access the IPv6 Internet, but also in terms of some details I skipped over above like how clients find relays, and how relays punch hole in the firewall/ NAT behind which clients are to enable communication from clients to the IPv6 Internet. Fascinating!

Romeo & Juliet go hole punching!

Imagine Romeo and Juliet are under house arrest by their families. The families don’t allow any phone calls through either, unless it’s from a number Romeo or Juliet had just called out to. So while Romeo or Juliet can call anyone, a call from Romeo to Juliet will be blocked by her family as it’s not in response to a call she had just made. Similarly a call from Juliet to Romeo will be blocked by his family as it’s not in response to a call he made. The two lovers are thus unable to talk to each other, what are they to do?

Here’s one thing they could do. If Juliet were to call Romeo somehow and he were to call her back, then the call would get through. The problem is when Juliet calls Romeo her call is discarded by his family and so there’s no way for Romeo to know she has called him and if he were to call back now it will be let through. But suppose they had a common friend willing to help them, could that make a difference?

What Romeo and Juliet could do is call this mutual friend and not hang up. When Juliet wants to call Romeo she’ll inform the mutual friend who will pass this news on to Romeo. (A nice twist here is that Romeo and Juliet could use burner phones. When they call the mutual friend they’ll pass on their current phone number to the friend who will pass it on to each other. This way the families have no idea what Romeo or Juliet’s current phone numbers are and they can’t block calls to these numbers!). Juliet will then call Romeo’s number. The call will be discarded by Romeo’s family as it is not in response to a call he made, but Juliet’s family does not know that and as far as they are concerned Juliet has just called Romeo and no one answered. Since Romeo knows Juliet will be calling him and getting blocked, he can now call Juliet and his call will be let through as her family thinks its in response to her previous call. The two lovers can thus talk to each other! Happy days!

What I have described here is what’s known as hole punching in Computer networks. The difference being instead of Romeo, Juliet, and their families, you have computers behind firewalls or NAT (Network Address Translation) routers. The computers behind the firewall or NAT are unable to accept incoming packets from the outside world. They can only accept replies to packets they have sent out. To get two such computers talking to each other we need to “punch a hole” in the firewalls. That is, have a third computer somewhere on the Internet that will co-ordinate between them and help with the initial discovery and connection (if any of the computers are behind NATs it is akin to Romeo or Juliet using a burner phone). This third computer doesn’t have to do much except help with the initial phase. And worst case, if the hole punching does not work for any reason, the third computer can act as a “relay” passing packets from one computer to the other.

What is RPC?

In computer programming a “procedure” is a set of instructions packaged into one unit. For instance: say you regularly need to open a text file, scan it for certain text, and output a yes or no depending on whether there’s a match. One way is to keep writing these instructions at each place in your program. Another way would be to put these instructions at one place in your program, give that logical grouping a name, and then on every time you need to run these instructions simply invoke that grouping. The latter is an easier approach, not only because it saves your typing (or copy-pasting) the same code all over the place, but also because if you ever need to improve the instructions you just have to do it at one place. This logical grouping is what’s known as a procedure. Other names for a procedure are function, subroutine, and method.

When a procedure is invoked at a point in the code, usually with any inputs that are to be passed on to the procedure, the inputs are saved someplace and the procedure is loaded and executed. The main program is on hold while the procedure runs, its current state is stored at a location and execution moves to the procedure. The procedure reads the inputs from the saved place, does its deeds, stores any outputs in a designated place, and passes execution back to the main program (whose location was previously stored). The main program then reads the output and continues.

To give an analogy: think of it as though you are reading a book and you find a word whose meaning you do not know, so you keep down the book and open a dictionary to find the meaning. “Reading a book” is the main program; the “looking up the meaning” is a procedure that takes as input a word whose meaning you want, scans a dictionary, and gives as output an understanding of the word. The word itself is stored in your memory, and so is the the output (the meaning). That’s pretty much how procedures work.

Procedures can be part of the code of the main program or separate from it. They can even be from a 3rd party, packaged into files (such as DLLs) containing multiple procedures. When writing code the author links to these files and invokes procedures from them. When compiling the code to turn them into machine instructions the compiler puts in the required details on how to find the procedures (their address and such). That’s how the processor, which actually executes the program, knows where to find the procedure and what it expects.

When executing a program the processor only has access to its memory space. And since a procedure is linked to the program and has to be executed by the processor, it too must be in the same memory space as the main program. So procedures have to be local to the main program, i.e. on the same computer as the program. They can’t be on a remote computer. If a procedure is on a different machine (a different memory space) the processor has has no way of connecting to it.

As computers started to be connected over networks though, it because useful to have procedures that were remote to the main program. This way you could distribute the load across multiple computers, and maybe use powerful computers for some tasks and regular computers for others. Thus came about the concept of Remote Procedure Calls (RPC). RPC is a way of running procedures on remote computers by tricking the local processor into thinking the procedure is local to it.

Here’s how RPC works:

  1. Instead of a local procedure as usual, you have a stub procedure. This is a local procedure that takes input parameters meant for the remote procedure as though it were the remote procedure. As far as the main program is concerned the procedure is local.
    • To avoid confusion later on, we will refer to this stub as a client stub henceforth.
  2. The client stub procedure converts the input parameters into a form that can be understood by the remote computer. This is necessary as the parameters are in a location (memory space) that cannot be accessed by the remote computer, and also because the local and remote computers may use different ways of representing data (big-endian vs little-endian, for instance) and so the data may need conversion to a standard format. This process is called marshaling.
    • On Windows the client stub takes the input parameters (as in copies the values of the input parameters) and passes on to a Client Runtime Library (called rpcrt4.dll). This library does the marshaling.
    • This standard format is called a Network Data Representation (NDR) format. On Windows there are two marshaling “engines” that do this conversion. NDR20 is for 32-bit programs. NDR64 is for 64-bit programs (thought 64-bit programs can also use NDR20). Both marshaling engines are part of the Client Runtime Library.
  3. The client stub procedure uses the OS to contact the remote end and sends it messages. The OS in turns uses transport layer network protocols.
    • The client stub knows where the remote end is because it is either configured with the information or it looks up a central location to find the remote end. Usually the network address and port number(s) of the remote end.
      • For instance Windows domain joined computers can use Active Directory. In Windows 2000 domains, RPC services register under System\RPCServices.
      • This remote network address and port(s) is called an Endpoint.
    • The remote end usually has an RPC service listening on a designated port(s) and network address. This service allocates dynamic Endpoints for client stubs to connect to.
      • This service is called an Endpoint Mapper (EPM). In Windows you have service called the RPC Endpoint Mapper. It listens on port 135.
    • During this negotiation phase the EPM protocol messages contain items known as towers and floors. I am not too clear on what they are except that they are ways of representing RPC data and also for identifying the host address and port. When doing a network capture of RPC traffic these can be seen (the last link in the list below has examples of capturing RPC traffic).
    • On Windows the Client Runtime Library has three protocol engines – 1) a Connection RPC protocol engine (used when a connection-oriented protocol is required), 2) a Datagram RPC protocol engine (used when a connection-less protocol is required), and 3) a Local RPC protocol engine (used when the remote end is on the same host as the client). These protocol engines are what do the actual message passing.
    • Connection-oriented protocols used are TCP (dynamically assigned ports), SPX (Sequenced Packet Exchange), named pipes (port 445), or HTTP (ports 80, 443, 593). Connection-less protocols used are UDP (dynamically assigned ports) or CDP (Cluster Datagram Protocol).
  4. The remote end has a server stub that receives data from the local stub and unmarshals it. If required, the server stub converts the data to the remote machine specific format.
    • Server stubs register themselves with the Endpoint Mapper. Server stubs have a UUID (a well known GUID that identifies the application).
    • Once client stubs get an Endpoint from the Endpoint Mapper the client and server stubs talk to each other directly.
  5. The server stub then invokes the remote procedure with the data it has. As far as the remote procedure is concerned the main program is local to it. It is not aware that it is remote to the main program.
  6. Output from the remote procedure is passed on to the server stub which sends it to the client stub following a similar process as above (marshaling, unmarshaling).

The neat thing about RPC is that programmers can write distributed applications and don’t have to worry about the network details. In terms of the Internet Protocol suite, RPC sits in the Application layer.

Here are some good links on RPC:

Notes on setting up ISATAP

IPv6 and IPv4 nodes can’t talk to each other. These are two separate protocols, both of the Internet layer. So when some nodes in a network have moved on to IPv6 we need a transition mechanism to enable the remaining IPv4 nodes to talk to these IPv6 nodes. ISATAP is one such transition mechanism.

ISATAP stands for Intra-Site Automatic Tunnel Address Protocol. From the name itself we get an idea of what it is about:

  1. Intra-Site: It is meant for use within a site. That is in an Intranet, LAN, private network. Not for communication over the public Internet.
  2. Automatic: All configuration is automatic. Once the network is “aware” that ISATAP is in use, all hosts will auto-configure themselves (without the need for DHCP, static IPs, etc).
  3. Tunnel: Communication happens via tunneling of IPv6 packets within IPv4 packets. That is how IPv4 nodes are able to communicate with IPv6 nodes and vice versa. IPv4 packets have a field in their header called “Protocol”. This field tells the receiving end of the type of data in the packet. Usually the data is TCP, UDP, or ICMP (protocol numbers 6, 17, or 1) but other types too are possible (when and IPv4 packet contains an IPv6 packet the Protocol field is set to 41). This is an efficient way of carrying IPv6 packets within IPv4 because the only additional bits to the IPv6 packet are the IPv4 headers (20 bytes). Since the MTU of Ethernet is 1500 bytes a single IPv4 packet can carry up to 1480 bytes of an IPv6 packet.
  4. Address: ISATAP interfaces automatically configure themselves with an IPv6 address. The 64 bit network prefix could be a link-local, unique local, or global prefix. The 64 bit interface address is the IPv4 address (32 bits) prefixed with 0000:5EFE (for a private IPv4 address) or 0200:5E5E (for a public IPv4 address) (32 bits). The IPv4 address is written in the dot-quad notation (a.b.c.d) so it is easy to identify the IPv4 address from an ISATAP IPv6 address.
  5. Protocol: ISATAP is a protocol that defines all the above as well as the communication between the IPv4 and IPv6 nodes. ISATAP is defined in RFC 5214

ISATAP is not meant to be a permanent solution or for organization-wide deployment. It is only a transition strategy, for use in test labs or for parts of the Intranet while IPv6 addressing schemes and routing are being planned.

Windows Vista and above have ISATAP built into the OS. Separate ISATAP interfaces are created for each physical interface connected to a different DNS suffix. On Windows Vista (with Service Packs) and above, the interfaces are left in a “media disconnected” state unless the name isatap.<dnssuffix> can be resolved. If the name can be resolved then ISATAP IPv6 addresses are assigned to that virtual interface. On Windows Vista ISATAP IPv6 addresses are assigned anyways and the state is not “media disconnected”. In both cases the ISATAP IPv6 addresses have a link-local prefix but additional ISATAP addresses can be assigned via SLAAC (more details later). This way two hosts with ISATAP interfaces can talk to each other through IPv6, without any additional configuration, simply by using the link-local ISATAP addresses.

The isatap.<dnssuffix> name is supposed to point to an ISATAP router.

Native IPv6 hosts discover the IPv6 routers on their network via multicast messages (they send a Router Solicitation message to the multicast address fe02:2 which stands for link-local scope, all routers, and is subscribed to by all routers on that link). Routers that receive this message respond with a Router Advertisement message to the host. But… on an IPv4 network there’s no way for hosts to discover an IPv6 router – unless they use IPv4 multicast as some transition technologies tend to do, something that may not always be possible – so what we need is for a way to specify the router address to hosts directly. This is what the isatap.<dnssuffix> address is supposed to be. It is an ISATAP router that will help hosts with auto-configuring as well as routing to other IPv6 networks. Hosts contact this router by sending Router Solicitation messages to this unicast address, and the router will respond.

The isatap.<dnssuffix> name is only one way of specifying the ISATAP router address. (The isatap.<dnssuffix> name can be resolved using DNS, the hosts file, or NetBIOS). Other ways are to specify the router address are directly via PowerShell, netsh, or GPOs.

Turning on ISATAP is a very easy process. Consider the network below:

IPv6-ISATAP

WIN-DC01, WIN-DC02, and WIN-GW01 are IPv6 enabled machines Windows Server machines. They have both IPv4 and IPv6 addresses, but the IPv4 addresses don’t matter as I want clients to connect to these servers via IPv6. WIN-CLIENT03 and WIN-CLIENT12 are Windows 8 and Windows 7 machines. They are in a separate IPv4 only network, connecting to the main network through a router. The fact that they are in a separate network doesn’t matter. I just put them in a separate network to illustrate something later.

I want WIN-CLIENT03 and WIN-CLIENT12 to connect to WIN-DC01 and WIN-DC02 through IPv6. Currently they can connect via IPv4 but soon the servers will be IPv6 only and I want to ensure everything works well over IPv6. As this is a temporary situation I will use ISATAP to ensure the clients can connect to the servers until the permanent solution is ready. Here’s what I have to do:

  1. Decide a machine that will act as the ISATAP router (any Windows Server would do; you can use Linux too). The ISATAP router will have a virtual ISATAP interface created on it. If the idea is to have ISATAP hosts connect to an IPv6 network then the router must have an IPv6 interface too on which it will send and receive IPv6 packets. On the other hand if hosts won’t be connecting to any IPv6 network and all you need is automatic configuring then the IPv6 interface is not needed.

    I chose WIN-GW01 as my ISATAP router. Note that it’s in a separate network from the clients but that doesn’t matter.

  2. Name this machine isatap.<dnssuffix> (or assign it as a CNAME via DNS or as as additional name via the hosts file). Alternatively set this machine as an ISATAP router for itself – via PowerShell, netsh, or GPOs.

    I used netsh:

    This enables ISATAP on WIN-GW01. It is still not ready as an ISATAP router, that will be done in the next step.

  3. To configure WIN-GW01 as an ISATAP router I have to enable advertising on the ISATAP virtual interface. Optionally, if I want the clients connecting to it to have a global or unique local prefix, I can configure one.

    fdcc:7c4e:3651::/48 is a unique local prefix. 2002 is a subnet I assigned for ISATAP hosts.

    WIN-GW01 is now ready as an ISATAP router. That’s it!

    Here’s the output of ifconfig for the ISATAP interface:

    Notice it has a link-local address – which would be the only address if I hadn’t specified a unique-local prefix – and also a unique-local address. Both have the IPv4 address embedded in it.

  4. Optionally: Since WIN-GW01 has an IPv6 and ISATAP interface, I can enable routing between them and also advertise WIN-GW01 as the default router for the ISATAP hosts connecting to this.

    On WIN-GW01, I do the following:

    The first command enables routing on the 10.50.0.0/24 interface. The second command enables routing on the ISATAP interface. Lastly, I advertise to hosts on the ISATAP network that WIN-GW01 is the default router:

  5. All that’s left to be done now is to enable ISATAP on the clients. One can use DNS, GPOs, PowerShell, or netsh. I went with netsh. The command is same as that on the ISATAP router:

    Here’s the output of ifconfig for the ISATAP interface on one of the clients:

    It has picked up a unique-local ISATAP address, auto-configured a link-local ISATAP address, and is correctly pointing to the ISATAP interface of WIN-GW01 as the default router. The unique-local address would have also registered with DNS (this is why I created unique-local addresses).

  6. And that’s it. Let’s test IPv6 connectivity via ping:

    Here’s the IPv6 routing table on the client:

The neat thing about such tunneling mechanisms is that the underlying IPv4 link is treated as a local link. Even though the clients and servers are in different networks, that does not matter for IPv6. As far as IPv6 is concerned they are on the same link and it is up to the underlying IPv4 protocols to route packets between the IPv4 networks.

Also notice from the routing table above how the fdcc:7c4e:3651:2002::/64 network has an on-link route rather than via the ISATAP router. This is because communication between ISATAP hosts happen directly, without router involvement. The hosts know each others IPv4 address from destination IPv6 address, so the ISATAP interfaces talk to each other over IPv4. The IPv4 address is still required because unlike native IPv6 which discovers the MAC address of an IPv6 address through Neighbour Discovery, ISATAP has no such luxury. It must discover the MAC address through other means – i.e. use the IPv4 address and leave it to the IPv4 protocols to do the routing.

And that’s all! Pretty straightforward stuff.

Update: To disable ISATAP if you set the router via netsh or PowerShell do either of the following:

-or

CentOS – initial configuration notes

Nothing new here, just stuff I can refer to later when I forget. I’ve usually worked with Debian but am not working with CentOS for a change so some things are new to me.

  1. If you install CentOS in a machine with less than 1024 MB RAM you don’t get the graphical installation option. So best to install with 1024 MB RAM and then downgrade RAM if you want to. I have a CentOS VM with less than 512 MB RAM. I don’t necessarily need a graphical installation but I prefer it as it’s friendly. I get to choose the hostname for instance, select the packages I want to install, and so on. Without a graphical install you are given a minimal install only with no options to choose.
  2. Set the hostname in /etc/sysconfig/networking.
  3. Network interfaces are managed by NetworkManager, which is useful if you have a GUI or like to use the nmcli command. I prefer old fashioned ifconfig so I manage my network devices via /etc/sysconfig/network-scripts/ifcfg-* files.
    • Set NM_CONTROLLED=no in this file to tell NetworkManager not to bother with this interface.
    • Set ONBOOT=yes to up this interface on boot up.
    • Use IPADDR=a.b.c.d, NETMASK=x.y.z.f, GATEWAY=p.q.r.s to set the IPv4 address, netmask, and gateway. Use BOOTPROTO=dhcp if you’d prefer DHCP instead.
    • Using IPV6INIT=no and IPV6_AUTOCONF=no is supposed to turn off IPv6 for that interface but it doesn’t seem to. A better way to turn off IPv6 (and/ or control its parameters) is via sysctl.
    • For more options this link from Oracle is a useful reference, as is the initscripts-ipv6 homepage.
  4. To disable IPv6 on an interface the following sysctl setting helps: net.ipv6.conf.eth0.disable_ipv6 = 1. Add entries like these to /etc/sysctl.conf.
  5. The following sysctl setting tells the OS to act as a router (i.e. forward packets): net.ipv4.ip_forward = 1
  6. Use /etc/resolv.conf to specify name servers, domain suffix (via the domain keyword), and list of search suffixes (via the search keyword with a space separated list of domains). Domain suffix is the domain name to be added to hosts without a domain name. Search suffixes are additional domain names that can be added to hosts when searching. Use only one of these. If both are specified the last one wins.
  7. Its best to put the Intranet DNS servers first in /etc/resolv.conf followed by Internet DNS servers. In my setup the Intranet servers don’t have access to the Internet and so any Internet name queries to them timeout. When this happens the DNS resolver will automatically try the next server in the list … up to when it reaches the Internet servers and get an answer. To speed the process up use option timeout:1 to set a timeout of 1 second. In contrast if I put the Internet servers first the don’t timeout – they try to resolve the non-existent domain and reply that it does not exist – so the Intranet servers aren’t queried.

Resolving names using Link-local Multicast Name Resolution (LLMNR)

A neat thing about IPv6 is that IPv6 nodes can be up and running and talking to each other without much infrastructure support.

For instance, IPv6 nodes can self configure their address via SLAAC. If a network prefix is published, well and good, hosts can use that; but if not hosts will have link-local addresses (fe80::/10). How will these stateless addresses talk to each other though? As in, say the name WIN-CLIENT02 has the link-local address fe80::e070:559a:8794:8c01, how will another client WIN-CLIENT01 know thus? These are stateless addresses, not published in DNS, so how will name resolution take place?

Similarly, even if the address were manually configured – a Unique-local or Global unicast address, for instance, set by the node user – how will name resolution take place? Organizations might have DNS to help with this, but home users can’t be expected to have DNS. With IPv4, one could hope to remember the IPv4 address, but IPv6 addresses are not human-friendly so some form of help is required.

This is where Link-Local Multicast Name Resolution (LLMNR) comes into the picture. It is a way of name resolution that works for the link-local scope only and uses multicast.

What is LLMNR?

All nodes on a link subscribe to a multicast address called ff02:1:3. From my previous post you’ll know what this means – ff00::/8 is the prefix for multicast addresses; 2 indicates the address has a link-local scope; and 1:3 is the group ID for LLMNR hosts. So the way things work is that when WIN-CLIENT01 wants to resolve the name of WIN-CLIENT02, it sends an LLMNR query (basically a UDP packet similar to a DNS query; remember multicast only works with UDP) to port 5355 of the ff02:1:3 multicast address. Since WIN-CLIENT02 is subscribed to this group it receives this query and responds with its addresses.

Here’s a Wireshark capture when one of my clients is trying to ping WIN-CLIENT02 and hence look up its name. First it queries DNS (step 5350), doesn’t get an answer (step 5353), so it sends a multicast query to ff02:1:3 asking for the IP address (step 5354). Note that the multicast query is sent to the IPv4 multicast address too (224.0.0.252) for the IPv6 AAAA record – this is because I had asked ping to only use the IPv6 address.

llmnr-reply

(The Wireshark output only shows LLMNR and DNS messages as I am filtering for traffic to ports 5355 and 53).

WIN-CLIENT02 replies (step 5357) and its reply has all its IPv6 addresses.

llmnr-reply2

Suppose I hadn’t insisted ping use the IPv6 address, the above queries would be for the A record instead (including the one to the IPv6 multicast address). And when no replies come from WIN-CLIENT02, NetBIOS over TCP/IP (NetBT) will be used to try and find the IPv4 address.

llmnr-no-answer

While playing with this I realized that by default Windows clients do not respond to LLMNR queries. This is because Network Discovery is turned off – even for Domain connected interfaces. So if you too find that LLMNR queries are being sent but not responded to, check the Network Discovery setting on your machine (go to “Network and Sharing Center” in the Control Panel > “Change Advances Sharing Settings” > and turn on Network Discovery for whichever network profile you are connected to) .

You can also enable Network Discovery via GPO for domain joined computers. The setting is at Computer Policy\Policies\Administrative Templates\Network\Link-Layer Topology Discovery – there are two settings there, the second one is what you want.

Multicast MAC address

On an unrelated note, just like Unicast IPs have a corresponding MAC address, so do Multicast IPs. They have a special MAC address that start with 33:33 and the remaining four octets (group of 8 bits or group of 2 Hex digits each) are the last four octets of the Multicast IP address. So for a multicast IPv6 address ff02::1:3 – which can be expanded as ff02::0001:0003 – who last four octets are 00, 01, 00, and 03, the MAC address would be 33:33:00:01:00:03. So all hosts subscribed to this multicast address will create a special MAC address on their interfaces so packets sent to this MAC are passed on to the upper layers for processing. That’s how multicast works behind the scenes.

On Windows machines, you can use the arp command to view IPv4 address to MAC address mappings. But arp doesn’t work for IPv6 as there’s no ARP protocol in IPv6. Instead we have the Neighbor Discovery Protocol (NDP) and to see the mappings discovered through that we have to use netsh. Here’s the full command:

In the output you can see many other MAC addresses starting with 33:33 for the various IPv6 multicast groups the node is automatically a member of.

mac-33

Enabling IPv6 Router Advertisements on Windows

From my previous post you know that IPv6 hosts can autoconfigure themselves if they get the network prefix from a router (or server running this service). Windows Server (and Clients) can send Router Advertisement messages without any additional software. This functionality is in-built and can be configure via the netsh command.

Here’s what I did on the Server Core 2012 which functions as the “router” for my test lab.

  1. Visit https://www.sixxs.net/tools/grh/ula/ and get a ULA prefix for myself. This is a /48 block – meaning I have to fill in the remaining 16 bits of subnet ID to make this a /64 block. I can have 2^16 subnets. I got the fdcc:7c4e:3651::/48 prefix, I’ll just use subnet 1 for now, so my network prefix will be fdcc:7c4e:3651:1::/64.
  2. I assigned an IPv6 address “fdcc:7c4e:3651:1::254”, netmask 64, to the server.
  3. Next, I issued the following command on the server to enable router advertisements:

    This enables Router Advertisements. But this doesn’t advertise any prefixes.

    Below is a Wireshark capture on that interface from a client:

    RA1

    Notice that Router Advertisement messages are being sent. The messages specify two options – MTU and the link-layer address of the router.

    Here is the result of ipconfig for that interface on the client:

    ipconfig1

    The only IPv6 address is the link local address. No gateway is set either.

  4. To specify prefixes the publish, I issued the following command on the server:

    In case that command gives an error – maybe you don’t have that route entry already – replace it with the following:

    This tells the server to publish this prefix on the Router Advertisement messages on that interface. Without publish=yes the command tells the server that the fdcc:7c4e:3651:1:::/64 network is on the “Local Area Connection” interface for routing purposes – that the prefix for devices on this interface is (or will be) fdcc:7c4e:3651:1:::/64. The publish=yes bit tells the server to publish this prefix information in Router Advertisement messages.

    Below is a Wireshark capture once prefix publishing is enabled:

    RA2

    Notice the prefix information is now published.

    And now ipconfig too shows the automatically generated addresses:
    ipconfig2

  5. So far the server isn’t functioning as a router (i.e. it is not forwarding packets). If we want the server to function as a router that can be enabled:

  6. Once the server functions as a router we can also tell it to include this information in the Routing Advertisement messages. This way clients can automatically pick up the router as a default gateway!

    Checking the Wireshark capture will now show a new option in the Router Advertisement messages:

    RA3

    And ipconfig will show a default gateway is automatically set:

    ipconfig3

    Ain’t that cool!

  7. To view the current configuration of that interface on the server, the following command can be used:

  8. The neat thing with Router Advertisement messages is that they can work in conjunction with DHCPv6. The Router Advertisement messages can tell clients to also contact DHCP for an IPv6 address and additional options, or contact DHCP not for an IPv6 address but only for additional options.

    For the former do this:

    For the latter do this:

    Here’s the Wireshark output after I enabled managed address. The output is similar to the previous ones except for the flags (it’s 0x80 now in contrast to 0x0 before). So I have expanded it.

    RA4

    The way clients will behave now is thus:

    1. If the client’s interface is set to Managed Address as disabled (i.e it is not looking for DHCPv6 address configuration), since the Router Advertisement now sets it to enabled it will start looking for DHCPv6 address configuration.
    2. If the client’s interface is set to Managed Address as enabled, since the Router Advertisement too sets it as enabled it will behave as before.

    In the opposite scenario – suppose Router Advertisement messages were setting Managed Address as disabled (which is the default) clients ignore this and continue working based on what their own Managed Address configuration is.

If you don’t want Windows clients to listen for Router Advertisements, do the following on the client:

As with the server, to view the interface configuration on the client the following works:

Lastly, suppose you have enabled prefix publishing on the server, and Wireshark shows the information is being sent but clients aren’t picking it up, the following might be helpful. By default the site prefix is set to /64 but if your network prefix is (say) /48 either by mistake or intentionally, clients will ignore this network prefix. Correct the prefix setting then, or the network prefix mask if it’s a typo. The first time I played with this I had forgotten to change the mask from /48 to /64 once I added the subnet ID bits, so the server was advertising it with /48 and clients were ignoring it.

Notes on IPv6

I spent the last two weeks studying IPv6. It began because I was experimenting with DirectAccess on my virtual lab and that uses IPv6, which I was sort of familiar with from my FreeBSD days but hadn’t explored in a while (and it’s been a while). What follows is my notes from the various blog posts and articles I read. It is not meant to be an explanatory post, mainly something for me to refer back to later.

  • IPv6 has 128 bit addresses vs 32 bit addresses of IPv4
  • The “v6” and “v4” stand for version 6 and version 4 respectively. Before I began reading about IPv6 I though the 4 and 6 stood for the number of dot or colon separated groups (which is all the more silly because IPv6 addresses have 8 groups and not 6!)
  • IPv4 puts these 32 bits as 4 groups of 8 bits (a byte). 8 bits gives you 256 combinations (2^8) so that’s why each group is a number from 0-255.
  • IPv6 puts these 128 bits in 8 groups of 16 bits each. That is, twice the number of groups as IPv4 and twice the bits in each group. Remember that!
    • 16 bits in each group means you have 65536 (2^16) combinations. So each group can have a number from 0-65535.
    • To make the number in each group smaller and easier to handle, one can increase the number of digits used. That is, instead of restricting to digits 0-9 we also add a-f, giving us a total of 16 digits to use (‘a’ is short for 10, ‘b’ short for 11, and so on). This way of representing numbers is called Hexadecimal and while the range was from 0-65535 with 10 digits (decimal), now it is from 0-FFFF with 16 digits (hexadecimal).
    • To avoid confusion hex digits can be prefixed with a 0x to indicate their nature. So 0xa is the hex digit a, not the alphabet a.
  • In an IPv4 unicast address (an address which specifies a single device on the network) the address has a variable host part and network part. But in the case of an IPv6 unicast address half the bits (64) are for the host and half the bits (64) for the network.

StateLess Address AutoConfiguration (SLAAC)

The advantages behind IPv6 are not only that it has more addresses, but also that it has features not present in IPv4. One of these is stateless address autoconfiguration (SLAAC).

  • Devices on the network use the Neighbour Discovery Protocol (NDP) to get details of the network (the network prefix, the MTU, duplicate addresses, redirects, etc). (See also this page).
    • IPv6 routers periodically send Router Advertisement messages. These are also sent in reply to Router Solicitation messages sent by devices on the network.
      • It is not not necessary to have routers for the Router Advertisement messages. Servers too can send this. Linux has the Router Advertisement Daemon. Windows has this in-built and it can be enabled & configured via netsh.
    • Devices also send each other Neighbour Solicitation messages and reply with Neighbour Advertisement messages. This way they can ensure the neighbour device is still reachable.
  • One of the items (optional) in the Router Advertisement is the network prefix. This is a globally unique 64-bit value. Consists of a 48-bit globally unique prefix followed by a 16-bit subnet ID that is unique within the network.
  • Devices can autoconfigure their IPv6 address using the network prefix and their MAC address and/or a random number.
  • This method is called Stateless because no one keeps track of the addresses. Routers don’t know the device IPv6 addresses, unlike a DHCP server. All routers do is to provide the network prefix, rest is up to the devices.
  • For more on SLAAC, including the process and advantages disadvantages see this article. SLAAC has its disadvantages (devices may use the SLAAC address instead of their DHCP or static address and you may not want that).
  • SLAAC can be disabled on the device. Or the router can be configured to not send the prefix information.
    • On Windows the netsh command can do both of this.

Writing the Address

While IPv4 addresses use a period to separate the groups, IPv6 uses colons. Since IPv6 addresses are very long, there are a few tricks to shorten them:

  • One or more leading zeroes in a group can be omitted. So 0017 can be written as 17.
  • Just once, one or more consecutive groups of zeroes – viz in 0000:0000:1234 – can be replaced with a double colon like this – ::1234. This can only be done once and the recommendation is to not do it for just one group of zeroes.
  • To give an example, say we are given an address such as 2001:12cd::1. To work out what this stands for we start with remembering that there must be 8 groups. There’s only 3 here, and since then double colons indicate groups of zeroes, in this case they indicate 5 groups of zeroes. Also the final group of 1 is actually 0001.
    • This is why the double colon can only be used once. If there were two double colons there would be no way of know how the hidden 5 groups of zeroes are to be distributed.

Addresses/ Prefixes

  • Unspecified address: :: (all 128 bits are 0)
    • Used as the source address, but never as destination address, when the interface is yet to get an address.
  • Default route: ::/0 (equivalent to 0.0.0.0./0)
  • Loopback address: ::1/128 (equivalent to 127.0.0.1/8)
    • Unlike IPv4 only ::1 can be used as the loopback address. Cannot use ::2 etc.
  • Global unicast addresses: 2000::/3 prefix
    • 64 bits of network prefix followed by 64 bits of host address.
    • Network prefix is 48 bits of Organization prefix followed by 16 bits of subnet ID.
      • First three bits are 001. How? 2000::/3 prefix means only the first 3 bits are set. So even though 2000 looks like a 16 bit hex number, only the first 3 bits are used, rest 13 don’t matter. 2 = 0010, so the first three bits are 001.
        • The first group is then written as (001x)(xxxx)(xxxx)(xxxx). (001x) can be 0010 or 0011 – i.e. 2 or 3. So the actual range of the first group is 2000::/16 to 3ffff::/16.
      • Next 45 bits are the global routing prefix. These are the assigned by whoever allocates you IPv6 addresses (usually an ISP).
    • Subnet ID is unique within the organization.
      • However, you don’t always get this – it is decided by the address allocator.
      • The allocator need not always go with a /48 (3+45 bits) block address. They could use the subnet ID bits too and assign a /64 block – leaving you with no subnets!
      • If a /64 block organization prefix is assigned, it means you have no bits for subnets. If a /60 block is assigned you have some bits (4 bits for subnet – so 2^4 = 16 subnets). And so on … This is what the /xx subnet mask of your network prefix means for you – how many subnets you can have.
        • /64 block => 0 subnets
        • /60 block => 16 subnets
        • /56 block => 256 subnets
        • /52 block => 4096 subnets
        • /48 block => 65536 subnets
    • The 64 bits of host address is manually specified, automatically generated via SLAAC or assigned by DHCPv6.
    • See this page for more.
  • Link-local addresses: fe80::/10 prefix; the actual assignment is from fe80::/64 so easier to remember it as /64. (equivalent to 169.254.0.0/16)
    • First 10 bits are FE8 (0xF is 1111, 0xE is 1110, 8 is 1000). Rest 54 bits are 0 – i.e a single subnet of all zero bits.
    • The 64 bits of host address are automatically assigned (randomly or by expanding the MAC address via EUI-64). These may also be assigned manually.
    • There can be multiple link-local addresses. In Windows when you disable and enable a network interface a new link-local address is assigned (in addition to the manually assigned addresses, if any).
    • Unlike IPv4, IPv6 requires a link-local address to be assigned to every network interface on which the IPv6 protocol is enabled, even when one or more routable addresses are also assigned. So IPv6 devices have more than one IPv6 address assigned to them.
    • On Windows link-local addresses are created by default for ISATAP too. The host bit is created by prefixing 0200:5efe (for public IPs) or 0000:5efe (for private IPs) in front of the 32 bits of IPv4 address.
    • Packets with source address as a link-local IP cannot cross the router. All IPv6 networks use the same fe80::/64 prefix.
    • These addresses are not to be published in DNS as they are unique only for the link. Not unique even within the organization.
    • Suggestion: Set fe80::1 as the router address for each interface. This way you don’t have to remember the longer address. Also easy to troubleshoot.
  • Unique local addresses (ULA): fc00::/7 (equivalent to 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16)
    • Similar to link-local addresses but routers will allow them to cross subnets within the site (no clear definition of what constitutes a site; it is up to your border router to block the fc00::/7 packets from crossing out – you define your site!).
    • Unlike link-local addresses, these are globally unique addresses (not guaranteed, but there is a high probability it will be globally unique if you followed the correct steps).
    • Since the mask is /7, only the first 7 bits are used. 0xF = 1111, 0xC = 1100, 0x0 = 0. So the first 7 bits are 1111110 – basically the 0xFC bits. The zeroes in fc00 are not really zeroes. They are only put as zeroes to complete the group and they don’t matter for the network prefix (because of the mask).
    • Notice though that while the first 7 bits are 1111 – 0xF – and 110 – 0xC – the latter can be written as 0xC only if the eight bit is 0. This is because 0xC is 1100, but since the last bit doesn’t come under the network mask it is not fixed and can be 1 too. In which case 0xC becomes 0xD.
      • This is intentional. This way the fc00::/7 prefix can be thought of as as two prefixes – fc00::/8 and fd00::/8.
      • This 8th bit is called an L flag. L == Local. The idea being that in future there could be a central authority that might assign globally unique ULAs. To avoid any clashes with such a scenario, addresses assigned by that authority will have the L flag set as 0 (not local) while addresses generated by non-central now will have the L flag set as 1 (local). Thus –
        • fd00::/8 == Unique Local locally assigned Addresses
        • fc00::/8 == Unique Local centrally assigned Addresses
      • All ULAs generated by users are supposed to be in the fd00::/8 prefix.
    • So that’s the first 8 bits. You still have 64 – 8 = 56 bits for the network prefix. These are made thus:
      • 40 bits of a random number. Generated by you. High probability of uniqueness. Suggested algorithm here.
      • 16 bits for subnets.
        • That’s 65536 subnets so an organization mostly only needs one 40 bit random number for all its networks. Everything can be a subnet of that.
    • ULAs can be published in DNS. As they work across the site.
    • See this page for more. And this. Also see RFC 4193 on how to generate, and the rationale behind ULAs.
    • Visit https://www.sixxs.net/tools/grh/ula/ to generate ULAs following the RFC 4193 method and also register the address with the website. If everyone were to do this, the website can ensure ULAs are unique.
  • Stateless IP/ICMP Transition address: ::ffff:0:0:0/96
    • If a device has IPv4 address a.b.c.d then its IPv6 address can be assigned as ::ffff:0:a.b.c.d.
    • The IPv6 network bit takes 96 bits. Balance is 32 bits for the host – which is the IPv4 address.
      • The network block is all zeroes (hence the ::).
      • The host bit has 16 bits of ones (ffff) followed by 16 bits of zeroes (0), followed by 32 bits of the IPv4 address.
    • This prefix was chosen to yield a zero-valued checksum to avoid changes to the transport protocol header checksum (RFC 2765).
  • Anycast address: Any IPv6 Unicast address (just like in IPv4).
    • One-to-closest-of-many
    • Any type of address valid for Unicast (Link-Local, Site-Local, ULA or Global) also is valid for Anycast.
    • A sending node does not do anything special when sending traffic to an Anycast destination. The network will route your traffic to the nearest one of the nodes (in the network metric sense) that has had that anycast address assigned to it.
    • An Anycast connection is still one-to-one, just like Unicast. It just is done to the closest node that has the Anycast address assigned to it. All three Transport Layer protocols (UDP, TCP and SCTP) work with Anycast.
  • Multicast address: ff00::/8 prefix (i.e. 8 ones and 8 zeros, but the 8 zeros don’t matter as the mask is /8. They are zeros just to complete the group)
    • One-to-many
    • Only for UDP and SCTP (TCP is too complex to do multicast; sender has to keep track of all the recipients, whether they received etc).
    • A multicast address contains a group.
    • Multicast address format is thus:
      • 8 bits of 1 (the “0xff”) – this is the mask so these are the fixed bits
      • 8 bits of flags & scope:
        • 4 bits of flags
        • 4 bits of scope – 0x0 to 0xf – so the last digit in the first group of the IPv6 address tells you the scope
          • Scope 1 = Interface-local
          • Scope 2 = Link-local (the subnet)
          • Scope 4 = Admin-local (smallest scope that can be administratively managed … whatever that means)
          • Scope 8 = Organization-local (the collection of sites managed by the organization, linked by VPN or whatever)
          • All the above scopes are LOCAL. The next one is global.
          • Scope E = Global (any node on the Internet, not filtered by routers).
      • 112 bits of group ID
      • There are many well known groups apart from the user defined ones. Examples:
        • Group 1 = Node
        • Group 2 = Router
        • Group 5 = OSPF IGP router
        • Group 1:2 = DHCPv6 servers/ relay agents
        • Group 1:3 = DHCPv6 servers or LLMNR hosts (depends on the scope)
        • Remember multicasts are only for UDP …
    • Every IPv6 node (that is not forwarding) is a part of the following groups:
      • ff01::1 – All nodes in local interface
        • Scope is 1 (interface-local), Group is 1 (node)
      • ff02::1 – All nodes in local link
        • Scope is 2 (link-local/ the subnet), Group is 1 (node)
    • Windows nodes are part of the following groups:
      • ff02::c – Simple Service Discovery Protocol
        • Scope is 2 (link-local/ the subnet), Group is c
      • ff02::1:3 – Link-Local Multicast Name Resolution
        • Scope is 2 (link-local/ the subnet), Group is 1:3 (LLMNR hosts)
    • Every IPv6 router is a part of the following groups:
      • ff02::5 – All nodes in local site
        • Scope is 2 (link-local/ the subnet), Group is 5 (IGP routers)
    • Solicited node multicast address: created for each unicast and anycast address
      • ff02::1:ff___, with last 24 bits equal to last 24 bits of the unicast/ anycast
      • Scope is 2 (link-local/ the subnet)
      • Used for link-layer address resolution via NDP.
      • From here:
        • The solicited-node address facilitates efficient querying of network nodes during address resolution. In IPv4, the ARP Request frame is sent to the MAC-level broadcast, disturbing all nodes on the network segment, including those that are not running IPv4. IPv6 uses the Neighbor Solicitation message to perform address resolution. However, instead of using the local-link scope all-nodes address as the Neighbor Solicitation message destination, which would disturb all IPv6 nodes on the local link, the solicited-node multicast address is used. The solicited-node multicast address consists of the prefix FF02::1:FF00:0/104 and the last 24-bits of the IPv6 address that is being resolved.
        • For example, for the node with the link-local IPv6 address of FE80::2AA:FF:FE28:9C5A, the corresponding solicited-node address is FF02::1:FF28:9C5A. To resolve the FE80::2AA:FF:FE28:9C5A address to its link layer address, a node sends a Neighbor Solicitation message to the solicited-node address of FF02::1:FF28:9C5A. The node that is using the address of FE80::2AA:FF:FE28:9C5A is listening for multicast traffic at the solicited-node address and, for interfaces that correspond to a physical network adapter, has registered the corresponding multicast address with the network adapter.
        • The result of using the solicited-node multicast address is that address resolution, which commonly occurs on a link, is not required to use a mechanism that disturbs all network nodes. In fact, very few nodes are disturbed during address resolution. In practice, because of the relationship between the Ethernet MAC address, the IPv6 interface ID, and the solicited-node address, the solicited-node address acts as a pseudo-unicast address for very efficient address resolution.
    • More info on this page.
    • Some well-known multicast groups:
      • Remember! Last digit of first group is the scope (link, organization, site, global, etc). Last digit of the last group is the group (nodes, routers, etc).
      • ff02::1 – Scope 2 (link-local), Group 1 (nodes)
      • ff05::1 – Scope 5 (organization-local), Group 1 (nodes)
      • ff02::2 – Scope 2 (link-local), Group 2 (routers)
      • ff05::2 – Scope 2 (organization-local), Group 2 (routers)
      • ff02::fb – Scope 2 (link-local), Group fb (DNS servers)
      • ff05::fb – Scope 5 (organization-local), Group fb (DNS servers)
      • ff02::1:3 – Scope 2 (link-local), Group 1:3 (DHCPv6 servers)
      • ff05::1:3 – Scope 5 (organization-local), Group 1:5 (DHCPv6 servers)
  • Teredo: 2001:0::/32 prefix.
  • 6to4: 2002::/16 prefix.
  • ISATAP: fe80::/64 (ISATAP addresses are link-local)
    • Host bit is 0200:5efe (for public IPs) or 0000:5efe (for private IPs) followed by 32 bits of IPv4 address

Scopes

  • Unicast has 1) Link-local, 2) Global, 3) Unique-local, and 4) Interface-local (loopback).
  • Multicast has 1) Link-local, 2) Global, 3) Interface-local, 4) Organization-local, 5) Site-local, and 6) Admin-local.
    • Note: No Unique-local.
    • The interface-local scope spans a single interface only. A multicast address of interface-local scope is useful only for loopback delivery of multicasts within a node, for example, as a form of interprocess communication within a computer. Unlike the unicast loopback address, interface-local multicast addresses can be joined on any interface (from this page, has good info on the other scopes too).
  • More here.

This is just the tip of the ice berg, of course!

Clients picking up the wrong DFSN target, dfsutil shows AccessStatus 0xc000003a

We use DFS at work and I noticed one of our laptops was always connecting to a target in a far away site. My own desktop was connecting to the wrong target but after a reboot it connected to the correct one. There weren’t any user complaints about slowness, so could be that other machines were ok and/ or they weren’t noticing the slowness.

DFS seemed to be set up correctly, and the laptop was picking up an IP in the correct site etc so it wasn’t any obvious misconfiguration.

To investigate further I used the dfsutil tool. I got the idea for this from an old KB article. The dfsutil tool is a part of the DFS Management tools and is present in %WINDIR%\System32. It can be used to administer DFS-N and is more powerful than the GUI. Since the laptop didn’t have this tool I copied it from my desktop (which had the DFS Management tools installed) to the laptop.

First I checked if the laptop is able to reach the correct DC or DFSN server. I do this by examining the domain cache on the client.

In the past the command used to be dfsutil /SpcInfo but now it’s the more intuitive dfsutil cache domain. The older switch still works though. (The domain cache is also called SPC cache, hence the switch is SpcInfo).

In the output the entries with a * are those which were obtained through the Workstation service. Like the domain name, the NETBIOS domain name, and the DC which the laptop is connected to.

All other entries (both – and +) are obtained through referrals by the DFSN client. Of these, entries with a + are currently used by the client. So in the case above, the laptop had got a referral to the rakhesh.local domain and was correctly latching on to the WIN-DC01 server which is the DC and DFSN server for its site.

Next I examined the referral cache of the client to see if there are any errors that point out to why it is not using the correct referral link.

DFSN stores information on the root name, the root servers, links, and target servers in a Partition Knowledge Table (PKT). This is stored in AD (in case of a domain based DFS) or the Registry (in case of a standalone DFS). Clients refer to this and cache it with then (so they don’t have to look it up always) and this is what we need to examine. Previously the command for this used to be dfsutil /PktInfo but now it’s replaced by the more intuitive dfsutil cache referral (notice how everything comes under the dfsutil cache sub-command).

And therein lies the problem! The laptop is correctly connecting to the WIN-DC01 DFSN server, this server is correctly returning the namespace (“pub”) and link (“downloads”) and referrals to the two targets for this link (on win-data01 and win-data02). But whereas we want the laptop to use win-data01, that is not set to active and has an error status returned. Instead win-data02 is set as the active target.

There doesn’t seem to be a definite cause behind the “0xc000003a” error code. What helped me finally was this forum post. I didn’t try the solution that helped the OP (though our office environment has a similar setup) as one of the suggested solutions did the trick for me.

It appears that if the link targets have different offline caching settings, it considers the targets with offline caching enabled first and only if that fails does it consider the preferred target. So I checked my targets to see if they have different caching settings and sure enough the preferred target didn’t have caching. Turned on caching for it, rebooted the laptop, and now it picks up the correct target!

An easy way to check whether all your targets have similar ACLs and offline caching settings is to us the dfsdiag command on the DC/ DFSN server. I ran the command thus:

More about the dfsdiag command in this TechNet post.

Hopefully this helps someone.