MBONE: Multicasting Tomorrow's Internet
To answer that question, you first have to understand what the MBONE is. The MBONE is a research project. All research is funded by entities like governments, universities, and private corporations, and the MBONE is no exception. This research includes the protocols that are used on the MBONE, the tools and applications that have been developed for the MBONE, and everything in the global MBONE virtual network (multicasting, encapsulation, and so on).
The funding also comes from the MBONE end users in one form or another. Users (which may consist of an entire site full of computer operators) pay an Internet provider for access to a number of services, which may include an MBONE tunnel or multicast routing. This Internet provider, in turn, pays a network provider for the use of its physical links. These physical links most often belong to telecommunications companies such as Sprint or MCI. A very small number of Internet providers own their physical links, but in the end, they still have to pay to have access to other sites on physical links that belong to the telecommunications companies.
Telecommunications companies seem to be the big winners in this scenario, but these companies have to spend a fortune to install all the cabling that goes from one end of the country to the other end. They certainly make a profit by owning this equipment, but because the Internet is constantly expanding, they also have to keep upgrading these links, and the maintenance it takes to perform this task is very expensive. (So no one really wins, except perhaps the governments who tax a packet of data traveling the networks several times before it reaches its destination. Such is the case in Canada, because the government taxes these services there.)
So, if you are paying for the Internet and MBONE, you may want to know how much your users access the MBONE so that you can charge the costs back to them and become part of this virtual food chain.
There are two ways that your users can conveniently use the MBONE:
People in these other entities use the feed that you provide to participate in all the sessions that they want. However, the number of sessions in which they are able to participate depends heavily on the bandwidth constraints that you impose on the feed. For example, in a tunnel line in the mrouted.conf file, the maximum bandwidth that a link is restricted to, by default, is 512 Kbps (see the rate column in Table 11-1). This bandwidth can be changed with the rate_limit option on the tunnel line, so you can use that option to implement various levels of service.
How do you know what sessions users participated in or how much data they sent? A partial answer can be obtained by using UNIX utilities. UNIX comes with a great deal of networking utilities, and some of them are useful for getting networking statistics on multicast traffic. First, take a look at netstat.
netstat is a utility that can provide you with unicast statistics. It can show you the routing tables on your machine, for example, or it can show you the usage of your network memory buffers. On some platforms ( FreeBSD is one of them), netstat can give you multicast statistics as well. The netstat -g command (as shown in Table 11-1) gives you all kind of useful information.
Virtual Interface Table Vif Thresh Rate Local-Address Remote-Address Pkts-In Pkts-Out 0 1 0 123.456.78.90 0 0 1 16 512 123.456.78.90 123.456.210.1 12406 112515353 2 8 512 123.456.78.90 123.456.3.4 5596 44642609 3 8 512 123.456.78.90 123.456.141.16 6 56865205 4 8 512 123.456.78.90 123.456.186.10 0 9664355 5 16 512 123.456.78.90 123.457.90.90 0 0 6 64 512 123.456.78.90 130.130.130.130 117148123 183497 7 8 512 123.456.78.90 123.456.101.182 0 0 8 32 512 123.456.78.90 124.222.4.3 149934 15749310 9 32 512 123.456.78.90 129.111.3.18 23821 5056246 Multicast Forwarding Cache Origin Group Packets In-Vif Out-Vifs:Ttls 130.240.3.81 224.2.195.52 39771 6 1:16 4:8 130.240.3.76 224.2.213.97 32600 6 1:16 3:8 4:8 192.153.117.35 224.2.252.231 357 6 1:16 4:8 128.138.213.1 224.2.0.1 75657 6 1:16 4:8 198.93.139.41 224.2.252.231 50856 6 1:16 4:8 128.55.192.227 224.2.143.24 56682 6 1:16 3:8 4:8 128.232.0.49 224.0.1.1 7747 6 1:16 3:8 4:8 9:32 146.88.1.106 224.2.245.77 716 6 1:16 4:8 128.40.64.5 224.2.244.129 72840 6 1:16 4:8 130.240.12.79 224.2.187.164 8381 6 1:16 4:8 130.199.130.175 224.2.0.1 11 6 1:16 4:8 192.16.123.88 224.2.213.97 98957 6 1:16 3:8 4:8 128.100.197.178 224.0.1.1 1 1 4:8 6:64 171.69.58.88 224.2.231.173 45701 6 1:16 3:8 4:8 131.188.34.89 224.2.0.1 81533 6 1:16 4:8 171.69.58.92 224.2.143.24 14998 6 1:16 3:8 4:8 204.188.121.18 224.2.127.255 11 6 1:16 4:8 128.3.112.2 224.2.0.1 63309 6 1:16 4:8 129.89.9.110 224.2.187.164 14420 6 1:16 2:8 3:8 4:8 9:32 129.240.200.196 224.2.236.244 80710 6 1:16 4:8 128.4.1.1 224.0.1.1 5513 6 1:16 3:8 4:8 204.62.246.98 224.2.248.110 131170 6 1:16 4:8 193.60.11.44 224.2.143.24 191421 6 1:16 3:8 4:8 128.84.247.156 224.2.0.1 58315 6 1:16 3:8 4:8 128.3.112.2 224.2.127.255 15 6 1:16 4:8 204.62.246.73 224.2.143.24 6814956 6 1:16 3:8 4:8 131.225.220.147 224.2.0.1 98359 6 1:16 3:8 4:8 131.225.220.147 224.2.231.173 299682 6 1:16 3:8 4:8 130.199.130.175 224.2.1.1 10651 6 1:16 4:8 130.240.3.81 224.2.219.172 66366 6 1:16 3:8 4:8 130.240.3.81 224.2.177.132 16363 6 1:16 4:8 199.104.80.5 224.2.248.110 1199 6 1:16 4:8 128.232.0.56 224.0.1.1 8709 6 1:16 3:8 4:8 9:32 18.77.1.243 224.2.251.237 197837 6 1:16 4:8 192.6.28.26 224.2.0.1 26537 6 1:16 4:8 129.242.6.80 224.2.189.16 73074 6 1:16 3:8 4:8 171.69.60.189 224.2.231.173 30666 6 1:16 3:8 4:8 13.2.116.196 224.2.127.255 153 6 1:16 3:8 4:8 8:32 9:32 130.199.130.175 224.2.187.164 4 6 1:16 4:8 130.240.3.38 224.2.213.97 32596 6 1:16 3:8 4:8 163.221.196.12 224.2.0.1 98133 6 1:16 4:8 171.68.225.139 224.2.0.1 120 6 1:16 4:8 150.83.68.41 224.2.143.24 75111 6 1:16 4:8 130.240.3.81 224.2.236.244 39975 6 1:16 3:8 4:8 171.69.129.220 224.2.0.1 14937 6 1:16 3:8 4:8 130.240.3.81 224.2.189.16 25754 6 1:16 4:8 171.69.60.152 224.2.231.173 45353 6 1:16 3:8 4:8 156.40.113.11 224.2.143.24 210 6 1:16 2:8 3:8 4:8 8:32 9:32 171.69.129.220 224.2.231.173 45548 6 1:16 3:8 4:8 129.242.6.80 224.2.177.132 37541 6 1:16 3:8 4:8 171.69.58.81 224.2.231.173 45643 6 1:16 3:8 4:8 171.69.58.81 224.2.0.1 15031 6 1:16 3:8 4:8 204.62.246.68 224.2.127.255 3787 6 1:16 4:8 198.93.139.41 224.2.187.164 50838 6 1:16 4:8 192.80.13.56 224.2.0.1 1880 6 1:16 3:84:8 128.182.61.159 224.2.0.1 138180 6 1:16 4:8 204.188.121.18 224.2.187.164 7050 6 1:16 4:8 128.4.1.24 224.0.1.1 21169 6 1:16 3:8 4:8 192.58.206.100 224.2.0.1 112335 6 1:16 3:8 4:8 204.62.246.76 224.2.245.77 1142781 6 1:16 4:8 204.160.73.10 224.2.231.173 2 6 1:16 4:8 9:32In Table 11-1, the first part of the output titled ìVirtual Interface Table,î shows multicast traffic statistics for each virtual interface on your multicast router, where a virtual interface can be a physical interface or a tunnel. You can use this output to find out how much traffic was received and sent by each of your down feeds. The Virtual Interface (VIF) 0 in Table 11-1 is the physical interface of the machine; because it doesn't send any multicast traffic to the local network, the number of packets that go in and out on it are zero.
The number of packets in for the VIF 6 is much higher than for the other VIFs, because VIF 6 is the feed that I use to receive all MBONE traffic from the rest of the world. You can also see that the number of packets out to VIF 6 is pretty low, at approximately 183 K-packets. What this means is that not very much traffic has been sent out of Canada; rather, the biggest part of the traffic has been sent within the Montreal area.
You will notice that for VIF 1, the number of packets out is almost as high as the number of packets in for VIF 6. The reason behind this is that the other end of this tunnel feeds a number of sites, of which one is an mrouted version 2.2 that doesn't support pruning (see Chapter 10 for a discussion on the importance of pruning).
As you can see, netstat can give you a fairly accurate picture of the MBONE usage of each of your feeds. There are drawbacks to using this method to check your feed stats though:
The fourth drawback is important here because it means that you cannot monitor how MBONE is used on an individual basis. For example, suppose that you are willing to let your customers use the MBONE for free except when they use it for events that are not related to work (such as musical events, radio, world news, and so on). netstat does not provide you with the level of granularity needed to discern between work-related uses and personal uses. You could use the second part of the netstat output shown in Table 11-1 for this purpose (the part titled ìMulticast Forwarding Cacheî), but this method also has major drawbacks:
Netstat is also an inadequate tool to use if you receive a single MBONE feed that will be used by all your internal users. Imagine a corporation placing the MBONE that is available to its employees in various departments. The problem with such a setup is that although you will be able to monitor the amount of data flowing in or out of your site, you will not be able to determine the individual sessions in which the traffic was sent or received. Again, this restriction presents a problem if you want to charge for MBONE usage.
The tool that comes to the rescue in these situations is called mlisten. The mlisten tool is part of a package that also includes mprocess.
mlisten listens to the control port of a specific multicast address and records the IP addresses of all the participants for that session. This tool uses a parameter file containing a few settings that influence the way that mlisten collects data.
mprocess takes the output produced by mlisten and creates a new output file in another form. This new output file contains four columns of summary information about the session that mlisten monitored. These four columns contain the following information:
Another tool that may be of use to you is traceroute (and its multicast equivalent, mtrace). You can't use this utility for doing accounting tasks per se, but it will help you establish policies at your site regarding who gets a feed from you and who doesn't, or who you get a feed from. This task is more important than you might think, because if you set up your network in a chaotic way, multiple MBONE feeds may end up crossing your network provider's physical links, costing you money. traceroute shows the route that a packet takes from your host to a destination host that you specified (see Figure 11-1). This trace will be useful to you if, for example, you are a network provider and a site requests an MBONE feed from you. By doing a traceroute to the site, you can determine whether you would be the best person to provide them with a feed.
Figure 11-1 and 11-2 show two examples of traceroute in action. Suppose that both sites in the examples requested an MBONE link from me, an Internet service provider. traceroute clearly shows me that I could give the first site a feed if they want one, but giving a feed to the second site would be out of the question.
traceroute to mix.net (198.168.73.2), 30 hops max, 40 byte packets 1 Therouter.CC.McGill.CA (123.456.78.9) 1.566 ms 1.367 ms 1.359 ms 2 McGill-gate.Provider.Net (123.456.79.1) 2.343 ms 1.747 ms 1.700 ms 3 192.77.58.10 (192.77.58.10) 1.601 ms 1.443 ms 1.507 ms 4 MIX.NET (198.168.73.2) 38.857 ms 43.820 ms 39.798 msFigure 11-1: The traceroute output to the first site.
traceroute to cam.org (198.168.100.5), 30 hops max, 40 byte packets 1 Therouter.CC.McGill.CA (123.456.78.9) 1.742 ms 1.456 ms 1.694 ms 2 McGill-gate.RISQ.Net (123.456.79.1) 1.628 ms 2.258 ms 2.140 ms 3 psp.qc.canet.ca (192.68.56.5) 57.032 ms 19.398 ms 5.093 ms 4 psp.ny.canet.ca (205.207.238.154) 8.557 ms 11.782 ms 11.525 ms 5 border4-hssi1-0.Boston.mci.net (204.70.23.5) 13.494 ms 14.662 ms 20.854 ms 6 core-fddi-1.Boston.mci.net (204.70.3.33) 12.987 ms 13.102 ms 12.707 ms 7 core-hssi-2.NewYork.mci.net (204.70.1.1) 21.413 ms 17.191 ms 16.772 ms 8 border2-fddi-0.NewYork.mci.net (204.70.3.18) 16.663 ms 15.848 ms 16.102 ms 9 sprint-nap.NewYork.mci.net (204.70.45.6) 19.688 ms 22.025 ms 21.864 ms 10 fd-0.enss218.t3.ans.net (192.157.69.4) 24.534 ms 23.664 ms 20.710 ms 11 t3-3.cnss32.New-York.t3.ans.net (140.222.32.4) 26.026 ms 25.305 ms 24.435 ms 12 cnss40.New-York.t3.ans.net (204.149.4.8) 27.959 ms 28.373 ms 29.114 ms 13 t3.New-York-Mtl.t3.ans.net (198.168.57.25) 38.749 ms 37.864 ms 36.716 ms 14 Hydro.CAM.ORG (198.168.73.132) 53.998 ms 40.718 ms 44.047 ms 15 * Ocean.CAM.ORG (198.168.100.5) 64.561 ms *Figure 11-2: A traceroute output for the second site.
The second site in Figure 11-2 should probably request a link either from ANS or MCI.
The multicast equivalent of traceroute is mtrace. It will show you the multicast route that a packet will use to go from point A to point B. This information not only enables you to determine whether giving a feed to a requesting party is appropriate for you, but also enables you to direct them to a more appropriate site for requesting a feed if you are unable to provide the feed.
Figure 11-3 shows an example of an mtrace output. I had to get some help from friends for this example because mtrace has to be supported by multicast routers for it to work. Because no mtrace path exists in Canada that is longer than 2 hops, and 2 hops is inadequate for providing a good example, here is an mtrace from MFSDatanet.com to uunet.net .
Mtrace from 137.39.246.98 to 150.225.14.20 via group 224.2.0.1 Querying full reverse path... * switching to hop-by-hop: 0 tacostand.MFSDatanet.COM (150.225.14.20) -1 comet.MFSDatanet.COM (150.225.14.11) DVMRP thresh^ 1 -2 wdcmon.MFSDatanet.COM (150.225.20.36) DVMRP thresh^ 1 -3 * mae-bone.psi.net (192.41.177.247) DVMRP thresh^ 64 -4 * MBONE1.UU.NET (137.39.43.34) DVMRP thresh^ 64 -5 * MBONE2.UU.NET (137.39.246.98) DVMRP thresh^ 64 -6 MBONE2.UU.NET (137.39.246.98) Round trip time 156 ms Waiting to accumulate statistics... Results after 10 seconds: Source Response Dest Packet Statistics For Only For Traffic 137.39.246.98 224.0.1.32 All Multicast Traffic From 137.39.246.98 v __/ rtt 152 ms Lost/Sent = Pct Rate To 224.2.0.1 137.39.246.98 MBONE2.UU.NET v ^ ttl 64 -1/62 = -1% 6 pps 0/0 = --% 0 pps 137.39.43.34 MBONE1.UU.NET v ^ ttl 65 -1/381 = 0% 38 pps 0/0 = --% 0 pps 192.41.177.247 mae-bone.psi.net v ^ ttl 66 2/469 = 0% 46 pps 0/0 = --% 0 pps 192.41.177.25 150.225.20.36 wdcmon.MFSDatanet.COM v ^ ttl 67 0/21 = 0% 2 pps 0/0 = --% 0 pps 150.225.14.11 comet.MFSDatanet.COM v \__ ttl 68 2 0 pps 0 0 pps 150.225.14.20 150.225.14.20 Receiver Query SourceFigure 11-3: An example of an mtrace output.
The first part of the mtrace output shows the multicast route used by the packets to get from the source host to the destination. Here we can see the thresholds for each hop on the path; this information is useful for determining the TTL value that you should use when you want to create an MBONE event and make sure that it reaches every location to which you want it to go.
The second part provides statistics about the path itself. This data includes the percentage of packet loss incurred along the routing path. You can use this information to locate the source of the problem when you are receiving traffic from an MBONE event and experiencing a high percentage of data loss.
I said earlier that all the multicast routers between the source host and the destination host have to support mtrace for mtrace to work. You can verify if a particular router supports mtrace by using the mrinfo utility. Figure 11-4 shows an example of an mrinfo output.
127.0.0.1 (localhost.CC.McGill.CA) [version 3.8,prune,genid,mtrace]: 123.456.78.90 -> 0.0.0.0 (local) [1/1/querier/leaf] 123.456.78.90 -> 123.456.210.1 (MBONE.Collaboration.CA) [1/16/tunnel/leaf] 123.456.78.90 -> 123.456.3.4 (deadbeef.Algo.McGill.CA) [1/8/tunnel/leaf] 123.456.78.90 -> 123.456.141.16 (LoveNotes.Harmo.McGill.CA) [1/8/tunnel/leaf] 123.456.78.90 -> 123.456.186.10 (crash.WaysToFly.mcgill.ca) [1/8/tunnel/leaf] 123.456.78.90 -> 123.457.90.90 (MBONE.UOtherSideCity.CA) [1/16/tunnel/down/leaf] 123.456.78.90 -> 130.130.130.130 (mbonehost.jvnc.net) [1/64/tunnel/leaf] 123.456.78.90 -> 123.456.101.182 (123.456.101.182) [1/8/tunnel/down/leaf] 123.456.78.90 -> 124.222.4.3 (MBONE.cs.FarEast.ca) [1/32/tunnel/leaf] 123.456.78.90 -> 129.111.3.18 (cythera.GOE.ca) [1/32/tunnel]Figure 11-4: An example of an mrinfo output.
To generate the mrinfo output in Figure 11-4, I simply issued the mrinfo command without arguments; this gave me the mrinfo for my localhost. Issuing the mrinfo command with the name of the multicast router as an argument would give you the mrinfo for that router. In Figure 11-4, the first line shows that my localhost supports mtrace and also supports pruning. mrinfo also shows which tunnel is down, and it (the remote host) is a leaf in the tree for mtrace. The output also shows every link with their metric and threshold. Doing an mrinfo on each of these nodes would tell me whether or not the thresholds are symmetric; if they aren't, I could then contact the remote site and have them change the thresholds.
These various tools can help you perform accounting tasks on your MBONE usage, as well as assist you in debugging your MBONE setup. These tools comprise a limited toolkit, however, and some people may say that the tools are not very user friendly, either. If you are willing to put in some supplementary programming effort, however, you'll find these tools to be very flexible in how you can use them and what they can provide for you.
The future will undoubtedly provide us with even more of these tools for working with the MBONE. And keep in mind that the MBONE is still a research project; as such, the structure of the MBONE itself will continue to change, and so too will the tools for managing it.