The ATOMIC LAN is a 640-Mbps packet-switched LAN designed by ISI and CalTech. The ATOMIC project succeeded in implementing a LAN prototype, including host interfaces and switches. ATOMIC-2 is investigating how to use the ATOMIC LAN as a production network.
This topology is designed so that each user machine has at least two independent paths to the gateway (center, light blue). The wiring also maximizes the number of connected machines and minimizes the hopcount.
- Light blue: “gateway” to ATM
- Magenta: singly-connected Lab switch (can be isolated/dashed line)
- Red: doubly-connected host switches (“tie-links”)
- Blue: triply-connected host switches (“backbone”)
- Green: user and lab hosts (6*6 + 4*5 = 56 user hosts, 7 lab hosts)
ATOMIC was developed from Caltech’s MOSAIC mesh-connected supercomputer technology. Myrinet is a commercial version of ATOMIC is being produced by our friends at Myricom. ATOMIC-2 is a member of the ARC and SCAN ATM testbeds.
Going the last meter
Before the ATOMIC LAN, the bottleneck was in the LAN. Host interfaces, file servers, and gateways could support a significant portion of the available LAN bandwidth (line widths are proportional to BW).
The presumed solution is to install a faster LAN, e.g., Fast-Ethernet, FDDI, ATM, or the ATOMIC LAN (again, line widths are proportional to BW):
The problem is that LAN bandwidth gets you only so far. Interfaces have limitations, due to driver software and hardware design. OS interactions further limit available bandwidth at the application layer.
ATOMIC-2 is aimed at addressing these remaining limitations. In telephony, it is well-recognized that the majority of costs and complexity lie in the local loop, the last mile to the customer’s phone. In the ATOMIC-2 project, we are addressing the equivalent networking concept…
“Going the last meter”
ATOMIC-ATM gateway and router design
High-speed file server
Integrated services (QoS)
ATOMIC2 has been extended to support high-level integrated services, which are important for real-time protocols (e.g., RTP) and QoS reservation mechanisms (e.g., RSVP). This required determining its “implict” switch packet scheduling that uses round-robin arbitration on a per-packet level and extending it to support bandwidth per source related to ratio of packet sizes. This emulation allowed the system to emulate router queueing disciplines by inserting interval gaps in packet emission, where the available BW is determined by gaps in other traffic and we reserve BW by regulating gaps in all traffic. It also involves modifying packet sizes via MTU discovery and leveraging centralized intra-LAN BW reservation.
ATOMIC link-layer multicast
Myricom only supports link-layer serial broadcast – there is no (switch-level) hardware multicast support. The source unicasts a separate copy of the multicast message to every node in the LAN, which leads to high multicast latency. Multicast messages are filtered by IP on each host, which wastes SBus bandwidth.
We developed an alternate approach for tree-based multicast involving other receivers to help distribute messages. It saves SBus bandwidth by only allowing the copy of multicast message reach a destination node in the multicast group.
This research addresses how to efficiently implement multicast services in cut-through switching networks, in the absence of hardware multicast support at intermediate switches. Minimum-time multicast algorithms are presented for direct networks including general n-dimensional meshes and hypercubes and indirect networks including multistage networks supporting turnaround routing. The results of implementations on a 64-node IBM SP-1 show that the proposed algorithm significantly outperforms the application-level broadcast primitives provided by currently existing collective communication libraries including the public domain MPI.
- `Unicast-Based Multicast Communication in Wormhole-Routed Networks”, (P. K. McKinley, H. Xu, A. H. Esfahanian, and L. M. Ni), IEEE Transactions on Parallel and Distributed Systems , vol. 5, no. 12, pp. 1252-1265, Dec., 1994.
- “Optimal Software Multicast in Wormhole-Routed Multistage Networks”, (H. Xu, Y. D. Gui, and L. M. Ni), Proceedings of Supercomputing’94 , pp. 703–712, Nov., 1994.
- “A Scalable Multicast Service in 2D Mesh Networks”, (H. Xu, P. K. McKinley, and L. M. Ni), Frontiers’92: The 4th Symposium on the Frontiers of Massively Parallel Computation , pp.156–163, Oct. 1992.
In the Gigabit Networking Workshop ’95 at Infocom ’95.Abstract
Electronic slides (HTML format)
Slides in compressed PostScript
“Improving PVM Performance Using the ATOMIC User-Level Protocol”
H. Xu and T. Fisher
In the High-Speed Network Computing Workshop ’95 at IPPS ’95.Workshop paper in compressed PostScript
“Performance Analysis of MD5”
In Proc. Sigcomm ’95, Boston, pp. 77-86.Abstract (available soon)
Conference paper in compressed PostScript
Slides in compressed PostScript
“Report and Discussion on the IEEE ComSoc TCGN Gigabit Networking Workshop 1995”
J. Sterbenz, H. Schulzrinne, J. Touch
In IEEE Network, July 1995, pp. 9-21.Journal paper in HTML format
“ATOMIC-2: Going the Last Meter for Gigabit LANs”
J. Touch, T. Faber, A. DeSchon, A. Sachdev
In the Gigabit Networking Workshop ’96 at Infocom ’96.Abstract
“Experience with a Production Gigabit LAN”
J. Touch, T. Faber, D. Jani
In the Gigabit Networking Workshop ’97 at Infocom ’97.Abstract
“Optimizing Throughput in a Workstation-Based Network File System over a High Bandwidth Local Area Network”
In Operating Systems Review vol. 32, no. 1, pp. 29-40, (January 1998)Postscript
“Dynamic Host Routing for Production Use of Developmental Networks”
J. Touch and T. Faber
In Proceedings of the IEEE International Conference on Network Protocols, Atlanta, Ga, October 28 – 31, 1997HTML
“Avoiding the TCP TIME_WAIT state at Busy Servers”
T. Faber, J. Touch, and W. Yue
“High-performance IP Forwarding Using Host Interface Peering”
Joe Touch, Anne Hutton, Simon Walton
IP Authentication Header kernel patches
Patches to test the performance of various algorithms, and parts of AH processing (header insertion/deletion, data touching, etc.)
See the README for information on the patch itself.
ATOMIC-2 is supported by the Defense Advanced Research Projects Agency (DARPA):