Understanding NVIDIA InfiniBand Networking: Routing, Switching, and Its Benefits for AI Infrastructure and High-Performance Computing (HPC)

What is InfiniBand Networking?

InfiniBand is a high-performance networking technology used in data centers, supercomputers, and AI environments to transfer large amounts of data quickly and efficiently. It is known for offering high bandwidth and low latency—key factors that make it ideal for environments requiring fast data movement, like AI training, machine learning, and high-performance computing (HPC).

In simple terms, InfiniBand is a super-fast "data highway" that allows computers to talk to each other almost instantly. It’s like upgrading from regular roads to a high-speed bullet train for your data!

Advantages of InfiniBand Over Traditional Networking

1. High Bandwidth: InfiniBand can handle up to 800 Gbps, allowing it to move large datasets quickly.
2. Low Latency: It delivers messages between nodes with very little delay (under a microsecond), making it ideal for real-time computing.
3. Scalability: InfiniBand can easily scale up to connect thousands of nodes, making it suitable for the largest supercomputing environments.
4. RDMA (Remote Direct Memory Access): This feature allows direct memory access between computers without involving the CPU, speeding up data transfers.

InfiniBand Use Cases in AI and HPC

Now that we know what InfiniBand is, let’s look at how it’s used in AI and High-Performance Computing (HPC):

1. AI Model Training

When training AI models, especially deep learning models, vast amounts of data must be processed and shared across multiple GPUs or servers. InfiniBand helps by making this data transfer fast and efficient.

2. Real-Time AI Inference

AI inference is when trained models are used to make predictions. For certain applications, like self-driving cars or robotic surgery, you need these predictions in real-time. InfiniBand helps achieve this by ensuring that the data moves quickly between the model and the decision-making system, reducing any lag.

3. High-Performance Computing (HPC)

HPC systems are used in fields like weather forecasting, scientific simulations, and drug discovery. These systems rely on thousands of computers working together to perform complex calculations. InfiniBand’s low-latency and high-bandwidth features make it ideal for these environments because the faster computers can exchange information, the quicker they can finish their calculations.

Advantages of InfiniBand Over Traditional Networking

• Speed: InfiniBand can deliver much faster data rates compared to Ethernet, making it ideal for tasks where every second counts.
• Efficiency: InfiniBand’s RDMA allows data to bypass the CPU, reducing load on computing resources and freeing them up for other tasks.
• Scalability: InfiniBand can easily scale to support the growing needs of data centers, HPC, and AI environments.

Routing and Switching in InfiniBand Networks

InfiniBand (IB) is a high-performance networking technology often used in environments such as high-performance computing (HPC), data centers, and AI/ML workloads. InfiniBand’s key attributes—low latency, high throughput, and scalability—make it ideal for large-scale computing tasks. The way routing and switching happen in an InfiniBand network is crucial to achieving these benefits.

Below is an explanation of how routing and switching occur in an InfiniBand network:

1. InfiniBand Network Components

Before diving into routing and switching, it's important to understand the basic components of an InfiniBand network:

• InfiniBand Nodes: These are devices such as servers, storage, and GPUs connected to the network.
• Switches: InfiniBand switches connect the nodes within a subnet. These switches forward traffic between nodes based on the destination's Local Identifier (LID).
• Router: InfiniBand routing is required between subnets, where Global Route Headers (GRH) and Global Identifiers (GIDs) are used to navigate the network. Static and adaptive routing
• Gateways:
• Subnet Manager (SM): The SM is responsible for initializing, configuring, and managing the InfiniBand network. It assigns addresses, computes paths, and monitors the network.
• InfiniBand Subnets: An InfiniBand network can be divided into multiple subnets, each managed by a subnet manager.

2. Switching in InfiniBand

InfiniBand switches operate at Layer 2 (Data Link Layer) and work by forwarding packets between nodes within a subnet. Switching is based on the Local Identifier (LID), which is a unique address assigned by the subnet manager to each node in the network.

How Switching Works:

• LID-Based Forwarding: Each node in an InfiniBand subnet is assigned a unique Local Identifier (LID) by the subnet manager during the network initialization process. When a node sends a message, it includes the destination node’s LID in the packet header. InfiniBand switches examine the LID in the packet and use their internal LID routing tables to forward the packet to the appropriate output port that leads to the destination node.

3. Routing in InfiniBand

While InfiniBand switches primarily operate within a subnet, routing is required when data needs to be sent between different subnets. InfiniBand uses LID routing within subnets and Global Route Headers (GRH) for communication between subnets.

Routing Mechanisms:

• LID Routing (Within a Subnet): Within a subnet, InfiniBand uses source-based routing. Paths are pre-computed by the subnet manager based on the topology. The switches in the subnet use the LID assigned to the destination node to forward the packet along the best path.
• Global Routing (Between Subnets): For communication between subnets, InfiniBand uses a Global Route Header (GRH). The GRH contains the Global Identifier (GID) of the destination, which includes both the destination subnet and node address. When a packet needs to traverse different subnets, routers are used. These routers inspect the GRH to determine the final destination's subnet and route the packet accordingly.

4. Subnet Manager (SM) and Path Computation

The Subnet Manager (SM) plays a critical role in the routing and switching process:

• Path Discovery: The SM discovers the network topology, assigns Local Identifiers (LIDs), and computes the optimal paths between nodes. This ensures efficient routing and minimal hops.
• Routing Table Setup: After computing the paths, the SM programs the routing tables of all the switches, mapping each LID to its corresponding output port.
• Network Monitoring: The SM continuously monitors the network, ensuring the routes remain optimal, and recalculates paths in case of failures or topology changes.

InfiniBand Routing Algorithms and their use cases:

1. Static Routing

• Overview: Static routing is the simplest form of routing in InfiniBand networks. It relies on pre-defined, fixed paths that are calculated and configured by the Subnet Manager (SM) when the network is initialized. Once these paths are set, they remain constant unless the network topology changes.
• How it works: The Subnet Manager discovers the network topology and assigns routes between all nodes in the subnet based on their Local Identifiers (LIDs). The paths are programmed into the switches’ forwarding tables, and these paths are used until a topology change or failure occurs.
• Advantages: Simple to implement and manage. Predictable paths with no dynamic recalculations.
• Disadvantages: Can lead to suboptimal performance in networks with fluctuating traffic patterns since it cannot adapt to changes in network load.
• Use case: Suitable for small to medium-sized networks where traffic patterns are predictable and do not change frequently.

2. Up/Down Routing Algorithm

• Overview: The Up/Down routing algorithm is designed to prevent loops and deadlocks in the network by restricting the direction that traffic can take. It builds a spanning tree of the network, assigning "up" and "down" directions to links between switches.
• How it works: The network is organized into a logical tree. When a packet is sent, it must first "move up" the tree toward higher-level switches and then "move down" toward the destination. This restriction prevents cycles and ensures deadlock-free routing.
• Advantages: Simple and loop-free by design. Ensures deadlock avoidance by controlling traffic flow.
• Disadvantages: Can lead to inefficient routing since some paths may be longer than necessary due to the up/down restriction. Does not account for network congestion, which may result in unbalanced traffic distribution.
• Use case: Effective in hierarchical or tree-like network topologies where the network can be easily divided into levels.

3. Adaptive Routing

• Overview: Adaptive routing is a dynamic routing algorithm that makes real-time decisions on the best path for each data packet based on current network conditions, such as congestion or link failures. It actively monitors network traffic and dynamically adjusts the path to avoid bottlenecks.
• How it works: Switches in the InfiniBand fabric continuously monitor the network load on their links. When a packet arrives at a switch, the switch selects the least congested link toward the destination. This allows the network to balance the load dynamically.
• Advantages: Improved performance in high-traffic networks by avoiding congested links. Better load balancing as traffic can be distributed more evenly across available paths.
• Disadvantages: More complex than static routing, requiring constant monitoring of traffic. Can introduce instability if route changes are made too frequently.
• Use case: Ideal for large-scale, high-performance environments like AI workloads and HPC, where traffic patterns are highly variable, and congestion can occur.

Adaptive routing is typically the most advanced and performance-oriented solution, especially for dynamic and large-scale environments like AI and HPC, while up/down or static routing may be more suitable for simpler, hierarchical networks.

By: Altaf Ahmad