Data Centre Networks

Data centre networks use spine-leaf fabrics, rack-level addressing, and overlay tunnels that require precise IPAM. Learn spine-leaf address design, rack-level allocation, VXLAN planning, DC migration, and capacity reporting with LightMesh IPAM.

Data centre networks use spine-leaf fabrics, rack-level addressing, structured cabling, and overlay tunnels such as VXLAN and GENEVE. Every rack, every top-of-rack switch, every server, every management interface, and every overlay network needs address space that is planned, documented, and reconciled against what is live. Data centre IPAM is the practice of holding that address space in a single source of truth and using it to plan migrations, refreshes, and capacity.

LightMesh documents data centre subnets, tracks rack-level assignments, supports migration planning with planned-vs-live reconciliation, and provides visibility into address utilisation and overlap across fabrics. It does not configure switches, push VXLAN VNI assignments, or orchestrate hypervisors. It is a documentation and planning layer, not a control plane.

This guide covers data centre network architecture, common operational challenges, and practical LightMesh modelling recommendations. For cloud and on-prem scenarios, see Hybrid Networks. For migration workflows, see Data Migration.

Why data centre networks matter

Data centres carry the densest, highest-value address space in most estates. A single rack may host 40 to 80 servers, each with a management IP, a production IP, and an overlay interface. A medium data centre with 20 racks easily reaches 5,000 documented IP assignments, plus storage, out-of-band, console, and PDU management networks.

Three pressures make data centre IPAM strategic:

  • Migrations and refreshes depend on precise planning. Hardware refreshes, fabric upgrades, and data centre consolidations run over months. Each wave moves subnets and renumbers devices. Without a planned-vs-live view, cutovers collide with active workloads and rollback is guesswork.
  • Overlay networks multiply the surface. VXLAN, GENEVE, and similar overlays carry their own VNI and subnet assignments that are invisible to traditional IPAM. A VNI conflict or a duplicate overlay subnet can drop traffic between hypervisors without a clear cause.
  • Physical and logical address space diverge. Rack location, server hardware, cabling, and power belong to DCIM. IP assignments, VLANs, VRFs, and overlays belong to the network. When the two views diverge, incident response and capacity planning both suffer.

Data centre teams need a trusted view of address space that spans the underlay (spine-leaf), the overlay (VXLAN), the management network, and the migration plan, all reconciled against live state.

Common data centre architecture

flowchart TB
  subgraph Spine["Spine Layer"]
    S1["Spine 1"]
    S2["Spine 2"]
  end
  subgraph Leaf["Leaf / ToR Layer"]
    L1["Leaf 1 - Rack A"]
    L2["Leaf 2 - Rack B"]
    L3["Leaf 3 - Rack C"]
  end
  subgraph Rack["Rack Level"]
    R1["Rack A - Servers, Mgmt, PDU"]
    R2["Rack B - Servers, Mgmt, PDU"]
    R3["Rack C - Storage, Compute"]
  end
  subgraph Overlay["Overlay - VXLAN"]
    VNI1["VNI 10010 - Tenant A"]
    VNI2["VNI 10020 - Tenant B"]
  end
  S1 <-->|"IP fabric"| L1
  S1 <-->|"IP fabric"| L2
  S1 <-->|"IP fabric"| L3
  S2 <-->|"IP fabric"| L1
  S2 <-->|"IP fabric"| L2
  S2 <-->|"IP fabric"| L3
  L1 --> R1
  L2 --> R2
  L3 --> R3
  R1 -.->|"VTEP"| VNI1
  R2 -.->|"VTEP"| VNI1
  R2 -.->|"VTEP"| VNI2
  R3 -.->|"VTEP"| VNI2

The spine-leaf underlay carries the fabric. Each leaf serves a rack with point-to-point /31 or /30 links to the spines and /24 or larger server subnets facing the rack. VXLAN overlays run on top, with VTEPs on the leaves and VNIs mapping to tenant subnets.

Common operational challenges

  • Rack-level address drift. Servers are added, removed, and renumbered constantly. The rack spreadsheet that was accurate at deployment is stale within weeks. When a server fails, the replacement gets a new IP that may already be in use.

  • Overlay networks are invisible to legacy IPAM. VXLAN VNI assignments, VTEP addresses, and overlay subnets live in the hypervisor or controller, not in the IPAM that tracks the underlay. VNI conflicts and duplicate overlay subnets cause silent traffic drops.

  • Migration cutovers depend on planned-vs-live accuracy. A data centre migration moves subnets in waves. If the planned state does not match live state, a cutover takes down workloads that were not supposed to move yet. Rollback requires knowing exactly what changed.

  • Capacity planning needs utilisation data. Spine-leaf fabrics have finite route and ARP capacity. A subnet that is 95% utilised cannot absorb the next migration wave. Spreadsheets do not report utilisation in real time.

  • Management networks are documented separately. Out-of-band management, IPMI, iLO, console, and PDU networks each have their own address space. When a server is unreachable on the production network, the management IP is the fallback, and it must be findable.

  • DCIM and IPAM disagree. DCIM knows the rack, server, and power. IPAM knows the IP, VLAN, and VRF. When the two are not reconciled, a server that is physically present but not in IPAM becomes a security and audit blind spot.

  • Multi-tenant overlay segmentation evidence. Cloud and hosting data centres run multiple tenants on shared fabrics. Segmentation evidence for auditors requires showing which subnets and VNIs belong to which tenant, and that no overlap exists.

How LightMesh helps

Model spine-leaf and rack-level addressing

Model each data centre as a Site, each fabric or functional area as a Zone, and each subnet as a Subnet. Use Network Containers as a view feature to group similar subnets across racks or fabrics:

Network Container Purpose
Underlay Fabric Spine-leaf point-to-point links (/31 or /30)
Server Subnets Rack-facing production subnets
Management IPMI, iLO, console, PDU management
Storage SAN, NAS, replication
Overlay - VXLAN VTEP addresses and overlay subnets by VNI
Migration Waves Planned subnets for in-flight migrations

Custom attributes on Zones and Subnets capture rack ID, row, fabric, tenant, VNI, and DCIM reference.

Overlay network documentation

Use custom attributes to document VXLAN overlays that legacy IPAM misses:

Custom Attribute Purpose
VNI VXLAN network identifier
VTEP Tunnel endpoint IP
Tenant Owning tenant or application
Underlay Subnet Associated underlay subnet
Controller VTEP controller or hypervisor

This makes overlay subnets and VNIs searchable alongside underlay addresses, so VNI conflicts and duplicate overlay subnets surface during planning instead of during an outage.

Migration planning with planned-vs-live reconciliation

Before any migration wave, compare planned state in LightMesh against live state from DHCP discovery or nmap scan sync:

  • Planned subnets with no live resources (not yet migrated)
  • Live subnets with no planned allocation (undocumented)
  • Overlaps between planned and existing ranges
  • Stale reservations for decommissioned racks or servers

This workflow supports phased cutovers with rollback documentation. See Data Migration for the full workflow.

Capacity and utilisation reporting

LightMesh reports subnet utilisation so capacity planning reflects reality. Before a migration wave, check:

  • Subnet utilisation across the target fabric
  • Available capacity in destination racks
  • Subnets above 80% utilisation that cannot absorb new workloads

Incident attribution

When the SOC or a workload owner reports an issue with an IP, LightMesh resolves the attribution in seconds:

  1. Search the IP in LightMesh
  2. See the site, zone, rack, fabric, tenant, and VNI
  3. View the support group and owner
  4. Check recent changes: who modified this subnet, when, and what changed
  5. Identify NAT mappings if the IP is translated

This workflow resolves IP → rack → tenant → owner → recent changes without phone calls or DCIM cross-referencing.

Best practices

  1. Model the underlay and overlay together. Document spine-leaf point-to-point links, server subnets, management networks, and VXLAN overlays in the same IPAM. Keeping them separate guarantees they drift apart.

  2. Use custom attributes for rack and VNI. Rack ID, row, fabric, VNI, tenant, and DCIM reference make subnets searchable and give capacity planning the data it needs.

  3. Plan migrations wave by wave. Use planned subnets and reservations for each migration wave. Compare planned-vs-live before every cutover. Keep rollback documentation current.

  4. Reconcile with DCIM regularly. Cross-check LightMesh records against DCIM at least quarterly. Servers that are physically present but not in IPAM are audit and security gaps.

  5. Document management networks separately. Out-of-band, IPMI, iLO, console, and PDU networks belong in their own zones. When production is unreachable, the management IP is the fallback and must be findable.

  6. Track utilisation before capacity runs out. Review subnet utilisation weekly on high-density fabrics. Subnets above 80% utilisation need a plan before the next migration wave lands.

  7. Use audit logging for change evidence. Use audit logging to record who changed which subnet and when. This supports incident response and data centre change audits.

  8. Separate tenant address space in multi-tenant fabrics. Model each tenant’s subnets and VNIs in separate zones with clear ownership. Overlap detection across tenants prevents silent traffic drops.

What LightMesh does not do

LightMesh is a read-only source of network intelligence for data centre environments. It does not:

  • Configure switches, routers, or firewalls. LightMesh documents address space. It does not push VLAN, VRF, VXLAN, or ACL configuration to network devices.

  • Assign VXLAN VNIs or configure VTEPs. LightMesh documents VNI and VTEP assignments for visibility. The hypervisor or controller manages overlay provisioning.

  • Replace DCIM. LightMesh complements DCIM with IP, VLAN, VRF, and overlay documentation. Physical asset, power, and cabling data belongs in DCIM.

  • Orchestrate hypervisors or storage. LightMesh documents address space for compute and storage networks. Provisioning remains with the hypervisor, storage controller, or orchestration platform.

  • Guarantee compliance. LightMesh provides evidence that supports data centre audits and segmentation reviews. It does not certify compliance.

FAQ

What is a data centre network? A data centre network is the collection of spine-leaf fabrics, rack-level subnets, management networks, and overlay tunnels (VXLAN, GENEVE) that connect servers, storage, and services in a data centre. It requires precise IPAM because address density is high and migrations depend on accurate planned-vs-live state.

What is a spine-leaf fabric and why does IPAM matter? A spine-leaf fabric uses spine switches connected to every leaf (top-of-rack) switch in a full mesh. Each spine-leaf link is a point-to-point /31 or /30. Each leaf serves rack-facing server subnets. IPAM matters because every link, every server subnet, and every overlay must be documented and non-overlapping for the fabric to route correctly.

How does LightMesh handle VXLAN and overlay networks? LightMesh documents VXLAN overlays using custom attributes for VNI, VTEP, tenant, and underlay subnet. This makes overlay subnets searchable alongside underlay addresses, so VNI conflicts and duplicate overlay subnets surface during planning instead of during an outage.

Can LightMesh plan a data centre migration? Yes. Use planned subnets and reservations for each migration wave, then compare planned-vs-live state before every cutover. LightMesh tracks what should exist, what actually exists, and the gap between them, which supports phased cutovers with rollback. See Data Migration.

Does LightMesh replace DCIM? No. LightMesh complements DCIM. DCIM tracks physical assets, power, and cabling. LightMesh tracks IP assignments, VLANs, VRFs, and overlays. Reconciling the two regularly keeps physical and logical views aligned.

How does LightMesh help with data centre capacity planning? LightMesh reports subnet utilisation so capacity planning reflects reality. Before a migration wave, check utilisation across the target fabric, identify subnets above 80% utilisation, and plan additional capacity before cutovers land.

Can LightMesh document out-of-band management networks? Yes. Model out-of-band, IPMI, iLO, console, and PDU networks in their own zones with custom attributes for rack and device class. When production is unreachable, the management IP is findable in LightMesh.

References

Previous
Next