Technical note on IP fabrics published

Technical note on IP fabrics published

A while (too long) ago, I promised that I would follow up on my discussion about Ethernet fabrics with a similar discussion on using IP fabrics to interconnect Calico nodes.  That work is (finally) done, and can be found in our technical documentation entitled, IP Interconnect Fabrics in Calico.

For those of you who will eventually get around to reading it, but want the spoiler right now, we conclude that IP interconnect fabrics (i.e. running a top of rack switch as a router) is feasible in Project Calico.  There are more considerations than in the Ethernet case, discussed in the Calico Over an Ethernet Inteconnect Fabric note, some of which include:

  • Size of the routing table in the Calico network and compared to the forwarding table size in the switches being used
  • Insuring proper autonomous system  boundaries and insuring continuity of those autonomous systems
  • Insuring correct next–hop behavior in the routing announcements

We propose two viable BGP models for a Calico IP interconnect fabric:

  • An AS per rack model
  • An AS per server model

Each of these models can have two variants, depending on whether the spine switches are also BGP routers, or separate Ethernet planes as in the Ethernet fabric option.

Please read the note, and comment, make suggestions, rant & rave, or congratulate our intelligence.

Christopher is the original architect of Project Calico and one of the project's evangelists. In his day job, he's the director of solutions architecture at Metaswitch Networks. Prior to Calico/Metaswitch, he's designed and run some bio-informatics OpenStack clusters, done some SDN architecture work at Big Switch Networks, Run architecture at two large carriers (Telstra - AS1221, and Cable & Wireless/iMCI - AS3561) and been the IP CTO for Alcatel in Asia. He's also run networks in Antarctica (hint, bend radius becomes REALLY important at -50C), and been foolish enough to do a stint as a wg co-chair in the IETF. Occasionally you can have the (mis-)fortune of hearing him speak at conferences and the like.