Initial Nexus 5548 Configuration

For a customer I installed a Nexus solution, based on 5548 switches (5k) en 2248 FEX (2k) devices. In this post I will show you how to configure the Nexus switches to function as the toplogy beneath shows.

nexus

Although there are two Nexus 5k switches in the drawing, they function as one logical unit after configuring.

Before you start on a Nexus device, enable the services you need. In our case the following services are enabled:
!
feature vcp
feature lacp
feature fex
!

First configure the managment ports on both switches, so we can access the devices from the network. Use the management vrf, the default vrf which is on the device out of the box.

!

interface mgmt0
ip address <ip address /subnet>
!
vrf context management
ip route 0.0.0.0/0 <next hop>
!

Now configure the Peer-Keepalive link, this is a “heartbeat” between the two Nexus 5k switches.  First configure a vPC domain.
!
vpc domain 1
role priority 1200
!
They actual Peer-Keepalive configuration is done under the vPC domain configuration

!
vpc domain 1
role priority 1200
peer-keepalive destination <target ip address> source <local ip address>
!

In our case we used the management port as the Peer-Keepalive link, but it is also possible to use a normal switchport for the Peer-Keepalive.
To verify the above configuration use the “show vpc peer-keepalive” commando.
show_vpc_peer-keepalive

Now we have to configure the Peer Link. This link has to be redundant, in our configuration we use a port-channel of 2 x 10Gb links. Traffic on this link is intra vPC traffic, therefore the only traffic on this link is the traffic of vlan’s that are used in the vPC’s.
!
int eth x/x – x
channel-group <nr> mode active
no shut
!
int Po<x>
switchport mode trunk
switchport trunk allowed vlan x,x,x,x,x
vpc peer-link
!

To verify the vPC Peer-Link use the “show vpc” command.
show_vpc

 

The basic configuration of the two Nexus 5k switches is done. Now we can start connecting the FEX’s.  A FEX can be compared with a line card in a modular switch like the Cisco 6500. Like a line card, the management of a FEX is done on the Nexus 5k switch. There are two possible options to connect a FEX.
1) Redundant uplinks to both Nexus 5k switches
2) Single connection to one of the Nexus 5k switches

Because the 2 Nexus 5k switches work as one logical unit and all servers are connected to a FEX on unit 1 and FEX on unit two, it’s not necessary to connect a FEX redundant to both Nexus 5k units.
It’s possible to connect 24 FEX devices per 5k unit in layer 2 mode on the standard license. So if the FEX’s only connect to one 5k unit, it is possible to connect up to 48 FEX units with two 5k units in layer 2 mode. If you connect a FEX redundant to both units, it will cost a license on both units, which means that only 24 FEX units can connect to your Nexus 5k units.

Fot this implementations the customer chose for option two as described above.

To connect a FEX to a 5k unit the following configuration is needed
!
interface port-channel10
switchport mode fex-fabric
fex associate 101
!
interface Ethernet1/1
switchport mode fex-fabric
fex associate 101
channel-group 10
!
interface Ethernet1/2
switchport mode fex-fabric
fex associate 101
channel-group 10
!

It will take some time before the FEX is available, to monitor the status of the FEX, use the “show fex <nr>”
sh_fex_nr

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: