I have cabled up two switches in my 3750 lab to demonstrate some Stackwise configuration / verification / behaviors at the end of this post, however there is a lot of theory before the labbing that is needed for exam day.
Stackwise Plus will not be discussed here as it is not in the CCNP R/S exam blueprint.
Fundamentals of Stackwise
Stackwise allows for up to 9 Cisco Catalyst switches to become one logical switch unit or switch stack, with one switch being elected the Master switch during an “Election” upon boot sequence, and stack members / masters can be added or reconfigured in the stack with no service disruption to end user connections.
That “Master” switch in the stack is responsible for stack management, being the only stack member allowed to handle SSH / Telnet / IP Services, meaning individual devices cannot be remotely logged into – However they can be accessed via Console cable!
The Master switch is responsible for maintaining synchronized information across the entire stack, including forwarding information (FIB and MAC Tables), up to the actual IOS image and config file running on the stack members.
The switch stack Master election / Configuration of subordinate stack members
During the POST of a switches boot sequence, a Stack Master election takes place, where the switch detects stack members on its Stackwise interfaces and elects a “Master” switch whether other stack members are detected or not.
This means even a single switch, not stacked, is still elected as the stack “Master” on boot.
The election process has an order it goes through of tie breakers to elect who will become the master, and it goes in the following order of considerations:
- Manual configuration on a switch to become Stack Master
- Best feature set based on the IOS image among stack members
- Non-Default switch configuration (if the switch has an existing config)
- Longest uptime among all members
- Lowest MAC Address of all members
- If a switch is manually configured to be the Master switch
- The preferred feature set is considered to be an IPS (IP Services) based IOS image, followed by an IPB (IP Base) IOS image, any other image type is less preferred
- This means if the switch has been configured with anything outside of the default, even just a host name, anything detected that is non-default
- Longest uptime of all members in the switch stack
- Lowest MAC Address is the final tie-breaker for the election process
Once the Master switch is elected it will download the IOS image from its own Flash memory and distribute it to stack members along with a copy of its config file, as all stack members share one config file to keep all members of the switch stack consistent.
TFTP can also be configured on the switch stack to allow the Master switch to download IOS images to be downloaded from a TFTP Server rather than its own Flash, which is then distributed to switch stack members in the same manner (from the Master).
All members of the stack must run an IOS image supported by the stack Master, if a subordinate stack member cannot run a supported IOS image it will be put into suspension, and will only come out of suspension once an image is provided that is supported by both subordinate switch member and Master stack member.
A “Master” elected dynamically will also dynamically change its Priority value to 15, which is discussed in further detail below, so if a Master switch has a Priority other than 15 it has been manually configured to become the stack Master!
Stackwise shared Topology information
The switch stack has only a single IP Address for all members, the Master of the stack handles all incoming IP traffic to the stack (management traffic), for IP services such as SSH / Telnet / Etc.
Traffic forwarding tables such as IP Routing and MAC tables are downloaded by the Master switch, which compiles and maintains Master tables of all stack members information combined, this is then copied to the existing (and new) stack members.
The Master switch also distributes routing policies such as QoS and ACLs to subordinate switches, while individual stack members (subordinates) keep their own local copy of the MAC Table along with the Master version of the MAC table, so all stack members are aware of all port information for all stack members.
Spanning-Tree is maintained locally by stack members per VLAN, upon deletion or creating of a new VLAN / STP instance all stack members are notified to update their database, and the Master switch maintains a table of this information as well.
Spanning-Tree will never put Stackwise ports into Blocking mode.
All stack members downloads their running configuration from the Master switch, this being the “shared configuration” for the stack, to keep all switches consistent for High Availability / Redundancy purposes in case the Master switch fails or a new Master switch is configured.
Stackwise High Availability / Redundancy mechanisms and services
The physical cabling itself offers a redundancy mechanism by taking the 32gbps link and separating it into two logical 16gbps bi-directional paths, with the “Egress Queue” of the Stackwise interface performing counter-rotating traffic forwarding over the two paths, which allows for load balancing / load distribution of traffic.
However if one of these two logical paths were to break or fail, the detecting switch will alert all stack members of this detection, and all stack members will throttle back their throughput to only use one of the two logical paths (cutting the throughput by 50%).
Upon the path being detected as back up / active, the path is restored and used again.
A very important note for exam day on throughput
The Full / 32gbps throughput is only available when there is a Full Ring topology of Stackwise cabling, meaning stack members cable to sequentially to one another and the last final switch connecting back up to the top switch, or it will run at 16gbps the same as if there were a break in the Stackwise cabling:
The Full ring stack on the right could scale down 9 switches, all just connected to its neighbor switches (as each 3750 only has two Stackwise ports), however the final switch has to connect back to the to first switch to create a “full circuit” or Full Ring.
Verifying and troubleshooting throughput will be reviewed in the lab section of this post.
NSF (Non Stop Forwarding) work in conjunction with RPR+ (Routing Processor Redundancy+) to keep switch stack members synchronized with the Master switch by replicating IP Services and Addressing of interfaces on subordinate switches to that of the Master switch, so upon Master failure they can immediately step in.
NSF and RPR+ are focused on Master switch failure or change over to keep traffic flowing, Layer 2 is handled by a “Distributed forwarding” method, essentially all switches continue to forward traffic independently to keep Layer 2 traffic moving until the new Master takes over the switch stack.
1:N Redundancy is a method of making all switch stack members Master switches, so in the event 1 switch completely fails, another can pick up immediately without the need to elect a new Master switch.
EtherChannels also have redundancy offered by Stackwise, but only with the open industry standard EtherChannel types, this means PAgP does not work.
My full post on different EtherChannel types can be found here, however to illustrate EtherChannel cross-stack redundancy, I’ve pulled some diagrams from that post:
I configured this on the SW1 stack of members SW1 and SW2 with an LACP EtherChannel back to SW3, and to demonstrate this working I just verified the interfaces in my active Port-Channel on the switch stack:
SW1(config-if-range)#do sh int po13
Port-channel13 is up, line protocol is up (connected)
Hardware is EtherChannel, address is 1ce6.c7c1.c803 (bia 1ce6.c7c1.c803)
MTU 1500 bytes, BW 400000 Kbit/sec, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, link type is auto, media type is unknown
input flow-control is off, output flow-control is unsupported
Members in this channel: Fa1/0/1 Fa1/0/2 Fa2/0/1 Fa2/0/2
This applies with “On” EtherChannel as well though I will spare the output, this demonstrates these two modes provide EtherChannel redundancy when using Stackwise technology, whereas with Cisco proprietary PAgP it’s a much different story.
I can’t show the Port-Channel interface for this demonstration because the physical interfaces never join the EtherChannel bundle, giving the following error upon configuration on the switch stack:
SW1(config)#int ra fa1/0/1 – 2
SW1(config-if-range)#channel-group 13 mode desirable
Creating a port-channel interface Port-channel 13
SW1(config-if-range)#
(Lots of output from physical interfaces bouncing Down / Up)
SW1(config-if-range)#
SW1(config-if-range)#
SW1(config-if-range)#int ra fa2/0/1 – 2
SW1(config-if-range)#channel-group 13 mode desirable
%With PAgP enabled, all ports in the Channel should belong to the same switch
Command rejected (Port-channel13, Fa2/0/1): Invalid etherchnl mode
% Range command terminated because it failed on FastEthernet2/0/1
SW1(config-if-range)#
Even in the same logical switch unit / switch stack, the interfaces must be on the same physical switch with PAgP, or they will not join the bundle!
Stackwise behaviors when creating / Modifying the switch stack / Misc info
Upon creating of a switch stack, all switches that detect a cable plugged into the Stackwise port will initiate creating a stack / electing a Master switch, which is done during the POST or Boot Sequence of the switch.
The election process will only take place if no Master switch has yet been designated, once a Master switch is designated, and subsequent joining members will be added as a subordinate switch stack member (and will receive its running config / QoS / ACLs / etc from the stack Master) until the Master is removed / reloaded / whole stack is reloaded.
Any upgrades to the switch stack results in all the stack members receiving the update as well, as stack members must either be running an IOS version compatible with the stack Master, or the switch will be suspended from the switch stack.
As mentioned if no other switches are detected as being connected with a Stackwise cable, the switch elects itself Stack master, and is referred to as a “single switch stack” in some corners of the networking world (verbiage may show up on exam day).
Configuration and verification of switch stack, and Boot sequence Master election
I’ll run through the few configuration options there are for Stackwise, as there is not a whole lot of configuration options available, however there are some odd quirks to know both for exam day and on the job as from forums I’ve seen some engineers struggle with some of these behaviors in the workplace.
To begin I want to demonstrate a command that is NOT for stackwise verification:
SW2#sh stack ?
<1-8192> Process to show stack detail on
| Output modifiers
<cr>
This command will give a list of all processes running on the switches software “stack”, and has nothing to do with Stackwise, so on exam day “show stack” is not a valid Stackwise verification command!
That being said, here IS a good verification command for Stackwise:
“show switch”
SW1#sh switch
Switch/Stack Mac Address : 1ce6.c7c1.c800
                                                                                           H/W  Current
Switch#    Role     Mac Address      Priority  Version  State
———————————————————-
  *1      Master    1ce6.c7c1.c800        15        0       Ready
    2          Member  5897.1eab.c800       1              0       Ready
SW1#
This gives a lot of basic information for each switch stack member by its switch # in the switch stack which I’ll dive into bullet point style:
- Switch # – The # assigned to the switch as a stack member ranging 1-9
- Â Role – Master or Member, all can be configured as Master for 1:N Redundancy
- MAC Address – The different local MAC Addresses of the stack members
- Priority # – The higher this priority #, the more preferred to be elected Master
- HW Version # – Actually not really sure on this, please leave a comment on this post if you know what this Version # refers to, the software version # is different from 0
- Â Current State – Indicates if a switch is active in the stack (Ready), booting up / joining the stack (Progressing), or whether a stack member was unable to join the group and will indicate why with an error like “Version mismatch” or “SDM Mismatch” as the stack members current state
There are some changes that can be made to these values which I will cover below.
To compare with its detailed counterpart, it actually gives the same output with an additional segment with Port information:
SW1#sh switch det
Switch/Stack Mac Address : 1ce6.c7c1.c800
                                                                                           H/W  Current
Switch#    Role     Mac Address      Priority  Version  State
———————————————————-
  *1      Master    1ce6.c7c1.c800        15        0       Ready
    2          Member  5897.1eab.c800       1              0       Ready
SW1#
                 Stack Port Status              Neighbors
Switch# Â Â Port 1 Port 2 Â Â Â Â Â Â Â Â Â Â Â Â Â Port 1 Port 2
——————————————————–
    1               Ok  Ok                                           2 2
    2               Ok  Ok                                           1 1
SW1#
This gives information in regards to the Stacking port states and mappings between stack members, which is actually a combination of a couple different verification commands:
“sh switch stack-ports”
SW1#sh switch stack-ports
Switch # Port 1 Port 2
——– —— ——
1 Â Â Â Â Â Â Â Â Â Â Ok Ok
2Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Ok Ok
SW1#sh switch neighbors
Switch # Port 1 Port 2
——– —— ——
1 Â Â Â Â Â Â Â Â Â Â Â Â 2 2
2Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 1 1
SW1#
So the exact same info as these two verification commands, though it should be noted these individual / less informative commands are valid verification commands for the info displayed if asked on exam day.
Now because I only have 2 Stackwise cables, I was only able to create a “Full Ring” Stackwise Topology with two switches, SW1 and SW2:
However this will be enough to demonstrate some different concepts to watch for on exam day and on the job!
First I want to review all configuration related options for stackwise:
Changing the Priority
SW1#conf t
Enter configuration commands, one per line. End with CNTL/Z.
SW1(config)#switch ?
<1-9> Switch Number
SW1(config)#switch 1 ?
priority Set the priority of the specified switch
provision Configure Switch provision / offline config
renumber Renumber the specified switch number
SW1(config)#switch 1 priority ?
<1-15> Switch Priority
SW1(config)#switch 1 priority 1 ?
<cr>
SW1(config)#switch 1 priority 1
Changing the Switch Priority of Switch Number 1 to 1
Do you want to continue?[confirm]
New Priority has been set successfully
Verification
SW1#sh switch
Switch/Stack Mac Address : 1ce6.c7c1.c800
                                                                                           H/W  Current
Switch#    Role     Mac Address      Priority  Version  State
———————————————————-
  *1      Master    1ce6.c7c1.c800        1       0       Ready
    2          Member  5897.1eab.c800       15           0       Ready
SW1#
As can be seen, SW2 or stack member 2 also had its Priority changed to 15, which makes it the preferred Master, however it will not change roles until a reload is performed.
I will reload the switch after some more configuration changes to see what happens!
Configuring Provisioning
SW1(config)#switch 1 provision ?
nme-xd-24es-1s provision an EtherSwitch SM with 24+1G interfaces
nme-xd-24es-1s-p provision an EtherSwitch SM with 24+1G interfaces
rulai RULAI test board
ws-c3750-24fs provision a Catalyst 3750 switch with 24FX+2G
interfaces
ws-c3750-24p provision a Catalyst 3750 switch with 24pwr+2G
interfaces
ws-c3750-24ts provision a Catalyst 3750 switch with 24+2G interfaces
ws-c3750-48p provision a Catalyst 3750 switch with 48pwr+4G
interfaces
(ETC…)
This is “provisioning” a switches interfaces for one model to be that of a different model, which I am not sure a case use for, so I just more want to know that this is here.
The only case use I have seen for provisioning is to correct a stack member when it goes into an error showing its State as “Provisioned” which I will demonstrate below.
Re-numbering stack members
SW1(config)#switch 1 renumber ?
<1-9> New switch number
SW1(config)#switch 1 renumber 9 ?
<cr>
SW1(config)#switch 1 renumber 9
WARNING: Changing the switch number may result in a
configuration change for that switch.
The interface configuration associated with the old switch
number will remain as a provisioned configuration.
Do you want to continue?[confirm]
Changing Switch Number 1 to Switch Number 9
New Switch Number will be effective after next reboot
Here we finally see the output informing us as the network administrator that this will not take effect until the next reboot, which is needed for the Priority as well.
There are two methods to go about reloading in Stackwise, a “reload” can be issued to reboot the entire Stack, or by reloading individual stack members with the following command:
SW1#reload slot ?
<1-9> Slot number of RP or line card
SW1#reload slot 2 ?
<cr>
This will reload the stack member # specified, whereas “reload” even when console’d into a particular switch will reload the entire stack.
Upon a reload of the entire stack, in the boot sequence it appears to take the changes:
Election Complete
Switch 9 booting as Member, Switch 2 elected Master
HCOMP: Compatibility check PASSED
Waiting for feature sync….
Waiting for Port download…Complete
Waiting for Master Ready…Complete
However upon checking the “show switch” output:
SW1#sh switch
Switch/Stack Mac Address : 5897.1eab.c800
                                                                        H/W     Current
Switch# Role      Mac Address Priority Version  State
———————————————————-
1          Member 0000.0000.0000  0       0         Provisioned
*2         Master  5897.1eab.c800  15      0            Ready
9Â Â Â Â Â Â Â Â Â Â Â Member 1ce6.c7c1.c800 Â 1Â Â Â Â Â Â Â Â Â Â Â Â 0Â Â Â Â Â Â Â Â Â Â Â Â Â Ready
I have not found a clear answer on what causes this behavior, however after a bit of troubleshooting on my own, I found use of that provisioning command (or more specifically negating it) to remove this non-existent stack member.
First I tried to “reload” the slot:
SW1#reload slot 1
%ERROR: Switch number 1 can’t be reached
SW1#
Then I tried to negate the provisioning, as that is what is shown as the “State” issue:
SW1(config)#no switch 1 provision
SW1(config)#do sh switch
Switch/Stack Mac Address : 5897.1eab.c800
                                                                        H/W     Current
Switch# Role      Mac Address Priority Version  State
———————————————————-
*2         Master  5897.1eab.c800    15      0            Ready
9Â Â Â Â Â Â Â Â Â Â Â Member 1ce6.c7c1.c800 Â Â Â 1Â Â Â Â Â Â Â Â Â Â Â 0Â Â Â Â Â Â Â Â Â Â Â Â Â Ready
No model # is needed when negating it, it just removes the stack member from the stack, oddly enough the provision command cannot be negated prior to reload to avoid this error:
SW1(config)#no switch 2 provision
%Switch can not be un-provisioned when it is physically present
SW1(config)#
Enabling / Disabling Stackwise interfaces (Done at User Exec level!)
SW1#switch 2 ?
stack Stack port enable or disable
SW1#switch 2 stack ?
port Stack port enable or disable
SW1#switch 2 stack port ?
<1-2> Stack port number to enable/disable
SW1#switch 2 stack port 2 ?
disable Disable stack port
enable Enable stack port
SW1#switch 2 stack port 2 disable
Enabling/disabling a stack port may cause undesired stack changes. Continue?[confirm]
SW1#
*Mar 1 00:37:32.861: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 9 has changed to state DOWN
*Mar 1 00:37:32.995: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 2 Switch 2 has changed to state DOWN
SW1#
This has now reduced the two switch stack to have a non-Full Ring Topology for Stackwise cabling, which will also reduce its throughput speed, which can be verified with the following command:
SW1#sh switch ?
<1-9> Switch Number
detail show detailed information about the stack ring
neighbors show each switch’s neighbors
stack-ports show the status of the stack ports
stack-ring show stack ring
| Output modifiers
<cr>
SW1#sh switch stack-ring ?
activity show stack ring activity
speed show stack ring speed
SW1#sh switch stack-ring speed
Stack Ring Speed : 16G
Stack Ring Configuration: Half
Stack Ring Protocol : StackWise
SW1#
It shows here that the Ring Configuration is at Half, 16gbps throughput as we’d expect, and to use verification to fix this issue:
SW1#sh switch stack-ports
Switch # Port 1 Port 2
——– —— ——
2Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Ok Down
9Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Down Ok
SW1#
Switch 2 port 2 was disabled, and it reflects that in the “show switch stack-ports” output, as well as it would the “sh switch det” output.
From that output, we can see that either switch 2 port 2 or switch 9 port 1 needs to be enabled, so I’ll start with switch 9 port 1 to see if that will work to kick start switch 2:
SW1#switch 9 stack port 1 enable
Enabling/disabling a stack port may cause undesired stack changes. Continue?[confirm]
SW1#
That did not bring the Stackwise ports back up, so switch 2 port 2 is required:
SW1#switch 2 stack port 2 enable
Enabling/disabling a stack port may cause undesired stack changes. Continue?[confirm]
SW1#
*Mar 1 00:47:29.861: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 9 has changed to state UP
*Mar 1 00:47:29.995: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 2 Switch 2 has changed to state UP
SW1#
The stackwise ports are now back up, and the speed verification command should indicate we are now back to full speed ahead:
SW1#sh switch stack-ring speed
Stack Ring Speed : 32G
Stack Ring Configuration: Full
Stack Ring Protocol : StackWise
SW1#
And the Full Ring is now back up and running!
One last note for exam day, please excuse the gigantic output here, is that interfaces will be listed beginning with the switch stack # or the “slot #” of the switch.
Generally it would be 1/0/x, 2/0/x, etc. However because I made one switch # 2 and the other # 9, this is our interface list between the two stacked 3750’s:
SW1#sh ip int brief
Interface IP-Address OK? Method Status Protocol
Vlan1 unassigned YES NVRAM administratively down down
FastEthernet2/0/1 unassigned YES unset down down
FastEthernet2/0/2 unassigned YES unset down down
FastEthernet2/0/3 unassigned YES unset down down
FastEthernet2/0/4 unassigned YES unset down down
FastEthernet2/0/5 unassigned YES unset down down
FastEthernet2/0/6 unassigned YES unset down down
FastEthernet2/0/7 unassigned YES unset down down
FastEthernet2/0/8 unassigned YES unset down down
FastEthernet2/0/9 unassigned YES unset down down
FastEthernet2/0/10 unassigned YES unset down down
FastEthernet2/0/11 unassigned YES unset down down
FastEthernet2/0/12 unassigned YES unset down down
FastEthernet2/0/13 unassigned YES unset down down
FastEthernet2/0/14 unassigned YES unset down down
FastEthernet2/0/15 unassigned YES unset down down
FastEthernet2/0/16 unassigned YES unset down down
FastEthernet2/0/17 unassigned YES unset down down
FastEthernet2/0/18 unassigned YES unset down down
FastEthernet2/0/19 unassigned YES unset down down
FastEthernet2/0/20 unassigned YES unset down down
FastEthernet2/0/21 unassigned YES unset down down
FastEthernet2/0/22 unassigned YES unset down down
FastEthernet2/0/23 unassigned YES unset down down
FastEthernet2/0/24 unassigned YES unset down down
GigabitEthernet2/0/1 unassigned YES unset up up
GigabitEthernet2/0/2 unassigned YES unset up up
FastEthernet9/0/1 unassigned YES unset down down
FastEthernet9/0/2 unassigned YES unset down down
FastEthernet9/0/3 unassigned YES unset down down
FastEthernet9/0/4 unassigned YES unset down down
FastEthernet9/0/5 unassigned YES unset down down
FastEthernet9/0/6 unassigned YES unset down down
FastEthernet9/0/7 unassigned YES unset down down
FastEthernet9/0/8 unassigned YES unset down down
FastEthernet9/0/9 unassigned YES unset down down
FastEthernet9/0/10 unassigned YES unset down down
FastEthernet9/0/11 unassigned YES unset down down
FastEthernet9/0/12 unassigned YES unset down down
FastEthernet9/0/13 unassigned YES unset down down
FastEthernet9/0/14 unassigned YES unset down down
FastEthernet9/0/15 unassigned YES unset down down
FastEthernet9/0/16 unassigned YES unset down down
FastEthernet9/0/17 unassigned YES unset down down
FastEthernet9/0/18 unassigned YES unset down down
FastEthernet9/0/19 unassigned YES unset down down
FastEthernet9/0/20 unassigned YES unset down down
FastEthernet9/0/21 unassigned YES unset down down
FastEthernet9/0/22 unassigned YES unset down down
FastEthernet9/0/23 unassigned YES unset down down
FastEthernet9/0/24 unassigned YES unset down down
GigabitEthernet9/0/1 unassigned YES unset up up
GigabitEthernet9/0/2 unassigned YES unset up up
SW1#
This demonstrates that depending on the switch stack # configured for each switch, the interface # will dynamically change along with it, after the switch is reloaded of course to apply the changes!
That covers everything that should be needed for Stackwise on exam day!
This should be way more than enough to make it through both SWITCH and TSHOOT, and that is all I will write about Stackwise, next up is more switch redundancy in the form of HSRP!
The DEVNET GRIND! 
One thought on “Stackwise – Fundamentals, Master election process, configuration / verification / behaviors labbed!”