Cisco Catalyst 9200 Switch Stacking

Categories Cisco, Switch

Things to know:

  • A switch stack can have up to eight stacking-capable switches connected through their StackWise ports.
  • Only homogenous stacking is supported -Catalyst 9200 switches will only allow other 9200s as stack members.
  • Switch members can’t have different licenses

Overview

  • up to eight stacking-capable switches connected through their StackWise ports
  • The stack as seen as one device across Layers 2 and 3.
  • Active switch controls the operation and managment of the entire stack.
  • The active switch contains the saved and running configuration files for the switch stack. The configuration files include the system-level settings for the switch stack and the interface-level settings for each stack member. Each stack member has a current copy of these files for back-up purposes.

Mac Address and Bridge ID (Layer 2)

  • Configuring MAC persistency so that the stack MAC address never changes to a different MAC address to avoid LACP and PAgP flaps/inconsistencies.
  • If the changes, the MAC address of the new determines the new bridge ID and router MAC address.
  • If the entire switch stack reloads, the switch stack uses the MAC address of the .

Upgrading Software

  • auto-upgrade and auto-advise features enable a switch with software packages that are incompatible with the switch stack to be upgraded to a compatible software version so that it can join the switch stack.

Priority

  • The switch with the higher priority becomes the stack master. This can be seen by using the show switch command and looking at the priority values. switch x priority x if switch 1 priority 15 and switch 2 priority 10 than switch 1 will be elected the master.

Adding a new member

  1. Power off the new switch.
  2. Connect the new switch to the stack using the StackWise cables.
  3. Power on the new switch.

Failure

  • If the stack master is removed or powered off the standby switch will become the new active switch. All other stack members in the stack remain as members and will not reboot.
  • If two devices become stack master one stack will have members the other stack master will show as a standalone device. Use the mode button and port LEDs on the device to identify which device is the master and which devices belong to that stack.

Troubleshooting: C9200L-48P-4X Fuji code Locked Up

Categories Cisco, Switch
  • A C9200L had locked up. All switchports were dead/no light and the Fiber modules were a solid amber lit color. All ports had hosts on the other end that were still alive and sending electrical signal to their ports. I attempted to power cycle by pulling the power cables and reseating the redundant power supplies. After two attempts the switch was still locked up with fans staying on high after POST should have cleared.

Troubleshooting Information

  • Do any lights at all illuminate? I see the switchports don’t, but is any activity seen on the front panel?
    Sfp ports stayed solid amber – switchports none lit, no activity when reseating connections
  • Do the PSUs appear to power on?
    Yes both PSUs appear to power on, switch fans kick on during POST and stay on full speed
  • Has the device ever powered on?
    Yes device was pulled from production
  • What version of code was running (if known/applicable)?
    Fuji – 16.9.4

While attempting to swap the 9200L with a loaner switch I ran into the following warning messages. NOTE: The fiber and SFP modules were being reseated into different members of the stack until the RMA could come in.

No Big Deal

  • I had never ran into the Duplicate GBIC error before. While attempting to do some research on this I ran into bug reports of this occuring on 3850s.

Solution/Work Around

  • Remove the old switch member
  • no errdisable detect cause gbic-invalid
  • reseat connections
  • admin shut/no shut the module ports.




  • I figured that removing the stack member, reseating the connections would be enough but for some reason the ports were still errdisabled.
  • I had to shut/no shut the ports twice after reseating each connection. Once I did this the ports moved out of errdisable.
 SWITCH-NAME(config)#
 *Feb 24 15:00:25.568: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te1/1/2 removed
 SWITCH-NAME(config)#
 *Feb 24 15:00:54.982: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/2
 SWITCH-NAME(config)#
 *Feb 24 15:02:52.913: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te1/1/3 removed
 SWITCH-NAME(config)#
 *Feb 24 15:04:47.672: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/3
 SWITCH-NAME(config)#Warning: [1 51] is dup of [3 50]
 SWITCH-NAME(config)#end
 SWITCH-NAME#sh logg
 Syslog logging: enabled (0 messages dropped, 7 messages rate-limited, 0 flushes, 0 overruns, xml disabled, filtering disabled)
 No Active Message Discriminator.
 No Inactive Message Discriminator.
 Console logging: level emergencies, 0 messages logged, xml disabled,                  filtering disabled Monitor logging: level debugging, 173 messages logged, xml disabled,                  filtering disabled     Logging to: vty2(7) Buffer logging:  level debugging, 46694 messages logged, xml disabled,                 filtering disabled Exception Logging: size (4096 bytes) Count and timestamp logging messages: disabled File logging: disabled Persistent logging: disabled
 No active filter modules.
 Trap logging: level informational, 46254 message lines logged     Logging Source-Interface:       VRF Name:
 Log Buffer (4096 bytes):
  port Gi1/0/26 and port Gi1/0/25
 *Feb 24 14:25:45.184: %SYS-6-LOGOUT: User pete has exited tty session 2(10.10.16.40)
 *Feb 24 14:46:24.069: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te2/1/2Warning: [2 50] is dup of [3 52]
 *Feb 24 14:46:24.069: %PM-4-ERR_DISABLE: gbic-invalid error detected on Te2/1/2, putting Te2/1/2 in err-disable state
 *Feb 24 14:46:49.152: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/42, changed state to down
 *Feb 24 14:46:50.168: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/42, changed state to down
 *Feb 24 14:46:58.360: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/1Warning: [1 49] is dup of [3 49]
 SWITCH-NAME#ter le 0
 SWITCH-NAME#sh logg
 *Feb 24 14:51:02.833: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/1Warning: [1 49] is dup of [3 49]
 *Feb 24 14:51:48.227: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/2Warning: [1 50] is dup of [3 51]
 *Feb 24 14:51:48.227: %PM-4-ERR_DISABLE: gbic-invalid error detected on Te1/1/2, putting Te1/1/2 in err-disable state
 *Feb 24 14:52:18.181: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te1/1/2 removed
 *Feb 24 14:52:38.420: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/2Warning: [1 50] is dup of [3 51]
 *Feb 24 14:53:07.578: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/3Warning: [1 51] is dup of [3 50]
 *Feb 24 14:53:07.578: %PM-4-ERR_DISABLE: gbic-invalid error detected on Te1/1/3, putting Te1/1/3 in err-disable state
 SWITCH-NAME#conf t
 Enter configuration commands, one per line.  End with CNTL/Z.
 SWITCH-NAME(config)#no errdisable detect cause gbic-invalid
 SWITCH-NAME(config)#exi
 SWITCH-NAME#
 *Feb 24 15:12:05.166: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te2/1/2 removed
 SWITCH-NAME#
 *Feb 24 15:12:32.313: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te2/1/2
 SWITCH-NAME#Warning: [2 50] is dup of [3 52]
 SWITCH-NAME#sh logg
 *Feb 24 14:46:24.069: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te2/1/2Warning: [2 50] is dup of [3 52]
 *Feb 24 14:46:24.069: %PM-4-ERR_DISABLE: gbic-invalid error detected on Te2/1/2, putting Te2/1/2 in err-disable state
 *Feb 24 14:46:49.152: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/42, changed state to down
 *Feb 24 14:46:50.168: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/42, changed state to down
 *Feb 24 14:46:58.360: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/1Warning: [1 49] is dup of [3 49]
 *Feb 24 14:46:58.360: %PM-4-ERR_DISABLE: gbic-invalid error detected on Te1/1/1, putting Te1/1/1 in err-disable state
 *Feb 24 14:47:00.408: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/42, changed state to up
 *Feb 24 14:47:02.420: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/42, changed state to up
 *Feb 24 14:50:28.930: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/42, changed state to down
 *Feb 24 14:50:29.942: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/42, changed state to down
 *Feb 24 14:50:32.982: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/42, changed state to up
 *Feb 24 14:50:35.463: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/42, changed state to down
 *Feb 24 14:50:38.714: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/42, changed state to up
 *Feb 24 14:50:39.922: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te1/1/1 removed
 *Feb 24 14:50:40.721: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/42, changed state to up
 *Feb 24 14:50:52.774: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/42, changed state to down
 *Feb 24 14:50:53.788: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/42, changed state to down
 *Feb 24 14:50:56.717: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/42, changed state to up
 *Feb 24 14:50:58.729: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/42, changed state to up
 *Feb 24 14:51:02.833: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/1Warning: [1 49] is dup of [3 49]
 *Feb 24 14:51:48.227: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/2Warning: [1 50] is dup of [3 51]
 *Feb 24 14:51:48.227: %PM-4-ERR_DISABLE: gbic-invalid error detected on Te1/1/2, putting Te1/1/2 in err-disable state
 *Feb 24 14:52:18.181: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te1/1/2 removed
 *Feb 24 14:52:38.420: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/2Warning: [1 50] is dup of [3 51]
 *Feb 24 14:53:07.578: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/3Warning: [1 51] is dup of [3 50]
 *Feb 24 14:53:07.578: %PM-4-ERR_DISABLE: gbic-invalid error detected on Te1/1/3, putting Te1/1/3 in err-disable state
 *Feb 24 14:56:02.489: %SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user: pete] [Source: 192.168.1.5] [localport: 23] at 14:56:02 UTC Wed Feb 24 2021
 *Feb 24 15:00:25.568: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te1/1/2 removed
 *Feb 24 15:00:54.982: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/2
 *Feb 24 15:02:52.913: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te1/1/3 removed
 *Feb 24 15:04:47.672: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/3Warning: [1 51] is dup of [3 50]
 *Feb 24 15:06:07.071: %SYS-5-CONFIG_I: Configured from console by pete on vty0 (192.168.1.5)
 *Feb 24 15:11:22.731: %SYS-5-CONFIG_I: Configured from console by pete on vty0 (192.168.1.5)
 *Feb 24 15:12:05.166: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te2/1/2 removed
 *Feb 24 15:12:32.313: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te2/1/2Warning: [2 50] is dup of [3 52]
 SWITCH-NAME#
 *Feb 24 15:14:09.249: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te1/1/2 removed
 SWITCH-NAME#
 *Feb 24 15:14:16.391: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/2
 SWITCH-NAME#Warning: [1 50] is dup of [3 51]
 SWITCH-NAME#show sw
 SWITCH-NAME#show switch
 Switch/Stack Mac Address : 10b3.d582.9880 - Local Mac Address
 Mac persistency wait time: Indefinite
                                              H/W   Current
 Switch#   Role    Mac Address     Priority Version  State
 1       Standby  4c71.0d81.xxxx     1      V01     Ready
  2       Member   7c21.0e62.xxxx     1      V01     Ready
  3       Member   0000.0000.xxxx     0      V01     Removed
 *4       Active   10b3.d582.xxxx     1      V01     Ready
 SWITCH-NAME#conf t
 Enter configuration commands, one per line.  End with CNTL/Z.
 SWITCH-NAME(config)#no switch 3 provision
 SWITCH-NAME(config)#
 *Feb 24 15:19:14.899: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te1/1/1 removed
 SWITCH-NAME(config)#
 *Feb 24 15:19:24.716: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/1
 SWITCH-NAME(config)#Warning: [1 49] is dup of [3 49]
 SWITCH-NAME(config)#int ra te 1/1/1 - 2
 SWITCH-NAME(config-if-range)#no shut
 SWITCH-NAME(config-if-range)#do sh logg
 Syslog logging: enabled (0 messages dropped, 7 messages rate-limited, 0 flushes, 0 overruns, xml disabled, filtering disabled)
 No Active Message Discriminator.
 No Inactive Message Discriminator.
 Console logging: level emergencies, 0 messages logged, xml disabled,                  filtering disabled Monitor logging: level debugging, 183 messages logged, xml disabled,                  filtering disabled     Logging to: vty2(17) Buffer logging:  level debugging, 46704 messages logged, xml disabled,                 filtering disabled Exception Logging: size (4096 bytes) Count and timestamp logging messages: disabled File logging: disabled Persistent logging: disabled
 No active filter modules.
 Trap logging: level informational, 46261 message lines logged     Logging Source-Interface:       VRF Name:
 Log Buffer (4096 bytes):
 PDOWN: Line protocol on Interface GigabitEthernet1/0/42, changed state to down
 *Feb 24 14:46:50.168: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/42, changed state to down
 *Feb 24 14:46:58.360: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/1Warning: [1 49] is dup of [3 49]
 *Feb 24 14:46:58.360: %PM-4-ERR_DISABLE: gbic-invalid error detected on Te1/1/1, putting Te1/1/1 in err-disable state
 *Feb 24 14:47:00.408: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/42, changed state to up
 *Feb 24 14:47:02.420: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/42, changed state to up
 *Feb 24 14:50:28.930: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/42, changed state to down
 *Feb 24 14:50:29.942: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/42, changed state to down
 *Feb 24 14:50:32.982: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/42, changed state to up
 *Feb 24 14:50:35.463: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/42, changed state to down
 *Feb 24 14:50:38.714: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/42, changed state to up
 *Feb 24 14:50:39.922: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te1/1/1 removed
 *Feb 24 14:50:40.721: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/42, changed state to up
 *Feb 24 14:50:52.774: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/42, changed state to down
 *Feb 24 14:50:53.788: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/42, changed state to down
 *Feb 24 14:50:56.717: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/42, changed state to up
 *Feb 24 14:50:58.729: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/42, changed state to up
 *Feb 24 14:51:02.833: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/1Warning: [1 49] is dup of [3 49]
 *Feb 24 14:51:48.227: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/2Warning: [1 50] is dup of [3 51]
 *Feb 24 14:51:48.227: %PM-4-ERR_DISABLE: gbic-invalid error detected on Te1/1/2, putting Te1/1/2 in err-disable state
 *Feb 24 14:52:18.181: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te1/1/2 removed
 *Feb 24 14:52:38.420: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/2Warning: [1 50] is dup of [3 51]
 *Feb 24 14:53:07.578: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/3Warning: [1 51] is dup of [3 50]
 *Feb 24 14:53:07.578: %PM-4-ERR_DISABLE: gbic-invalid error detected on Te1/1/3, putting Te1/1/3 in err-disable state
 *Feb 24 14:56:02.489: %SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user: pete] [Source: 192.168.1.5] [localport: 23] at 14:56:02 UTC Wed Feb 24 2021
 *Feb 24 15:00:25.568: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te1/1/2 removed
 *Feb 24 15:00:54.982: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/2
 *Feb 24 15:02:52.913: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te1/1/3 removed
 *Feb 24 15:04:47.672: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/3Warning: [1 51] is dup of [3 50]
 *Feb 24 15:06:07.071: %SYS-5-CONFIG_I: Configured from console by pete on vty0 (192.168.1.5)
 *Feb 24 15:11:22.731: %SYS-5-CONFIG_I: Configured from console by pete on vty0 (192.168.1.5)
 *Feb 24 15:12:05.166: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te2/1/2 removed
 *Feb 24 15:12:32.313: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te2/1/2Warning: [2 50] is dup of [3 52]
 *Feb 24 15:14:09.249: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te1/1/2 removed
 *Feb 24 15:14:16.391: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/2Warning: [1 50] is dup of [3 51]
 *Feb 24 15:19:14.899: %PLATFORM_PM-6-MODULE_REMOVED: SFP module with interface name Te1/1/1 removed
 *Feb 24 15:19:24.716: %PLATFORM_PM-6-MODULE_INSERTED: SFP module inserted with interface name Te1/1/1Warning: [1 49] is dup of [3 49]
 SWITCH-NAME(config-if-range)#do sh clock
 *15:33:51.262 UTC Wed Feb 24 2021
 SWITCH-NAME(config-if-range)#shut
 SWITCH-NAME(config-if-range)#no shut
 SWITCH-NAME(config-if-range)#
 *Feb 24 15:34:28.246: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/1/1, changed state to up
 *Feb 24 15:34:28.259: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/1/2, changed state to up
 SWITCH-NAME(config-if-range)#
 *Feb 24 15:34:31.578: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/1/1, changed state to up
 *Feb 24 15:34:31.757: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/1/2, changed state to up
 SWITCH-NAME(config-if-range)#exi
 SWITCH-NAME(config)#int te 2/1/2
 SWITCH-NAME(config-if)#shut
 SWITCH-NAME(config-if)#no shut
 SWITCH-NAME(config-if)#
 *Feb 24 15:34:54.208: %LINK-3-UPDOWN: Interface TenGigabitEthernet2/1/2, changed state to up
 SWITCH-NAME(config-if)#
 *Feb 24 15:34:57.425: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1/2, changed state to up