Use EEM Part 1 – Syslog and EEM

The following example shows a basic EEM Script to model your own network management.
But first let’s ask ourselves the question “Why change anything to network management from a device point-of-view?”

For network management most companies rely on the built-in SYSLOG and SNMP daemons to tell network management programs that something is wrong or something worth checking. But all of these companies have tiered helpdesks to manage all triggered actions from the network through IT operations. Mostly these Helpdesk’ers have never touched a router or switch and will just interpret (or at least try to) a message triggered from syslog, snmp or any other triggering mechanism. The following example case shows how to alter certain messages. Discussions of using SNMP or Syslog or Syslog converted to SNMP for active or pro-active monitoring is not included in this post.

Interface x on router INTERNET-SW1 goes down. The message that will be send through SYSLOG will be similar to “Interface FastEthernet0/0, changed state to down”
The helpdesk guy or girl sees the message and thinks “oeh oeh, I know that one! An interface went down!”. But does the helpdesk’er also know what will happen after the message has been triggered and what kind of impact it will have on the business. This interface will definitely not be the only interface around in the network.

Let’s say this interface is an uplink to ISP1 router. The company has a dual homes ISP connection to ISP1 and ISP2. The first and active connection has a line speed of 40 Mb, the second has a line speed of 10 Mb. Just for cutting the costs in this economy 🙂 When the active link failed the internet connection will fall back to 10Mb and the employees will notice the downgrade and people will start calling the helpdesk.
We can change the caption and the severity of the message to represent a clearer message for the helpdesk so they can instruct both the tier-2 engineers as well as the customers or employees.

!
event manager applet IF_DOWN 
 event syslog pattern "Interface FastEthernet0/0, changed state to down"
 action 2.0 syslog priority critical msg "Interface to ISP1 on node INTERNET_SW1 went down. Internet experience will decrease in speed until interface is restored. Please notify a TIER-2 engineer!"
!

Basically what the script does is it will pick up a syslog message description and will trigger action 2.0. Action 2.0 will send an additional syslog message with a clearer message and an alternate severity. Normally interface down will trigger severity 5 (Notification), but in this case we want it to be a severity 2.

the following syslog messages will appear:

*Mar  1 00:49:26.615: %LINK-5-CHANGED: Interface FastEthernet0/0, changed state to administratively down
*Mar  1 00:49:27.615: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0, changed state to down
*Mar  1 00:49:27.615: %HA_EM-2-LOG: IF_DOWN: Interface to ISP1 on node INTERNET_SW1 went down. Internet experience will decrease in speed until interface is restored. Please notify a TIER-2 engineer!

Here is how to check how many times the script was triggered:

INTERNET_SW1#show event manager history events 
No.  Time of Event             Event Type          Name
1    Fri Mar 1  00:10:38 2002  syslog              applet: IF_DOWN 
2    Fri Mar 1  00:40:45 2002  syslog              applet: IF_DOWN 
3    Fri Mar 1  00:42:31 2002  syslog              applet: IF_DOWN 
4    Fri Mar 1  00:49:27 2002  syslog              applet: IF_DOWN 
5    Fri Mar 1  01:01:05 2002  syslog              applet: IF_DOWN 
INTERNET_SW1#

 

The second trick shows you how to use a backup Layer-2 link without the use of routing protocol. The backup link (known as SECONDARY link) will take over when the PRIMARY link fails and when the failed PRIMARY comes back up again the SECONDARY will be shutdown again.
Please note that this is just to see what the basic options are with EEM scripting. For the sake of speed, convergence and stability this is not the way to go in a business whom relay on continuous operations.

This is the setup:
EEM_backup_link

We’ve applied the following basic configuration on R1 and R2:

R1 (Source)
!
interface FastEthernet1/0
 description *** PRIMARY LINK TO SOURCE ***
!
interface FastEthernet1/1
 description *** SECONDARY LINK TO SOURCE ***
!
interface Vlan1
 ip address 192.168.1.1 255.255.255.0
!

R2 (Destination)
!
interface FastEthernet1/0
 description *** PRIMARY LINK TO SOURCE ***
!
interface FastEthernet1/1
 description *** SECONDARY LINK TO SOURCE ***
 shutdown
!
interface Vlan1
 ip address 192.168.1.1 255.255.255.0
!

Now we apply the following two scripts:

!
event manager applet PRIMARY_DOWN
 event syslog pattern " Interface FastEthernet1/0, changed state to administratively down"

I’ve used the admin down pattern to trigger the action events. In real life this would be “changed state to down”

 action 1.0 syslog msg "PRIMARY LINK DOWN... Activating SECONDARY LINK"
 action 1.1 cli command "enable"
 action 1.2 cli command "conf t"
 action 1.3 cli command "int f1/1"
 action 1.4 cli command "no shut"
 action 1.5 cli command "end"
 action 1.6 cli command "clear mac-address-table interface FastEthernet1/0"

To speed things up I’ve included a specific MAC address flush on the old interface. For speed you don’t want to wait until the mac times out. When using admin down for the lab, this command has no use since the mac will be removed almost immediately

 action 2.0 syslog msg "SECONDARY LINK Activated!"

The following script will be used to de-activate the backup link in case the primary comes up again.

!
event manager applet PRIMARY_UP
 event syslog pattern " Interface FastEthernet1/0, changed state to up"
 action 1.0 syslog msg "PRIMARY LINK UP AGAIN… De-activating SECONDARY LINK"
 action 1.1 cli command "enable"
 action 1.2 cli command "conf t"
 action 1.3 cli command "int f1/1"
 action 1.4 cli command "shut"
 action 1.5 cli command "end"
 action 1.6 cli command "clear mac-address-table interface FastEthernet1/1"
 action 2.0 syslog msg "De-activating SECONDARY LINK"
!

You can also give a higher severity on the syslog messages if needed.

The following syslog will be display when F1/0 will be shut down.


R1#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R1(config)#int f1/0
R1(config-if)#shut
R1(config-if)#
R1(config-if)#
R1(config-if)#
R1(config-if)#
*Mar  1 02:14:39.683: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan1, changed state to down
*Mar  1 02:14:40.683: %LINK-5-CHANGED: Interface FastEthernet1/0, changed state to administratively down
*Mar  1 02:14:40.711: %HA_EM-6-LOG: PRIMARY_DOWN: PRIMARY LINK DOWN... Activating SECONDARY LINK
*Mar  1 02:14:40.919: %SYS-5-CONFIG_I: Configured from console by  on vty1 (EEM:PRIMARY_DOWN)
*Mar  1 02:14:40.943: %HA_EM-6-LOG: PRIMARY_DOWN: SECONDARY LINK Activated!
*Mar  1 02:14:41.683: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/0, changed state to down
*Mar  1 02:14:43.843: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/1, changed state to up
*Mar  1 02:15:11.895: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan1, changed state to up
R1(config-if)#
R1(config-if)#
R1(config-if)#
R1(config-if)#
R1(config-if)#no shut
R1(config-if)#
R1(config-if)#
R1(config-if)#
*Mar  1 02:15:41.263: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/0, changed state to up
*Mar  1 02:15:41.291: %HA_EM-6-LOG: PRIMARY_UP: PRIMARY LINK UP AGAINb& De-activating SECONDARY LINK
*Mar  1 02:15:41.407: %SYS-5-CONFIG_I: Configured from console by  on vty1 (EEM:PRIMARY_UP)
*Mar  1 02:15:41.423: %HA_EM-6-LOG: PRIMARY_UP: De-activating SECONDARY LINK
*Mar  1 02:15:43.355: %LINK-5-CHANGED: Interface FastEthernet1/1, changed state to administratively down
*Mar  1 02:15:44.355: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/1, changed state to down
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s