Introduction Scott Rife Senior Network Management Consultant Tavve

  • Slides: 55
Download presentation
Introduction • Scott Rife • Senior Network Management Consultant • Tavve Software Company/Onion Peel

Introduction • Scott Rife • Senior Network Management Consultant • Tavve Software Company/Onion Peel Solutions - Booth 300 – – – PRM - Performance reporting tool using NNM and http/html. Event. Watch - Active root cause correlation with NNM. Amerigo - Mapping tool for NNM with active distribution. Ego - Event. Watch/Amerigo integration - root cause submap popup. Event. Reach - Event. Watch/PRM on WAP enabled phones. • Develop and Improve processes to effectively manage business applications and networks. • Scott. Rife@tavve. com for a copy of this presentation.

Overview • NNM databases – – Map Topology Object Event • Tools - Unix

Overview • NNM databases – – Map Topology Object Event • Tools - Unix based only (ksh, awk, sort) • NNM utilities and formatting • Event configuration scripts – OV daemon dies – full filesystem • Status and configuration polling (netmon) • Database cleaning

NNM Relationships • Maps – 1 Map to N Submaps • Submaps – 1

NNM Relationships • Maps – 1 Map to N Submaps • Submaps – 1 Submap to N Symbols • Symbols – N Symbols to 1 Object • Objects – Unique objects for each: • • • device interface submap application (ip map, Amerigo) symbol

NNM Databases • Object – Every symbol, every submap has an underlying object. •

NNM Databases • Object – Every symbol, every submap has an underlying object. • Topology – Information necessary to characterize network topology. – Information about status changes. • Map – Submaps – Symbols – Display parameters/settings within ovw interface • Events – traps become events in the NNM system – Events displayed in Alarm browser – Stored in database and, optionally, trapd. log

Tools • • ksh - sh/bash awk sort ovobjprint ovtopodump ovmapdump ovdumpevents

Tools • • ksh - sh/bash awk sort ovobjprint ovtopodump ovmapdump ovdumpevents

ksh • • • #!/bin/ksh myprog parm 1 parm 2 $#, $1, $2…, shift

ksh • • • #!/bin/ksh myprog parm 1 parm 2 $#, $1, $2…, shift VAR=Value, let VAR=Numeric Value echo ${VAR} $VAR [ “$VAR” -lt “VALUE” ] [[ “$VAR” = “PATTERN ]] #!/bin/ksh VAR=“Hello World” if [ -n “$VAR” ]; then echo $VAR contains the string ”$VAR” fi • $VAR contains “Hello World”

awk • • • awk [ -F fs ] ‘COND{command}’ [fn] command | awk.

awk • • • awk [ -F fs ] ‘COND{command}’ [fn] command | awk. . . fs = . or , or : typically. Default is “ “ <SPACE> COND = /regexp/ or NF=5 or $3==string command – print $1 – printf “string is %-25 sn”, $0 – if ( $1 == “string” ) { print substr($1, 1, 4); VAR=1 }; print $3 • /usr/xpg 4/bin/awk on Solaris

sort (and uniq) • sort [-bcdfi. Mmnru] [-o output] [-T directory] [-ykmem] [-t char]

sort (and uniq) • sort [-bcdfi. Mmnru] [-o output] [-T directory] [-ykmem] [-t char] [+pos 1 [-pos 2]] [-k field_start[type][, field_end[type]] [file. . . ] • ovobjprint -a “SNMPAgent” | sort -u # sort and then unique on whole line. “How many different SNMP Agents in ovw databases? ” • sort -k 3 n, 3 -u # sort on column three only (numeric) and then unique on the column • sort -n +2 | uniq # same as above • sort -k 3 n, 3 -k 1, 2 -u # sort on column three (numeric), then columns 1 and 2, then unique on those columns

ovobjprint • Usage: ovobjprint [-S] -or-: ovobjprint [-T] -or-: ovobjprint [-a <field name> [<attribute>=value]]

ovobjprint • Usage: ovobjprint [-S] -or-: ovobjprint [-T] -or-: ovobjprint [-a <field name> [<attribute>=value]] -or-: ovobjprint [-o <object id>] -or-: ovobjprint [-f [<field id>]] -or-: ovobjprint [-s [<selection name>]] -or-: ovobjprint [-e [<field id>]]

ovobjprint Output (Node) OBJECT: 3017 FIELD ID FIELD NAME FIELD VALUE 10 Selection Name

ovobjprint Output (Node) OBJECT: 3017 FIELD ID FIELD NAME FIELD VALUE 10 Selection Name ”test_gw 1" 11 IP Hostname ”test_gw 1" 14 OVW Maps Exists 1 15 OVW Maps Managed 1 17 vendor Unset(0) 27 is. Node TRUE 29 is. Computer TRUE 30 is. Connector FALSE 31 is. Bridge FALSE 32 is. Router FALSE 33 is. Hub FALSE 52 is. IP TRUE 517 IP Status Normal(2) 521 is. IPRouter FALSE 561 is. SNMPSupported FALSE 568 SNMPAgent Unset(0) 580 Top. M Interface Count 1 587 Top. M Interface List "lan 0 Normal iso 88023 Csmacd " 588 is. Mc. Cluster. Member FALSE 589 is. Collection. Station. Node FALSE 2 10. 1. 2. 3 255. 0 <None>

ovobjprint Output (Interface) OBJECT: 3018 FIELD ID 10 14 15 43 44 52 53

ovobjprint Output (Interface) OBJECT: 3018 FIELD ID 10 14 15 43 44 52 53 514 516 517 569 571 573 582 583 584 FIELD NAME FIELD VALUE Selection Name ”test_gw 1: lan 0: 10. 1. 2. 3" OVW Maps Exists 1 OVW Maps Managed 1 is. Card TRUE is. Interface TRUE is. IPX FALSE IP Address ” 10. 1. 2. 3" IP Subnet Mask "255. 0" IP Status Normal(2) SNMP if. Type IEEE 802. 3 CSMACD(7) SNMP if. Descr "lan 0" SNMP if. Index 0 Top. M Network ID 3019 Top. M Segment ID 3020 Top. M Node ID 3017

ovobjprint Improved • ovobjprint | awk ‘ /OBJECT/{print OBJVAR, IPAVAR, ISRVAR; OBJVAR=“NULL”; IPAVAR=“NULL”; ISRVAR=“NULL”;

ovobjprint Improved • ovobjprint | awk ‘ /OBJECT/{print OBJVAR, IPAVAR, ISRVAR; OBJVAR=“NULL”; IPAVAR=“NULL”; ISRVAR=“NULL”; OBJVAR=$2; } /IP Address/{IPAVAR=$3} /is. Router/{ISRVAR=$3} END {print OBJVAR, IPAVAR, ISRVAR}} ‘ | tee /tmp/ttt • 571 “ 10. 1. 2. 3” TRUE 1571 “ 192. 168. 112. 245” FALSE

ovobjprint Improved Revisited • ovobjprint | awk ‘ /OBJECT/{if ( ISRVAR == “TRUE” )

ovobjprint Improved Revisited • ovobjprint | awk ‘ /OBJECT/{if ( ISRVAR == “TRUE” ) print OBJVAR, IPAVAR, ISRVAR; OBJVAR=“NULL”; IPAVAR=“NULL”; ISRVAR=“NULL”; OBJVAR=$2; } /IP Address/{gsub(/”/, ””); IPAVAR=$3} /is. Router/{ISRVAR=$3} END {if ( ISRVAR == “TRUE” ) print OBJVAR, IPAVAR, ISRVAR}} ‘ | tee /tmp/ttt • 571 10. 1. 2. 3 TRUE 1571 192. 168. 112. 245 FALSE

ovobjprint Improved Revisited • ovobjprint | awk ‘ /OBJECT/{if ( ISRVAR == “TRUE” )

ovobjprint Improved Revisited • ovobjprint | awk ‘ /OBJECT/{if ( ISRVAR == “TRUE” ) printf “%6 d %-16 s %-sn”, OBJVAR, IPAVAR, ISRVAR; OBJVAR=“NULL”; IPAVAR=“NULL”; ISRVAR=“NULL”; OBJVAR=$2} /IP Address/{gsub(/”/, ””); IPAVAR=$3} /is. Router/{ISRVAR=$3} END {if ( ISRVAR == “TRUE” ) printf “%-6 d %16 s %sn”, OBJVAR, IPAVAR, ISRVAR} ‘ | tee /tmp/ttt • 571 10. 1. 2. 3 TRUE 1571 192. 168. 112. 245 FALSE

ovobjprint Improved Continued • for OBJ in `awk ‘{print $1}’ /tmp/ttt` do ovobjprint -o

ovobjprint Improved Continued • for OBJ in `awk ‘{print $1}’ /tmp/ttt` do ovobjprint -o $OBJ done • ovobjprint -a is. SNMPSupported – 571 1 • ovobjprint -a “Selection Name” is. SNMPSupported=TRUE – 571 “test_gw 1” • ovobjprint -a “IP Address” – 571 “ 10. 254. 10. 1” • ovobjprint -a “IP Address” | sed ‘s/”//g’ • ovobjprint -a “IP Address” | awk ‘{gsub(/”/, “”); print $0}’

ovtopodump • Usage: ovtopodump [-Cd. Il. Lo. Ors. Sv] [-f filt] [-c host] [-T

ovtopodump • Usage: ovtopodump [-Cd. Il. Lo. Ors. Sv] [-f filt] [-c host] [-T [-p port]] [objnm. . ] -c host Connect to remote collection station (via SNMP). -C Print collection station for objects. -d Dump SNMP packets (-c option only). -f filt Print objects that pass the filter. -I Print invisible versions of objects. -l Long version of output. -L Include Link Address for interfaces. -o Include SNMP sys. Object. ID for nodes. -O Use old-style output format. -p port Remote port (default is 2532) (TCP only).

ovtopodump Continued … -r Recursive printing of contained objects. -s Include server information in

ovtopodump Continued … -r Recursive printing of contained objects. -s Include server information in global info. -S Print secondary versions of objects (implies -C). -T Use TCP connection to remote station. -v Verbose, same as -l. • ovtopodump -RISC

ovtopodump Output • OBJECT ID(S) OBJECT STATUS IP ADDRESS STATIONS: 1015 - nms-mars 1

ovtopodump Output • OBJECT ID(S) OBJECT STATUS IP ADDRESS STATIONS: 1015 - nms-mars 1 Normal NETWORKS: 3019 IP test. net Normal 10. 157. 138. 0. . . SEGMENTS: 3020 - test. net. Segment 1 Normal. . . NODES: 3017 IP test_gw 1 Normal 10. 157. 138. 1 3017/3018 IP test_gw 1 Normal 10. 157. 138. 1. . .

ovtopodump -l Output TOPO OBJECT ID: 1014 TOPO OBJECT UUID: f 197 a 394

ovtopodump -l Output TOPO OBJECT ID: 1014 TOPO OBJECT UUID: f 197 a 394 -0 cdd-71 d 5 -0 bc 20 a 0101010000 TIME CREATED: Tue Feb 27 13: 25: 45 2001 TIME MODIFIED: Thu Mar 01 16: 31: 39 2001 SYMBOL CHANGE TIME: Tue Feb 27 13: 25: 45 2001 LABEL CHANGE TIME: Tue Feb 27 13: 25: 45 2001 STATUS CHANGE TIME: Tue Feb 27 13: 25: 45 2001 FLAGS CHANGE TIME: Tue Feb 27 13: 25: 45 2001 GLOBAL FLAGS: CASE_INSENSITIVE_HOSTNAMES NUMBER OF STATIONS: 1 NUMBER OF NETWORKS: 5 NUMBER OF SEGMENTS: 5. . .

ovtopodump -l Output NUMBER OF NODES: 6 NUMBER OF INTERFACES: 79 NUMBER OF GATEWAYS:

ovtopodump -l Output NUMBER OF NODES: 6 NUMBER OF INTERFACES: 79 NUMBER OF GATEWAYS: 1 NUMBER OF CONNECTORS: 1 NUMBER OF MANAGED NODES: 6 NUMBER OF LICENSED NODES: 250 NUMBER OF LICENSED CONNECTORS: 250 LICENSE EXPIRATION DATE: Sat Apr 28 14: 25: 45 2001 EVENT SEQUENCE NUMBER: 0

ovtopodump -l www-server-com Output HOSTNAME: www-server-com NODE ID: 3037 NODE UUID: f 6 d

ovtopodump -l www-server-com Output HOSTNAME: www-server-com NODE ID: 3037 NODE UUID: f 6 d 931 aa-0 e 87 -71 d 5 -1438 -cd 9 d 8 af 00000 NODE STATION: local (id = 0) NODE REMOTE ID: 3037 CREATE TIME: Thu Mar 01 16: 15: 20 2001 MODIFIED TIME: Fri Apr 06 12: 18: 02 2001 SYMBOL CHANGE TIME: Thu Mar 01 16: 15: 20 2001 LABEL CHANGE TIME: Fri Mar 02 16: 28: 45 2001 STATUS: Normal LAST STATUS CHANGE: Fri Apr 06 12: 18: 02 2001 REMOTE STATUS: Normal. . .

ovtopodump -l www-server-com Output Cont HOSTNAME: www-server-com … LAST FLAGS CHANGE: Thu Mar 01

ovtopodump -l www-server-com Output Cont HOSTNAME: www-server-com … LAST FLAGS CHANGE: Thu Mar 01 16: 15: 20 2001 DESCRIPTION: LOCATION: CONTACT: SNMP SYSNAME: IPX SERVER NAME: SNMP OBJECT ID: DOES IP FORWARDING: NO SUPPORTS SNMP: NO NETMON FLAGS: 0 x 0 SNMP ADDRESS: 209. 238. 200. 102. . .

ovtopodump -l www-server-com Output Cont HOSTNAME: www-server-com … CYCLE TIME: 5 minutes ROOT GROUP/PORT:

ovtopodump -l www-server-com Output Cont HOSTNAME: www-server-com … CYCLE TIME: 5 minutes ROOT GROUP/PORT: <unset> NUMBER OF INTERFACES: 1 INTERFACE DESCRIPTION: lan 0 INTERFACE ID: 3038 INTERFACE UUID: f 6 d 99 ee 2 -0 e 87 -71 d 5 -1438 -cd 9 d 8 af 00000 INTERFACE STATION: local (id = 0) INTERFACE REMOTE ID: 3038 INTERFACE CRITICAL INTERFACE UUID: 00000000 -0000 -000000. . .

ovtopodump -l www-server-com Output Cont HOSTNAME: www-server-com … CREATE TIME: Thu Mar 01 16:

ovtopodump -l www-server-com Output Cont HOSTNAME: www-server-com … CREATE TIME: Thu Mar 01 16: 15: 20 2001 MODIFIED TIME: Fri Apr 06 12: 18: 02 2001 SYMBOL CHANGE TIME: Thu Mar 01 16: 15: 20 2001 SEGMENT CHANGE TIME: Thu Mar 01 16: 15: 20 2001 LABEL CHANGE TIME: Thu Mar 01 16: 15: 20 2001 STATUS: Normal LAST STATUS CHANGE: Fri Apr 06 12: 18: 02 2001 REMOTE STATUS: Normal IF FLAGS: CONNECTED LAST FLAGS CHANGE: Thu Mar 01 16: 15: 20 2001 IP ADDR: 209. 238. 200. 102. . .

ovtopodump -l www-server-com Output Cont HOSTNAME: www-server-com … IP MASK: 255. 0 IF NUMBER:

ovtopodump -l www-server-com Output Cont HOSTNAME: www-server-com … IP MASK: 255. 0 IF NUMBER: 0 IF TYPE: iso 88023 Csmacd PHYSICAL ADDRESS: <none> NODE/SEG/NET IDS: 3037/3040/3039 REMOTE NODE/SEG/NET IDS: 3037/3040/3039 IF SEG TYPE: UNKNOWN_SEG_TYPE NETMON FLAGS: 0 x 0 LLA SOURCE: none. MASK FROM: Unknown source PORT CLASS: single port 0

ovtopodump -l www-server-com Improved • ovtopodump -l www-server-com | awk -F: ‘ /HOSTNAME/{HNVAR=$2; }

ovtopodump -l www-server-com Improved • ovtopodump -l www-server-com | awk -F: ‘ /HOSTNAME/{HNVAR=$2; } /LAST STATUS CHANGE/{LSCVAR= sprintf(“%s: %s”, $2, $3, $4} /t. NODE/SEG/NET/{split($2, OBJARRAY, ///); NODEVAR=OBJARRAY[1]; SEGVAR=OBJARRAY[2]; NETVAR=OBJARRAY[3]} END {print HNVAR, LSCVAR, NODEVAR, SEGVAR, NETVAR} ‘ | tee /tmp/ttt • www-server-com Fri Apr 06 16: 41: 31 2001 3037 3040 3039

ovmapdump • Usage: ovmapdump [-v] [-m map] [-s snap. ID | -S snap_name] -or-

ovmapdump • Usage: ovmapdump [-v] [-m map] [-s snap. ID | -S snap_name] -or- ovmapdump [-lv | -L] -l List maps available. -L List maps available long form ( same as -lv ). -m map Print map instead of default map. -s snap. ID Print snapshot by identifier. -S snap_name Print snapshot by name. -v verbose printing. Default is to print default, current map in short form.

ovmapdump Output • ELEMENT MAP NAME PERMISIONS CREATION TIME Map default Read/Write Tue Feb

ovmapdump Output • ELEMENT MAP NAME PERMISIONS CREATION TIME Map default Read/Write Tue Feb 27 13: 38: 22 2001 ELEMENT ID NAME POLICY PARENT LAYOUT Submap 43 test 1. Segment 1 Shared 3046 Bus Submap 42 IP test 1 Shared 3045 Point-to-Point ELEMENT ID OBJECT STATUS COMPOUND STATUS Object 3018 Normal Unknown Object 3022 Normal Unknown ELEMENT ID LABEL OBJECT SUBMAP VARIETY Symbol 44 connection 3018 25 Connection Symbol 43 lan 0 3018 26 Icon

ovmapdump -l Output • GLOBAL MAP INFORMATION: Map Name: default Map Snapshot Directory: current

ovmapdump -l Output • GLOBAL MAP INFORMATION: Map Name: default Map Snapshot Directory: current Map Snapshot Name: Map User Access Permissions: R/W Map Comments: Map State: Layout. On Show. User. Plane Map Creation Time: Tue Feb 27 13: 38: 22 2001 Map Last Closed Time: Fri Mar 02 16: 28: 45 2001 Map Root Submap ID: 1 Map Home Submap ID: 1 Map Compound Status Scheme: Default. . .

ovmapdump -l Output Continued • GLOBAL MAP INFORMATION: . . . Map Compound Status

ovmapdump -l Output Continued • GLOBAL MAP INFORMATION: . . . Map Compound Status Warning Threshold: 0 Map Compound Status Marginal Threshold: 0 Map Compound Status Major Threshold: 0 Map Compound Status Critical Threshold: 0 Map Last Symbol ID: 176 Map Last Submap ID: 49 Map Application List: Application: IP Map Application Object Id: 1016

ovmapdump -l Output Continued • SUBMAP INFORMATION: Submap ID: 43 Submap State: Layout. On

ovmapdump -l Output Continued • SUBMAP INFORMATION: Submap ID: 43 Submap State: Layout. On Show. User. Plane New. Conns Overlay. On Submap Name: optimsys. Segment 1 Submap Comments: Submap Background Graphics file: Submap Creating Application: IP Map Submap Parent Object ID: 3046 Submap Parent Submap ID: 42 Submap Policy: Shared Submap Type: 4 Submap Layout Style: Bus. . .

ovmapdump -l Output Continued • SUBMAP INFORMATION: … Submap Width: 625 Submap Height: 112

ovmapdump -l Output Continued • SUBMAP INFORMATION: … Submap Width: 625 Submap Height: 112 Submap Zoom Factor: 0. 000000 Submap Base Zoom Width: 0 Submap Base Zoom Height: 0 Submap Window Width: 600 Submap Window Height: 500 Submap Window X Coordinate: -1 Submap Window Y Coordinate: -1 Submap Bus Upper X: 25 Submap Bus Upper Y: 75. . .

ovmapdump -l Output Continued • SUBMAP INFORMATION: … Submap Bus Row Width: 550 Submap

ovmapdump -l Output Continued • SUBMAP INFORMATION: … Submap Bus Row Width: 550 Submap Bus Row Height: 112 Submap Bus Num Rows: 1 Submap Bus Last X: 175 Submap Circle Radius: 0 Submap Circle Center X: 0 Submap Circle Center Y: 0 Submap Drop App Name: Submap Drop action:

ovmapdump -l Output Continued • Object ID: 3036 Object State: Object Comments: Object Status:

ovmapdump -l Output Continued • Object ID: 3036 Object State: Object Comments: Object Status: Unknown Object Compound Status: Normal

ovmapdump -l Output Continued • SYMBOL INFORMATION: Symbol ID: 44 Symbol State: No. Label.

ovmapdump -l Output Continued • SYMBOL INFORMATION: Symbol ID: 44 Symbol State: No. Label. Display Symbol Label: connection Symbol Text Annotation: Symbol Object ID: 3018 Symbol Submap ID: 25 Symbol Variety: Connection Symbol Type: Connection: Generic Symbol Plane: Application Symbol Status: Normal Symbol Status Source: Object Symbol Placement Type: None. . .

ovmapdump -l Output Continued • SYMBOL INFORMATION: . . . Symbol X Coord: 0

ovmapdump -l Output Continued • SYMBOL INFORMATION: . . . Symbol X Coord: 0 Symbol Y Coord: 0 Symbol Predecessor Symbol ID: 0 Symbol Connection Endpoint 1 Symbol ID: 42 Symbol Connection Endpoint 2 Symbol ID: 0 Symbol Applications: 1 IP Map Symbol Behavior: Explodable Symbol Executable Application: Symbol Executable Action: Symbol Executable Target Policy: 0 …

ovmapdump -l Output Continued • SYMBOL INFORMATION: . . . Symbol Executable Targets: Symbol

ovmapdump -l Output Continued • SYMBOL INFORMATION: . . . Symbol Executable Targets: Symbol Secondary Status: 0 Symbol % Full: 0 Drop App Name: Drop action:

ovdumpevents Output • usage: ovdumpevents [-f <filename>] [-c | -s <stream. Name>] [-d <database

ovdumpevents Output • usage: ovdumpevents [-f <filename>] [-c | -s <stream. Name>] [-d <database path. Name>] • EPOCH# DATE TIME NODE MSG_SRC MSG • • • 986589493 1 Fri Apr 06 16: 38: 13 2001 test 1. Segment 1 M Seg normal; 1 17. 1. 0. 50790401 3040 986589493 1 Fri Apr 06 16: 38: 13 2001 nms-mars 1 - Indicate correlation for be 753780 -2 acc-71 d 5 -0 b 3 a-cd 9 d 8 af 00000; 1 17. 1. 0. 58916879 0 986589493 1 Fri Apr 06 16: 38: 13 2001 nms-mars 1 - Acknowledge 0 bed 1998 -2 acc 71 d 5 -0 b 3 a-cd 9 d 8 af 00000 by ECS; 1 17. 1. 0. 58916876 0 986589493 1 Fri Apr 06 16: 38: 13 2001 www-server-com N If lan 0 status Normal (was Critical) station nms-mars 1; 1 17. 1. 0. 40000073 3037 986591394 4 Fri Apr 06 17: 09: 54 2001 nms-mars 1 - Interface nms-mars 1 down. ; 2. 1. 3. 6. 1. 4. 1. 2668. 1. 1. 6. 0. 1 0 986609616 6 Fri Apr 06 22: 13: 36 2001 nms-mars 1 M Network Node Manager license expires on Sat Apr 28 14: 25: 45 2001; 4 17. 1. 0. 59179228 0

Ovdumpevents - MSG Type A : Agent (SNMP trap received through UDP) C :

Ovdumpevents - MSG Type A : Agent (SNMP trap received through UDP) C : xnmcollect D : snmp. Collect (Data Collector) E : xnmevents F : ovtopofix I : ipmap J : ovalarmsrv L : xnmloadmib M : ovtopmd (map) N : netmon P : pmd R : ovrepld. . .

Ovdumpevents MSG Type Continued … T : ovtrapd a : generic application (#14, for

Ovdumpevents MSG Type Continued … T : ovtrapd a : generic application (#14, for user-sendable events) c : xnmtopoconf d : nmdemandpoll l : loadhosts m : netmon mask change n : xnmpolling o : ovactiond p : ovspmd / ovpause / ovresume r : remote pmd s : xnmsnmpconf t : xnmtrap

ovdumpevents Improved • ovdumpevents | awk ‘$8==“www-server-com”{print $0}’ • ovdumpevents |  awk ‘$8==“www-server-com”{print

ovdumpevents Improved • ovdumpevents | awk ‘$8==“www-server-com”{print $0}’ • ovdumpevents | awk ‘$8==“www-server-com”{print $0}’ | sort -k 8, 8 -k 1 n, 1 # Sort based on node then time (epoch)

Event Configuration • Options->Event Configuration Parameter Explanation Sources for this action Pop Up Notification

Event Configuration • Options->Event Configuration Parameter Explanation Sources for this action Pop Up Notification Automatic Actions

Event Configuration • • Automatic Actions field $OV_LOG/ovactiond. log Single threaded Background (&) or

Event Configuration • • Automatic Actions field $OV_LOG/ovactiond. log Single threaded Background (&) or not?

Email Script • #!/bin/ksh LOGFILE=/var/log/email. ksh. log if [ $# -lt 1 ]; then

Email Script • #!/bin/ksh LOGFILE=/var/log/email. ksh. log if [ $# -lt 1 ]; then echo ERROR: At least one recipient required echo Usage is: echo “ “ echo MSG | email. ksh recipient [recipients] fi MSG=`line -t 1` if [ -n “$MSG” ]; then echo “`date`: $MSG sent $*” echo $MSG | mail $* else echo ERROR: No message provided. fi

Filesystem Cleanup • This process requires that the ovw user has a $HOME/. rhosts

Filesystem Cleanup • This process requires that the ovw user has a $HOME/. rhosts on the error producing system. • If $HOME/. rhosts is not an option then we can set up an email script. That is another discussion completely. • Options->Data collecting and thresholding->snmp->disk%util • Configure threshold event • Automatic action: • #!/bin/ksh find $1 -name core -exec rm -f {} ; find /home/*/. netscape/cache -mtime -15 -exec rm -f {} ; find $OV_SHARE_HTDOCS -mtime -45 -exec rm -f {} ;

OV Daemon’s died • All “well behaved” applications register with ovspmd when started with

OV Daemon’s died • All “well behaved” applications register with ovspmd when started with ovstart. This means that ovspmd knows when they die and will generate an event (assuming that pmd is up…). • OV_App. Ungraceful. Exit. 1. 3. 6. 1. 4. 1. 11. 2. 17. 1. 0. 59179058 • Automatic action: – /opt/OV/bin/ovstart – This will be logged in the event database. Accessible with ovdumpevents.

Netmon measures • netmon -M 1 2 4 8 16 32 64 128 Trace

Netmon measures • netmon -M 1 2 4 8 16 32 64 128 Trace ICMP Echo requests. Trace ICMP Echo replies and timeouts. Trace SNMP requests. Trace SNMP replies and timeouts. Trace traps generated. Trace traps received. Trace main loop timeouts. Trace filtering activities. • netmon -a (netmon -a ? ) 0 1 2 3 4 dump all nodes and interfaces (actions 3, 4, 1, 2) dump ifaces long (actions 21, 22, 23, 14, 28) dump nodes long (actions 25, 26, 27) dump ifaces short (actions 11, 12, 13, 14, 18) dump nodes short (actions 15, 16, 17)

Netmon measures Continued • netmon -a continued 5 6 7 8 10 11 12

Netmon measures Continued • netmon -a continued 5 6 7 8 10 11 12 13 14 15 16 17 18 20 21 dump sizes of ping and SNMP lists dump interface search lists dump oid_to_type tree dump crit. Route. Seed. List dump check. Crit. Route. List (short) dump ping. Wait & ping. Down. Wait (short) dump ping. List (short) dump paused. Ping. List (short) dump region iface lists (short) dump snmp. Wait (short) dump snmp. List (short) dump un. Snmp. List (short) dump un. Ping. List (short) dump check. Crit. Route. List (long) dump ping. Wait & ping. Down. Wait (long)

Netmon measures Continued • netmon -a continued 22 23 25 26 27 28 29

Netmon measures Continued • netmon -a continued 22 23 25 26 27 28 29 51 52 53 100 101 102 103 104 dump ping. List (long) dump paused. Ping. List (long) dump snmp. Wait (long) dump snmp. List (long) dump un. Snmp. List (long) dump un. Ping. List (long) dump Loaded Topology Info (short) dump http. Pre (long) dump http. List (long) dump http. Wait (long) dump interface search list - in a Topo only and oids. (long) dump ping. List - in a Topo only and oids. (long) dump paused. Ping. List - in a Topo only and oids. (long) dump ping. Wait and ping. Down - in a Topo only and oids. (long) dump all ping lists - in a Topo only and oids. (long)

Netmon measures Continued • snmpget – – – . 1. 3. 6. 1. 4.

Netmon measures Continued • snmpget – – – . 1. 3. 6. 1. 4. 1. 11. 2. 17. 4. 4. 1. 1. 2. 1. 3. 6. 1. 4. 1. 11. 2. 17. 4. 4. 1. 2. 1. 1. 3. 6. 1. 4. 1. 11. 2. 17. 4. 4. 1. 2. 2. 1. 3. 6. 1. 4. 1. 11. 2. 17. 4. 4. 1. 2. 3 Status poll list length Status polls in the next minute Status poll - Seconds until next SNMP poll list length SNMP polls in the next minute SNMP poll - Secons until next

Cron • Crontab -l > ~/crontab. date [where date=YEARMODY] vi ~/crontab. 20010402 crontab /crontab.

Cron • Crontab -l > ~/crontab. date [where date=YEARMODY] vi ~/crontab. 20010402 crontab /crontab. 20010402 • Format of file MINUTE HOUR DAY MONTH WEEKDAY 12 * * # 12 minutes after the hour every day 12 0 1 * * # 00: 12 on the first of the month 12 0 1 1 * # 00: 12 on the first of January 12 0 * * * 0 # 00: 12 on every Sunday

Clean Databases • Occasionally the OV Databases will get out of “sync”. This happens

Clean Databases • Occasionally the OV Databases will get out of “sync”. This happens more often when you delete objects. • I suggest running this once a week on Sunday morning – 20 2 * * 0 /usr/local/bin/synch_nnm • /usr/local/bin/synch_nnm #!/bin/ksh exec >> /tmp/$0. `date +%Y%m%d%H%M` exec 2>&1 echo $0 started on `date` ovstop netmon ovw -mapcount -r. Ru. D ovtopofix -chs touch /tmp/ovw. mapcount. new /tmp/ovtopofix. new let STOPVAR=0. . .

Clean Databases Continued while [ $STOPVAR -lt 2 ] do let STOPVAR=0 mv /tmp/ovw.

Clean Databases Continued while [ $STOPVAR -lt 2 ] do let STOPVAR=0 mv /tmp/ovw. new /tmp/ovw. hist mv /tmp/ovtopofix. new /tmp/ovtopofix. hist ovw -mapcount -ru. D > /tmp/ovw. new ovtopofix -chs > /tmp/ovtopofix. new if [ -z “`diff /tmp/ovw. new tmp/ovw. hist`” ]; then let STOPVAR=$STOPVAR+1 fi if [ -z “`diff /tmp/ovtopofix. new /tmp/ovtopofix. hist`” ]; then let STOPVAR=$STOPVAR+1 fi done ovstart netmon echo $0 finished on `date`

Questions? • What other topics does this raise for you? • What other topics

Questions? • What other topics does this raise for you? • What other topics would you like to see at the Forum? – – – Data collecting and thresholds Remote data collecting and thresholds Event configuration Scripting Reconfiguring SNMP community strings Utilities you’d like to see in NNM • backup script for inclusion in /opt/OV/contrib • send_email for inclusion in /opt/OV/contrib • Others?

Introduction • Scott Rife • Senior Network Management Consultant • Tavve Software Company/Onion Peel

Introduction • Scott Rife • Senior Network Management Consultant • Tavve Software Company/Onion Peel Solutions - Booth 300 – – – PRM Event. Watch Amerigo Event. Reach • Develop and Improve processes to effectively manage business applications and networks