Network Security

New Site for Posts

2009-09-17T20:49:00.002-04:00

Chesapeake Netcraftsmen has created a new website with a blogging section. I've moved my blog posts to that website. I just posted a blog on "Creating Policing with Cisco NCM".

Getting Started with Expect for Cisco NCM Command Scripts

2009-07-09T14:32:00.003-04:00

Advanced Command scripts, with NCM, can be written in Perl or Expect. When there is interactive input, Expect is normally recommended. This blog provides a starting point for using Expect to aid in the creation of Cisco NCM command scripts

1 Reference Material

One of the best references for Expect is the “Exploring Expect” book published by O’Reilly. I’ll be adding page number references from the book to the sections below.

Another good reference is the included help files with ActiveState Expect.

2 Downloading and Installing ActiveState TCL with Expect Extensions

One way to learn how to use Expect is to use the free version, from ActiveState, on your Windows PC. This provides the opportunity to learn how TCL and Expect work and test scripts before deploying within NCM. Use the following steps to download and install Expect

1. Go to http://www.activestate.com and download the free TCL application

2. Install TCL

3. Open a command prompt

4. Type “teacup install Expect” to install the Expect extension to TCL

3 Create Your First Script

Now that TCL and Expect are installed, you can run through the following test script to see how Expect works. This test script shows how to telnet to a router and execute “show version”. Within the example script, replace anything in red with the appropriate information from your test router.

There are minor differences between this script and the script executed within NCM. The one main difference is the use of the “exp_send” command. Within NCM, this should just be “send”.

1. Open Notepad

2. Paste the following script into Notepad and save as sample.txt. This script assumes that you have AAA turned on an authorization set to have the user authorized into privileged exec mode. If this configuration is not already entered, you can add it by entering the following to add AAA with local authentication and authorization

aaa new-model

aaa authentication login default local

aaa authorization exec default local

username cisco privilege 15 password cisco

Script

#!/bin/sh

# \

exec tclsh "$0" ${1+"$@"}

package require Expect

log_user 1

spawn telnet 192.168.137.100

expect "Username" {

exp_send "cisco\r"

}

expect "assword" {

exp_send "cisco\r"

}

expect "R1#"

exp_send "terminal length 0\r"

expect "R1#"

exp_send "sh version\r"

expect "R1#"

exp_send "exit\r"

3. Run the script by opening a command prompt, navigating to the directory with the script, and executing “tclsh85 sample.txt”. The output of the script should look similar to the following

C:\Tcl>tclsh85 showversion.txt

Welcome to Microsoft Telnet Client

Escape Character is 'CTRL+]'

Connecting To 192.168.137.100...

Put something here

User Access Verification

Username: cisco

Password:

R1#terminal length 0

R1#sh version

Cisco IOS Software, 3600 Software (C3640-JK9O3S-M), Version 12.4(13a), RELEASE S

OFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Compiled Tue 06-Mar-07 20:25 by prod_rel_team

ROM: ROMMON Emulation Microcode

ROM: 3600 Software (C3640-JK9O3S-M), Version 12.4(13a), RELEASE SOFTWARE (fc1)

R1 uptime is 5 minutes

System returned to ROM by unknown reload cause - suspect boot_data[BOOT_COUNT] 0

x0, BOOT_COUNT 0, BOOTDATA 19

System image file is "tftp://255.255.255.255/unknown"

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

Cisco 3640 (R4700) processor (revision 0xFF) with 124928K/6144K bytes of memory.

Processor board ID 00000000

R4700 CPU at 100MHz, Implementation 33, Rev 1.2

17 FastEthernet interfaces

DRAM configuration is 64 bits wide with parity enabled.

125K bytes of NVRAM.

8192K bytes of processor board System flash (Read/Write)

Configuration register is 0x2102

R1#

C:\Tcl>

4 Regular Expressions

Expect scripts normally include the use of regular expressions to evaluate output from routers and switches. The best way to show how this works is through a simple example. Let’s take the output from “show interface fa1/0”. Here’s the output to use as a reference for the example to follow.

R1#sh int fa1/10

FastEthernet1/10 is up, line protocol is down

Hardware is Fast Ethernet, address is cc00.1264.f10a (bia cc00.1264.f10a)

MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Auto-duplex, Auto-speed

ARP type: ARPA, ARP Timeout 04:00:00

Last input never, output never, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 0 bits/sec, 0 packets/sec

5 minute output rate 0 bits/sec, 0 packets/sec

0 packets input, 0 bytes, 0 no buffer

Received 0 broadcasts, 0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 input packets with dribble condition detected

0 packets output, 0 bytes, 0 underruns

0 output errors, 0 collisions, 2 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier

0 output buffer failures, 0 output buffers swapped out

With this script, we want to evaluate the up/down state of the interface and line protocol and exit out of the script with an error message if either the interface or line are down.

#!/bin/sh

# \

exec tclsh "$0" ${1+"$@"}

package require Expect

log_user 1

spawn telnet 192.168.137.100

expect "Username" {

exp_send "cisco\r"

}

expect "assword" {

exp_send "cisco\r"

}

expect "R1#" {

exp_send "sh int fa1/10\r"

}

expect {

-re "is (\[a-z]*), line protocol is (\[a-z]*)" {

set int_status $expect_out(1,string)

set line_status $expect_out(2,string)

exp_continue

}

"R1#" {

exp_send "exit\r"

}

if {$int_status == "down" || $line_status == "down"} {

puts "\n!!!Interface is down!!!\n"

exit 1

}

else {

puts "\n!!!Interface is up!!!\n"

exit 0

}

Let’s look at the regular expression portion of the script

expect {

-re "is (\[a-z]*), line protocol is (\[a-z]*)" {

set int_status $expect_out(1,string)

set line_status $expect_out(2,string)

exp_continue

}

"R1#" {

exp_send "exit\r"

}

The “-re” specifies that the next portion should be treated as a regular expression (pg 109 Exploring Expect). The portion [a-z] specifies a single character matching any lower case character. The “*” after the [a-z] specifies a match for the rest of the lower case characters that follow the initial character. You’ll notice that there is a \ before the first [. This is required, by Expect, to have the [ ] treated correctly as a regular expression delineator (pg 91 Exploring Expect). The () specifiy that anything within them should be treated as a variable. The variable is saved in the line after the regular expression with the line (pg 111 Exploring Expect).

set int_status $expect_out(1,string)

The “1” specifies that it is the first variable created by the (). The second () is captured as the variable “line_status”. You’ll notice that the “1” has been replaced by a “2” in this case (pg 111 Exploring Expect).

The next thing to notice is the “exp_continue” command. This specifies that the expect command that surrounds it should be run until there is no remaining input to be evaluated (pg 145 Exploring Expect).

Here is the output of the script

C:\Tcl>tclsh85 sample.tcl

Welcome to Microsoft Telnet Client

Escape Character is 'CTRL+]'

Connecting To 192.168.137.100...

Put something here

User Access Verification

Username: cisco

Password:

R1#sh int fa1/10

FastEthernet1/10 is up, line protocol is down

Hardware is Fast Ethernet, address is cc00.1264.f10a (bia cc00.1264.f10a)

MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Auto-duplex, Auto-speed

ARP type: ARPA, ARP Timeout 04:00:00

Last input never, output never, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 0 bits/sec, 0 packets/sec

5 minute output rate 0 bits/sec, 0 packets/sec

0 packets input, 0 bytes, 0 no buffer

Received 0 broadcasts, 0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 input packets with dribble condition detected

0 packets output, 0 bytes, 0 underruns

0 output errors, 0 collisions, 2 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier

0 output buffer failures, 0 output buffers swapped out

R1#

!!!Interface is down!!!

5 Limiting Script Output

In the first script, we saw that there was a lot of output. We could limit this output to specific information that we wanted to display. This is done by changing “log_user 1” to “log_user 0”(pg 175 Exploring Expect). Once this is done, specific output can be displayed using the “puts” command. Here is the script from the regular expression section that will only output the interface information

#!/bin/sh

# \

exec tclsh "$0" ${1+"$@"}

package require Expect

log_user 0

spawn telnet 192.168.137.100

expect "Username" {

exp_send "cisco\r"

}

expect "assword" {

exp_send "cisco\r"

}

expect "R1#" {

exp_send "sh int fa1/10\r"

}

expect {

-re "is (\[a-z]*), line protocol is (\[a-z]*)" {

set int_status $expect_out(1,string)

set line_status $expect_out(2,string)

exp_continue

}

"R1#" {

exp_send "exit\r"

}

if {$int_status == "down" || $line_status == "down"} {

puts "\n!!!Interface is down!!!\n"

exit 1

}

else {

puts "\n!!!Interface is up!!!\n"

exit 0

}

Here is the output that is displayed in this case

C:\Tcl>tclsh85 sample.tcl

!!!Interface is down!!!

Automation with Cisco NCM Command Scripts

2009-06-23T19:24:00.023-04:00

One of the strongest features of Cisco NCM is the capability to create scripts to be executed against any number of devices. These scripts can be as simple as running a sequence of Cisco IOS commands or as complex as a multi-page Expect script. In this blog I'll show how easy it is to create an Expect script, within NCM.

For this example, let's assume we're trying to figure out a way to allow the Network Operations Center (NOC) to shut down interfaces on access switches, but not on distribution or core switches. Additionally, the NOC should not be able to be able to shut down the uplink ports located on ports fastethernet 0/23 and fastethernet 0/24.

NCM provides an easy way to get started with the script. This is done by creating a SSH session to the switch and executing the commands that would be executed in the command script. Then, the commands can be viewed and automatically converted to an Expect or Perl script. From there, the script can be customized to provide the logic to accomplish the task above. Let's walk through how each step is done.

1. Click on "Devices > Inventory".

2. In the resulting screen, click on the SSH button next to the device you would like to connect to

3. In the Java SSH window that appears, type in an example of the commands that would be used to shutdown an interface and exit.

4. Navigate back to the device by, once again, going to "Devices > Inventory". Click on the device name

5. In the resulting screen, click on "View > Telnet/SSH Sessions"

6. Click on "View Commands Only" to view the commands that you just entered

7. Click on "Convert to Expect Script" to automatically create a script from the commands entered.

8. At this point, an Expect script is created with the code needed to execute the commands entered previously

9. There a few places that need customization. First, the interface used in the script, fastethernet0/9 should be a variable that the NOC can define at execution time. This can be done by replacing fastethernet0/9 with a NCM variable that the NOC will be prompted to enter when they execute the script. To do this, replace fastethernet0/9 with $interface$. A string with $ at the beginning and end signifies an NCM variable.

send "interface fa0/9\r" -----------> send "interface $interface$\r"

10. When this is added, the "Pull Variables" button can be clicked to create the prompt that the NOC will see when they execute the script

11. This brings up another screen that requires information to be entered for the prompt

12. That's the basics of the script. The only thing left to do is add the restrictions for the script. First, the NOC should only be able to change access layer switches. The naming convention for the switches state that access switches start with "A". We can use this as a check to make sure an access switch is being used. Below is the corresponding Expect code snippet

if [string match "^A*" $enable_prompt] {
} else {
puts "\nThis is not an access layer switch\n"
exit 1
}

This snippet checks to see if the pre-defined $enable_prompt variable starts with an A. If so, it is an access layer switch. If not, an error message is displayed and the script is exited with error status.

13. The second check was to make sure fastethernet0/23 or fastethernet0/24 are not used. This is accomplished with the snippet below.

set protected_int {"1/6" "1/7"}
set int $interface$

set i 0
foreach i $protected_int {
if [string match "*$i" $int] {
puts "\nShutdown of uplink ports is not permitted\n"
exit 1
}
}

In this portion, the uplink interfaces are put into an array named "protect_int". The interface, that the NOC chooses, is stored in the $interface$ variable. A for loop checks to see if there is match between an uplink interface and the chosen interface. If so, an error message is sent and the scripted is exited.

14. When the script is created, the Expect command "log_user 1" is set. This means that whatever is sent in the script is also sent to the script output. In order to stop this from happening, set "log_user 0". With this set, only the "puts" output is displayed. In general, this is what you will want to see.

15. To run the script, click "Devices > Device Tools > Command Scripts"

16. Select "Run" on the script to execute.

17. Select the devices to run the script on, the interface to shutdown, and click "Save Task" to execute the script

That's all there is to creating a command script in NCM. I would highly recommend purchasing the "Exploring Expect" book written by Don Libes and published by O'Reilly. Additionally, I would recommend downloading ActiveTCL from Activestate.com. Expect is actually an extension of TCL. After installing TCL, you can load the Expect extension by entering "teacup install Expect" from a CMD prompt.

Microsoft Swiss Army Toolkit

2009-06-20T21:31:00.003-04:00

Every once in awhile there are instances where you need information about a Microsoft Windows XP computer that you think should be easy to find but seems impossible to uncover. The tool to find the information is a built-in command line tool call wmic.exe. This command has a slew of command line options that can uncover practically any information about your computer. Here are some examples.

Want to find out the OS version of your computer? Run "wmic os". The output shows up in columns with a ton of information. The version information show up as shown below

C:\>wmic os
Version
5.1.2600

Here's a list of all the command line parameters available

C:\> wmic /?

[global switches]

The following global switches are available:
/NAMESPACE Path for the namespace the alias operate against.
/ROLE Path for the role containing the alias definitions.
/NODE Servers the alias will operate against.
/IMPLEVEL Client impersonation level.
/AUTHLEVEL Client authentication level.
/LOCALE Language id the client should use.
/PRIVILEGES Enable or disable all privileges.
/TRACE Outputs debugging information to stderr.
/RECORD Logs all input commands and output.
/INTERACTIVE Sets or resets the interactive mode.
/FAILFAST Sets or resets the FailFast mode.
/USER User to be used during the session.
/PASSWORD Password to be used for session login.
/OUTPUT Specifies the mode for output redirection.
/APPEND Specifies the mode for output redirection.
/AGGREGATE Sets or resets aggregate mode.
/AUTHORITY Specifies the for the connection.
/?[:] Usage information.

For more information on a specific global switch, type: switch-name /?

The following alias/es are available in the current role:
ALIAS - Access to the aliases available on the local system
BASEBOARD - Base board (also known as a motherboard or system board) management.
BIOS - Basic input/output services (BIOS) management.
BOOTCONFIG - Boot configuration management.
CDROM - CD-ROM management.
COMPUTERSYSTEM - Computer system management.
CPU - CPU management.
CSPRODUCT - Computer system product information from SMBIOS.
DATAFILE - DataFile Management.
DCOMAPP - DCOM Application management.
DESKTOP - User's Desktop management.
DESKTOPMONITOR - Desktop Monitor management.
DEVICEMEMORYADDRESS - Device memory addresses management.
DISKDRIVE - Physical disk drive management.
DISKQUOTA - Disk space usage for NTFS volumes.
DMACHANNEL - Direct memory access (DMA) channel management.
ENVIRONMENT - System environment settings management.
FSDIR - Filesystem directory entry management.
GROUP - Group account management.
IDECONTROLLER - IDE Controller management.
IRQ - Interrupt request line (IRQ) management.
JOB - Provides access to the jobs scheduled using the schedule service.
LOADORDER - Management of system services that define execution dependencies.
LOGICALDISK - Local storage device management.
LOGON - LOGON Sessions.
MEMCACHE - Cache memory management.
MEMLOGICAL - System memory management (configuration layout and availability of memory).
MEMPHYSICAL - Computer system's physical memory management.
NETCLIENT - Network Client management.
NETLOGIN - Network login information (of a particular user) management.
NETPROTOCOL - Protocols (and their network characteristics) management.
NETUSE - Active network connection management.
NIC - Network Interface Controller (NIC) management.
NICCONFIG - Network adapter management.
NTDOMAIN - NT Domain management.
NTEVENT - Entries in the NT Event Log.
NTEVENTLOG - NT eventlog file management.
ONBOARDDEVICE - Management of common adapter devices built into the motherboard (system board).
OS - Installed Operating System/s management.
PAGEFILE - Virtual memory file swapping management.
PAGEFILESET - Page file settings management.
PARTITION - Management of partitioned areas of a physical disk.
PORT - I/O port management.
PORTCONNECTOR - Physical connection ports management.
PRINTER - Printer device management.
PRINTERCONFIG - Printer device configuration management.
PRINTJOB - Print job management.
PROCESS - Process management.
PRODUCT - Installation package task management.
QFE - Quick Fix Engineering.
QUOTASETTING - Setting information for disk quotas on a volume.
RECOVEROS - Information that will be gathered from memory when the operating system fails.
REGISTRY - Computer system registry management.
SCSICONTROLLER - SCSI Controller management.
SERVER - Server information management.
SERVICE - Service application management.
SHARE - Shared resource management.
SOFTWAREELEMENT - Management of the elements of a software product installed on a system.
SOFTWAREFEATURE - Management of software product subsets of SoftwareElement.
SOUNDDEV - Sound Device management.
STARTUP - Management of commands that run automatically when users log onto the computer system.
SYSACCOUNT - System account management.
SYSDRIVER - Management of the system driver for a base service.
SYSTEMENCLOSURE - Physical system enclosure management.
SYSTEMSLOT - Management of physical connection points including ports, slots and peripherals, and proprietary connections points.
TAPEDRIVE - Tape drive management.
TEMPERATURE - Data management of a temperature sensor (electronic thermometer).
TIMEZONE - Time zone data management.
UPS - Uninterruptible power supply (UPS) management.
USERACCOUNT - User account management.
VOLTAGE - Voltage sensor (electronic voltmeter) data management.
VOLUMEQUOTASETTING - Associates the disk quota setting with a specific disk volume.
WMISET - WMI service operational parameters management.

For more information on a specific alias, type: alias /?

CLASS - Escapes to full WMI schema.
PATH - Escapes to full WMI object paths.
CONTEXT - Displays the state of all the global switches.
QUIT/EXIT - Exits the program.

For more information on CLASS/PATH/CONTEXT, type: (CLASS | PATH | CONTEXT) /?

Features of Cisco Network Compliance Manager (NCM)

2009-05-29T22:58:00.004-04:00

Cisco Network Compliance Manager (NCM) is a powerful tool that can ease network and security management by automating many of the more tedious aspects. This is all on top of the policies that can be used to ensure compliance with various compliance standards such as SOX, PCI, COBIT, ITIL, COSO, GLBA, and HIPAA. Here's a list of the top features that NCM can provide for an enterprise

Bare Metal Provisioning: This feature allows NCM to configure a device from scratch. The assumption is that the device console port is connected to a terminal server. NCM connects to the terminal server and discovers the device through the console port. It then pushes a config to the device and sets it up for use on the network
Device Configuration Template: This is a full configuration for a device that can be used as your "golden config". Every time a new device is configured, it can use this device configuration template as a baseline for the configuration. Unique portions, such as IP addresses, can be added through device variables that are defined at implementation time.
Command scripts: These are code snippets that can be run as a script. This is a great way to allow NOC personnel to safely execute commands without worrying about misconfigurations. The command scripts can even be forced to go through a workflow process. This could ensure that a higher level engineer reviews the command script before it is sent out.
Policies: These are checks that are done against the device snapshots that are periodically taken. This is one of the strongest features of NCM. There are number of different ways that policies can be used. One way is to check for stale configurations that should not exist. An example of this would be old SNMP server configurations that should be removed. The policy can also be configured to auto-remediate the problem and remove the stale configuration. To make this safer, the auto-remediation could be sent through a workflow for approval before it is actually implemented as a task. The second benefit of policies is standards based policy compliance checks. These would be policies, such as SOX and PCI. The third benefit of policies is automated checking of software vulnerabilities. This is provided with NCM Alert center. This is a subscription based service that is used to check for software vulnerabilities in Cisco devices. When a Cisco vulnerability announcement is released, Cisco creates an NCM policy to check for the software vulnerability. NCM downloads that policy, from Cisco, and uses it to check the devices it supports. If a vulnerability is found, it shows up in the police compliance report. The great thing about this is that the Cisco created policy checks for both the software version and the feature that causes the vulnerability. If the feature is not used, the device will not show up as vulnerable. This granularity ensures that only the devices truly vulnerable to a PSIRT are flagged.
Software Image Management (SWIM): NCM collects all the information that is needed to determine the software version that should be used on the devices. By using SWIM, downloading updated software images from Cisco is just a matter of a few mouse clicks. Deployment of the software images is also just a few mouse clicks.
Searching: The search functionality built into NCM is excellent. The searches are extremely flexible. When trying to search for a set of information about devices I usually find an extremely easy way of creating the search. Additionally, searches can be saved as a user report. This saves a lot of time. An initial search may take awhile to define which fields should show up and what information should be searched on. Once this is defined and saved as a user report, the information can be retrieved in a few mouse clicks.
Inventory for Cisco SW Maintenance: By using the search functionality, a comprehensive list of devices and serial numbers can be retrieved. This information can be used to define the devices that need to be covered under Cisco SW maintenance for the upcoming year.
Reporting: There are a number of great reports that NCM generates that show management level reports as well as detailed reports about the network environment
Diagrams: NCM can create L2, L3, L3 port, and other diagrams that show the network in a JPG, interactive JPG, or Visio format. You can also define which devices show up in the diagram to provide unique views showning the connectivity of different devices in the network.

I'll be providing further blogs, in the future, showing screenshots of the features listed above. Feel free to shoot me an email if there's a specific topic you wanted me to cover.

Tracking down high bandwidth users

2009-05-14T15:40:00.004-04:00

When your internet bandwidth is being maxed out, it's nice to have an easy way to determine which users are causing the problem. One way of doing this is by using Netflow statistics. One free toolset to collect Netflow data is called flow-tools.

If you're using ASA FWs, a quick and dirty way to check out the offending users is to use the built in statistics in Cisco ASDM. This is located on the firewall dashboard as shown below

On this dashboard is a "top usage status" box that shows the offending user IP addresses along with the percentage of bandwidth used. This is shown below

This data can be used as a starting point for tracking down the high bandwidth users on your network

SNMP testing with net-snmp

2009-05-03T18:33:00.006-04:00

With most network management systems and network security systems, SNMP is a critical component. One great tool for checking SNMP functionality is net-snmp. This tool works with Windows and Linux. From a security perspective, this net-snmp can be used as another troubleshooting tool to ensure that Cisco MARS and Cisco NCM are working correctly.

One basic tool, included with the toolset, is snmpwalk. This can be used to determine the OIDs used for a network device. Here's a partial execution of the command against a Cisco 2523 router.

C:\Apps\net-snmp\bin> snmpwalk -c cisco -v 1 10.1.1.200 more
SNMPv2-MIB::sysDescr.0 = STRING: Cisco Internetwork Operating System Software
IOS (tm) 2500 Software (C2500-IK8OS-L), Version 12.2(32), RELEASE SOFTWARE (fc1)
Copyright (c) 1986-2005 by cisco Systems, Inc.
Compiled Fri 02-Dec-05 16:15 by
SNMPv2-MIB::sysObjectID.0 = OID: SNMPv2-SMI::enterprises.9.1.27
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (1229180359) 142 days, 6:23:23.59
SNMPv2-MIB::sysContact.0 = STRING:
SNMPv2-MIB::sysName.0 = STRING: termserv-R5
SNMPv2-MIB::sysLocation.0 = STRING:
SNMPv2-MIB::sysServices.0 = INTEGER: 78
SNMPv2-MIB::sysORLastChange.0 = Timeticks: (0) 0:00:00.00
IF-MIB::ifNumber.0 = INTEGER: 4
IF-MIB::ifIndex.1 = INTEGER: 1

You can see that all the MIB OID values by adding the "-O n" option. By just typing "snmpwalk" you can get the full list of command line options. The use of "-O n" is shown below

C:\Apps\net-snmp\bin> snmpwalk -O n -c cisco -v 1 10.1.1.200
.1.3.6.1.2.1.1.1.0 = STRING: Cisco Internetwork Operating System Software
IOS (tm) 2500 Software (C2500-IK8OS-L), Version 12.2(32), RELEASE SOFTWARE(fc1)
Copyright (c) 1986-2005 by cisco Systems, Inc.
Compiled Fri 02-Dec-05 16:15 by
.1.3.6.1.2.1.1.2.0 = OID: .1.3.6.1.4.1.9.1.27
.1.3.6.1.2.1.1.3.0 = Timeticks: (1229164596) 142 days, 6:20:45.96
.1.3.6.1.2.1.1.4.0 = STRING:
.1.3.6.1.2.1.1.5.0 = STRING: termserv-R5
.1.3.6.1.2.1.1.6.0 = STRING:
.1.3.6.1.2.1.1.7.0 = INTEGER: 78
.1.3.6.1.2.1.1.8.0 = Timeticks: (0) 0:00:00.00
.1.3.6.1.2.1.2.1.0 =
INTEGER: 4
.1.3.6.1.2.1.2.2.1.1.1 = INTEGER: 1

You can then use this information to be more specific with your SNMP requests by using snmpget. Using the example above, we can just get the version information for the 2523 router by executing the command below

C:\Apps\net-snmp\bin> snmpget -c cisco -v 1 10.1.1.200 .1.3.6.1.2.1.1.1.0
SNMPv2-MIB::sysDescr.0 = STRING: Cisco Internetwork Operating System Software
IOS (tm) 2500 Software (C2500-IK8OS-L), Version 12.2(32), RELEASE SOFTWARE (fc1)
Copyright (c) 1986-2005 by cisco Systems, Inc.
Compiled Fri 02-Dec-05 16:15 by

NAC API Example

2009-04-29T15:34:00.001-04:00

The NAC API provides a method of inputting configuration into the NAC using a Perl script instead of typing it in manually. This is really nice if you have a large number of items to add or if you want to automate a particular process. One example would be to add the MAC addresses of printers to the device filter list. Another example would be to a method of automatically creating guest user accounts. Both of these features can be completed by the NAC Profiler and NAC Guest Server for larger deployments

There's a good example of using the NAC API in the NAC Manager FAQ

Packet Captures with Cisco IOS

2009-02-17T22:59:00.008-05:00

With Cisco IOS 12.4(20)T, Cisco now supports packet captures on interfaces. This is a welcome addition to the Cisco feature set. Previous to this addition, network admins had to rely on span ports and capturing traffic on end devices to troubleshoot network problems. Now, network admins can capture packets entering and exiting router interfaces. There are a couple of disappointments with the the packet capture. The first is that the configuration is a little bit difficult to use. There are a number of commands that need to be entered to execute a capture. It would have been nice if there was just one command, similar to tcpdump. The second disappointment is that the ability to view the packets, on the router, is very limited. In order to get a good view of the packet capture, it is necessary to export the capture file and view the file using Wireshark.

I'd like to go through an example of how to use the new packet capture feature. As stated above, there are number of steps to go through to create the capture. In this example, I will show how to create a 512 Kbytes circular buffer to hold the data and collect the data on the gigabitethernet 0/1 interface.

1. The first step is to create the buffer. In the configuration below I am calling the buffer "buf1"

monitor capture buffer buf1 size 512 max-size 512 circular

2. The next step is to define which interface will listen for the traffic. This is done by creating a capture point. In the configuration below I am calling the capture point "cap1". I am setting the capture point to capture ip packets sent and received on gigabitethernet 0/1.

monitor capture point ip cef cap1 gigabitethernet0/1 both

3. The next step is to associate the buffer with the capture point.

monitor capture point associate cap1 buf1

4. The next step is to to start the capture

monitor capture point start cap1

5. After the required data is captured, the capture is stopped

monitor capture point stop cap1

6. The data can then be viewed with the command below. As you can see in the associated information shown with the command, there is not alot of detail given

show monitor cap buffer buf1 dump
23:50:18.669 EDT Feb 17 2009 : IPv4 LES CEF : Gi0/1 None

499A6280: 00192F06 0C09001B D5FF3C05 08004500 ../.....U.<...E. 499A6290: 008000BA 0000F611 881DAD4F 20284465 ...:..v...-O (De 499A62A0: 29B91194 1194006C 00003668 E2340000 )9.....l..6hb4.. 499A62B0: 51D5B8B1 90BFB446 3F7011AF 78C98F42 QU81.?4F?p./xI.B 499A62C0: 696F3833 023841E8 5EF6988B C741F5E9 io83.8Ah^v..GAui 499A62D0: 4ACD925F 074DC56C 10B731B2 797F9C03 JM._.MEl.712y... 499A62E0: 28BF4C53 2ADF0EEF AE0F3526 98442EE2 (?LS*_.o..5&.D.b 499A62F0: 5A8C348A 246ABF28 3EFA15CB 11ABF76C Z.4.$j?(>z.K.+wl
499A6300: EC586E86 E802FF30 343BE135 9A0300 lXn.h..04;a5...

7. A better way to view the data is to export it. The data can be exported to another computer. The capture file can then be viewed with Wireshark. The export supports FTP, HTTP, HTTPS, PRAM, RCP, SCP, and TFTP. The example below shows TFTP.

monitor capture buffer buf1 export tftp://192.168.1.10/buf1.pcap

Here are some references to use for further information

Cisco IOS SSL VPN Example

2009-02-15T16:46:00.000-05:00

Cisco has made a big push into the SSL VPN market. Currently, their main focus appears to be on beefing up their SSL VPN support of the ASA FW. SSL VPN does exist within IOS as well, though. Version 12.4(20)T added a lot of new features. The IOS SSL VPN features are definitely lagging behind the ASA SSL VPN, but the basic functionality is available within IOS SSL VPN. The IOS SSL VPN supports clientless, thin client, and full client modes. The clientless mode uses a web portal. Thin client augments the web portal with port forwarding capability. The full client uses the AnyConnect SSL client. The IOS SSL VPN does not have RDP, telnet, ssh, etc plugin capability that exists in the ASA SSL VPN. It also does not support the dynamic access policy(DAP) also available in the ASA SSL VPN.

I wanted to show an example of using clientless and thin client features in this blog entry. Before going into the example, I wanted to point out the general methodology for implementing the IOS SSL VPN. The "webvpn context" command is the container that houses the individual parameters for the VPN. The "webvpn gateway" and "policy group" provide the parameters that are added to the "webvpn context".

The example is based on the diagram below

For this example, the router needs to provide a user on the 192.168.137.x network secure access to R1 through an SSL web portal. HTTP acccess, to R1, is provided through a URL link. HTTPS and SSH access, to R1, is provided by port forwarding. In a real world example, this type of access could allow emergency access for a network administrator from any computer.

The first step is to set up the authentication method for the user. The IOS SSL VPN uses the default AAA method by default. For this example, we will use local authentication with the commands below

aaa new-model
aaa authentication login default local
username cisco password cisco

The next step is to setup the IP and port information for connectivity to the SSL VPN. The IOS SSL VPN allows the IP to be based on the interface IP of the router or a virtual IP address. Additionally, the port can be the standard 443/tcp or it can be another manually assigned port. For this example, we will use the fa1/0 interface of the router and port 8000/tcp. This is shown below.

webvpn gateway SSL1
hostname SSL1
ip address 192.168.137.100 port 8000
ssl trustpoint TP-self-signed-4294967295
inservice

Notice the "ssl trustpoint" in the configuration. This is automatically created when the "inservice" command is added to active the configuration.

The next step is to create the "webvpn context". As stated earlier, this is the container for the VPN parameters. Within the "webvpn context" container, there are number of parameters that are defined and applied. For example:

A URL can be defined
The URL can be applied to a policy group
The policy group can then be applied to the context

This is all within the "webvpn context" container. An example is shown below

webvpn context SSL
url-list "R1"
heading "R1"
url-text "R1-http" url-value "http://192.168.1.2"
policy group default
url-list "R1"
default-group-policy default

This shows the URL list, R1, being defined and then applied to the policy group, default. The policy group, default, is then applied to the context with the "default-group-policy" command.

In a similar manner, the IOS SSL VPN can support port forwarding. This is shown below.

webvpn context SSL
port-forward "R1"
local-port 5000 remote-server "192.168.1.2" remote-port 443 description "R1 HTTPS"
local-port 5001 remote-server "192.168.1.2" remote-port 22 description "R1 SSH"
policy group default
port-forward "R1" auto-download
default-group-policy default

This portion shows how to forward ports. When a user uses a web browser to access https://127.0.0.1:5000, they are redirected to https://192.168.1.2 through the SSL connection. Similarly, when a users uses an SSH client to access 127.0.0.1 on port 5001, they are redirected to 192.168.1.2 on port 22. In the "port-forward" command, notice the "auto-download" parameter. This causes the port forward connectivity to launch automatically, instead of requiring the user to click on the "thin client" start button shown below

The screenshots below show the GUI experience based on the configuration above.

1. The user accesses the web portal at https://192.168.137.100:8000

2. The user logs in and is presented with the web portal

3. At the same time as step 2, the port forwarding window appears with the setting for port forwarding

Below is the full relevant config for the example above

aaa new-model
!
!
aaa authentication login default local
username cisco password 0 cisco
no ip http server
no ip http secure-server
webvpn gateway SSL1
hostname SSL1
ip address 192.168.137.100 port 8000
ssl trustpoint TP-self-signed-4294967295
inservice
!
webvpn context SSL
ssl authenticate verify all
!
url-list "R1"
heading "R1"
url-text "R1-http" url-value "http://192.168.1.2"
!
!
port-forward "R1"
local-port 5000 remote-server "192.168.1.2" remote-port 443 description "R1 HTTPS"
local-port 5001 remote-server "192.168.1.2" remote-port 22 description "R1 SSH"
!
policy group default
url-list "R1"
port-forward "R1" auto-download
banner "Welcome to the IOS SSL Lab"
default-group-policy default
gateway SSL1
inservice

Auto reconnect with Cisco AnyConnect VPN

2009-02-11T20:07:00.006-05:00

One of the great features about the Cisco AnyConnect VPN client is auto reconnect. This feature automatically reconnects a VPN session if the users internet connectivity drops. This means that if a user is connected to the VPN and the user's computer is accidentally unplugged, the VPN connection will be automatically reestablished when the computer is plugged back in. Another example would be a similar situation when a 3G wireless connection drops. If using the IPSec VPN client, a temporary loss of signal would cause the VPN connection to drop. With the AnyConnect VPN client, the VPN connection would be automatically reestablished.

This feature is turned on by default. As long as the user's computer does not go into a suspend state the AnyConnect VPN client will attempt to reconnect. This setting is defined in an XML file that is described in the Cisco AnyConnect VPN Administrator Guide, Release 2.3. The XML file is located in the \Documents and Settings\All Users\Application Data\Cisco\Cisco AnyConnect VPN Client\Profile\AnyConnectProfile.tmpl file. By modifying this file, the auto reconnect functionality can be disabled, if desired.

The portion of the file pertinent to auto reconnect appears as follows

Moving Cisco CSA Management Center to a New Server with New Name and New IP

2009-02-09T00:11:00.003-05:00

The Cisco Security Agent Management Center (CSA MC) provides the central management for CSA. The Agents check in with the CSA MC to make sure they have the latest policy and version of software. For this reason, it is important that the CSA MC be a stable server that is always operational with the same IP address and name. Unfortunately, there may be unavoidable instances where the CSA MC must be moved to a new server with a different IP and name. This guide will go through how to move the CSA MC application and database to a new server and migrate the Agents, running on the end computers, to use the new CSA MC.

The following assumptions are being made

The database is MS SQL Server 2005 Express Ediation and is housed on the same server as the CSA MC
CSA version 6.0
Screenshots show the old CSA MC with name "CSA" and the new CSA MC with name "CSA2"

Creating the New CSA MC
The first step is to install the exact same version of CSA MC application on the new server. The license does not need to be installed on the new CSA MC because the license will be moved over from the old CSA MC as part of the database move.

The next step is to move the database from the old CSA MC to the new CSA MC. This will allow all the registered agents, groups, rule modules, rules, etc to be moved to the new CSA MC. The detailed steps are listed below.

Note: It is important to note that doing this fully replaces the existing database. This means that any SSL certificate, Agent, policy, group,etc information on the existing database will be removed. We will see how this affects the Agent registration for the new CSA MC later.

First, on the new and old CSA MCs, stop the CSA MC service and the Agent, running on the CSA MC, using the following commands

Open Start->Run... and type "services.msc". Stop the Management Console, MSSQL, and Agent services. The services should look similar to the names below.

On the new CSA MC, rename the c:\Program Files\Cisco\CSAMC\CSAMC60\db directory to a new name, such as db-backup. Next, copy the c:\Program Files\Cisco\CSAMC\CSAMC60\db directory from the old CSA MC to the new CSA MC. At this point you have fork lifted the database from the old CSA MC into the new CSA MC. Reboot the new CSA MC.

Updating Agent Kits
As mentioned earlier, the old CSA MC database has been fork lifted into the new CSA MC and fully replaced the existing database on the new CSA MC. This means that any existing agent kits and any new agent kits generated on the new CSA MC still reference the old CSA MC. To correct this, we need to dig into the database and modify some entries.

Note: The database changes are based on my own research. Use at your own risk

The CSA MC installation did not include the Management console to view the SQL Server 2005 database. Microsoft offers the Microsoft SQL Server Management Studio Express to manage the database. Download this application to the CSA MC and install it. After it has been installed, launch it. Take the default login settings as shown below

Navigate down the database tree to the following section

Within this tree scroll down and look for "dbo.mc_config". Right click on the name and click "Open Table" as shown below

The table shows the entries that get sent as part of the Agent Kit. We will modify the parameters to reflect the new CSA MC instead of the old CSA MC.

This screenshot shows the database before any changes were made. The name CSA references the old CSA MC as does the IP 192.168.137.100

This screenshot shows the database after the changes were made. The name CSA2 references the new CSA MC as does the IP 192.168.137.15.

In addition to the changes listed above, you should probably modify the sslca.crt, sslca.ns, sslca.key, sslhost.crt, sslhost.csr, and sslhost.key files in the database. In my testing, everything seemed to work fine without modifying the values, but for completeness sake, I would recommend modifying both the names and the data to reflect the information on the new CSA MC server. The data can be retrieved from the c:\Program Files\cisco\CSAMC\CSAMC60\cfg directory. Right click on each file and open with notepad. Then copy the entire contents and paste it into the value portion of the database.

After the database has been changed, the agent kits need to be refreshed to incorporate the new values. This can be done by running the following command

C:\Program Files\Cisco\CSAMC\CSAMC60\bin>webmgr makekits_refresh
Generating rule programs...
Regenerating agent kits...
Done.

Once this is complete, the agent kits will have the correct information referencing the new CSA MC.

Fixing the Agent on the New CSA MC
Now, access the web GUI and navigate to Systems > Hosts. You'll notice that the new CSA MC server is not listed. This is because we are using the database from the old server. This is shown below.

The old CSA MC does not need to be listed here, so you can move the server named "CSA.lab.com" to the recycling bin.

The Agent running on the new CSA MC, CSA2.lab.com, is not showing up because we are using the database from the old CSA MC. The new CSA MC was registered in the original database on the new CSA MC. This was the database that we renamed "db-backup". Confirmation of the problem is shown in the agent logs on the new CSA MC

1504: CSA2: Feb 06 2009 21:58:10.773 -0600: %CSA-4-REGISTRATION_FAILED_INVALID_REGID: %[Component=Csamanager][PID=1076]: No deployment host exists on server with registration ID={90A55B27-4F2B-490A-A27E-081D8747E3F5}.

We need to get a new registration that is recognized by the current database on the new CSA MC. In order to do this, the following steps should be followed:

Remove the Agent on the new CSA MC
Reboot
Download and install the "Servers CSA Management Center V6.0.0.201" Agent Kit from the new CSA MC.
Reboot
Verify that the new CSA MC shows up in the CSA MC web GUI in "Systems > Hosts".

Following these steps will allow the Agent, running on the new CSA MC, to be properly registered and displayed.

Handling Existing Agents
The registration IDs for the existing Agents have been ported to the new CSA MC with the database move. There are just two steps that need to occur, on the computers with the Agent installed, to complete the migration of the already deployed Agents from the old CSA MC to the new CSA MC

Modify the c:\Program Files\Cisco\CSAgent\cfg\agent.bundle file to reference the new CSA MC
Replace the c:\Program Files\Cisco\CSAgent\cfg\sslca.crt file to reference the new CSA MC

To complete step 1, you will first need to stop the Agent with the "net stop csagent" command. After that, open the agent.bundle file and make the modifications as shown in the screenshots below. In the screenshots, CSA and 192.168.137.100 reference the old CSA MC and CSA2 and 192.168.137.15 reference the new CSA MC. The original file is shown on the left and the new file is shown on the right

The second step was to replace the sslca.crt file. The CSA MC creates its own root certificate and distributes it in the CSA Agent Kit. The Agents then trust the SSL certificate provided by the CSA MC because it is signed with the root certificate. When the CSA MC is changed, the root certificate also needs to be changed to reference the new CSA MC. The new sslca.crt can be obtained from the new CSA MC in the c:\Program Files\Cisco\CSAMC\CSAMC60\cfg directory. It needs to replace the c:\Program Files\Cisco\CSAgent\cfg\sslca.crt file on each computer with the Agent.

The two steps above can normally be accomplished with a central application used to push out applications. Some examples of these applications would be Microsoft SMS or Altiris.

Once the two step above are completed, you should see the following screen on the Agent status page

References

Easy way to find the SNMP OIDs on an ASA FW

2009-02-08T23:12:00.005-05:00

One excellent command for viewing ASA SNMP OIDs is "show snmp-server oidlist". It appears that this is a hidden command, based on the ASA 8.0 Reference Guide. This command provides information on the OID and name associated with it. I've included an example below

show snmp-server oidlist

-------------------------------------------------
[0] 1.3.6.1.2.1.1.1. sysDescr
[1] 1.3.6.1.2.1.1.2. sysObjectID
[2] 1.3.6.1.2.1.1.3. sysUpTime
[3] 1.3.6.1.2.1.1.4. sysContact
[4] 1.3.6.1.2.1.1.5. sysName
[5] 1.3.6.1.2.1.1.6. sysLocation

I found this command at the link below. All credit should go to Joe Harris.
http://6200networks.com/2008/10/23/asa-snmp-oids/

Cisco NAC Version 4.1.8 released

2009-02-02T22:24:00.002-05:00

Cisco NAC version 4.1.8 has been released. The release notes spell out the changes.

The general NAC Manager/Server enhancements are

CAS Fallback Behavior Enhancement
CAS HA Pair Link-Detect Configuration Enhancement
DHCP Failover Behavior Enhancement

Additionally, there are number of agent fixes.

Creating a self-signed SSL certificate for Microsoft IIS

2009-02-01T14:31:00.002-05:00

I wrote a previous blog entry that gave information creating a self-signed SSL certificate with Apache. I recently also had to create a self-signed SSL certificate for Microsoft IIS. It turns out that this is really easy.

Microsoft provides an IIS resource kit that provides an application to create a self-signed SSL certificate. The application is called selfssl.exe. The steps for creating the SSL certificate are listed below.

Download and install the IIS resource kit.
Open a cmd prompt and type: cd "c:\Program Files\IIS Resources\SelfSSL"
Create the self-signed SSL certificate by just typing: selfssl.exe

There are a number of command line parameters you can enter to modify the default settings. They are listed below

C:\Program Files\IIS Resources\SelfSSL>selfssl /?
Microsoft (R) SelfSSL Version 1.0
Copyright (C) 2003 Microsoft Corporation. All rights reserved.

Installs self-signed SSL certificate into IIS.
SELFSSL [/T] [/N:cn] [/K:key size] [/S:site id] [/P:port]

/T Adds the self-signed certificate to "Trusted Certificates"
list. The local browser will trust the self-signed certificate
if this flag is specified.
/N:cn Specifies the common name of the certificate. The computer
name is used if not specified.
/K:key size Specifies the key length. Default is 1024.
/V:validity days Specifies the validity of the certificate. Default is 7 days.
/S:site id Specifies the id of the site. Default is 1 (Default Site).
/P:port Specifies the SSL port. Default is 443.
/Q Quiet mode. You will not be prompted when SSL settings are
overwritten.

The default behaviour is equivalent with:

selfssl.exe /N:CN=SERVER /K:1024 /V:7 /S:1 /P:443

Below is an example of creating the self-signed SSL certificate

C:\Program Files\IIS Resources\SelfSSL>selfssl
Microsoft (R) SelfSSL Version 1.0
Copyright (C) 2003 Microsoft Corporation. All rights reserved.

Do you want to replace the SSL settings for site 1 (Y/N)?y
The self signed certificate was successfully assigned to site 1.

C:\Program Files\IIS Resources\SelfSSL>cd "c:\Program Files"

NAC Architectures Presentation

2009-01-31T00:17:00.005-05:00

Chesapeake Netcraftsmen hosts a monthly Cisco Users Group Meeting. Last month we had a presentation on NAC Architectures and on Troubleshooting ASA. You can view the presentation slides at http://www.netcraftsmen.net/cmug

TCP and UDP Ports used for the Cisco VPN Client

2009-01-23T22:33:00.000-05:00

The Cisco VPN client is the client side application used to encrypt traffic from an end user's computer to the company network. IPSec is used to encrypt the traffic. When using standard IPSec, IKE is used for the key negotiation and IPSec to encrypt the data. IKE uses UDP port 500 and IPSec uses IP protocol 50, assuming ESP is used.

In most situations, there is a PAT device between the VPN client and the head end VPN device. PAT works by differentiating users by the UDP or TCP port used. Since IPSec uses IP protocol 50, it is impossible for more than one user to connect to the VPN device, through the PAT. This is because the IP protocol operates at layer 3 of the OSI reference model and PAT functionality exists at layer 4. For this reason, there are three different methods of tunneling IPSec traffic. It is important to understand the ports used for the different methods to ensure that those ports are not blocked.

NAT Traversal - This method still uses 500/udp for IKE negotiation, but then tunnels IPSec data traffic within 4500/udp packets. This is the default method for UDP tunneling with the Cisco VPN client
IPSec over UDP - This method still uses 500/udp for IKE negotiation, but then tunnels IPSec data traffic within a pre-defined UDP port. The default port for this traffic is 10000/udp.
IPSec over TCP - This method tunnels both the IKE negotiation and IPSec data traffic within a pre-defined TCP port. The default port for this traffic is 10000/tcp. This is the only method that tunnels both IKE and IPSec within the same stream.

Cisco Mid-Atlantic User's Group Meeting - Jan 15, 16

2009-01-06T14:53:00.004-05:00

On Jan 15 and 16, Chesapeake Netcraftsmen will be hosting the Cisco Mid-Atlantic User's Group Meeting. For more details, go to http://www.netcraftsmen.net/cmug.

Below are the speakers and topics

* Rob Chee, CCIE, Sr. Consultant at Chesapeake NetCraftsmen, will explain How to Deploy Cisco Network Access Control. Rob will show you various deployment methods you can use to get the most out of Cisco NAC. If you have been considering NAC for remote users, office users, or wireless users, you won't want to miss this informative presentation.

* Having problems with your PIX/ASA firewall? Eric Stuhl, CCIE, Sr. Consultant at Chesapeake NetCraftsmen will talk on Troubleshooting PIX/ASA Firewalls. You will learn expert techniques for diagnosing and repairing problems with your firewall.

Java and SSL Certificates

2008-12-28T10:54:00.010-05:00

Overview

I have a Linux box at home, running Apache 2.2, that I use to archive pictures. I use an application called Gallery as a front end to organize and view the photos. I'm using a Java application called Gallery Remote to upload pictures to the server. I've also added SSL encryption so that the username and password, to access the site, are not sent in the clear.

Problem and Solution

The problem was that Gallery Remote wasn't able to connect to the server. It seemed to be having problems with the SSL certificate I had on the web server. I was using a self-signed SSL certificate, so that was definitely possible. I checked out the SSL certificate and found that it was expired. I regenerated a new certificate using the instructions on the Apache Website. The relevant text is shown below.

How do I create a self-signed SSL Certificate for testing purposes?
Make sure OpenSSL is installed and in your PATH.
Run the following command, to create server.key and
server.crt files:
$ openssl req -new -x509 -nodes -out server.crt -keyout server.key
These can be used as follows in your httpd.conf
file:
             SSLCertificateFile    /path/to/this/server.crt
             SSLCertificateKeyFile /path/to/this/server.key
 
It is important that you are aware that this
server.key does not have any passphrase.
To add a passphrase to the key, you should run the following
command, and enter & verify the passphrase as requested.

$ openssl rsa -des3 -in server.key -out server.key.new
$ mv server.key.new server.key

Please backup the server.key file, and the passphrase
you entered, in a secure location.

After restarting the webserver, I was still having problems with Gallery Remote. I then found out that Java has it's own repository of trusted SSL certificates. My SSL certificate was a self-signed certificate, so it definitely wasn't in the default SSL Certificate trust list. One method of adding the certificate is by going through the Java control panel. Another method is to add it through the command line. This was described on the Gallery Remote FAQ page. The relevant text is shown below.

Using HTTPS
You can use https:// URLs with Gallery Remote to connect to secured web sites. This functionality is only available on Java 1.4 and later. If the site you are attempting to connect to uses a server certificate that is not certified by a trusted certificate authority, Gallery Remote will be unable to connect. If this happens, you will need to add the site's certificate to the Java registry of trusted certificates:
For Windows:
Go to the site with Internet Explorer
Go to menu File>Properties
In the Properties window, click Certificates
On the Details tab, click Copy to File...
In the wizard, select DER-encoded X.509 certificate and save it to a
file
Open a console window (cmd.exe)
Type the following command-line:
     keytool -import -trustcacerts -file path_to_cer_file 
-keystore %JAVA_HOME%/jre/lib/security/cacerts -alias arbitrary_name
You'll be prompted for the store password, which by default is
changeit

I used that and it worked. It's interesting to note that Java uses it's own keystore and that there's a default password used if using the command line.

Allowing CSA Management Center to access WSUS server

2008-12-16T14:29:00.020-05:00

The pre-configured CSA 6.0 policy for the CSA Management Center does not allow for connectivity to the WSUS server. Rule 269 blocks the access as shown below

You'll notice that the screenshot is from the events shown on the agent GUI on the management center. This is because rule 269 does not log by default. Because of this, the denied packets do not show up in the management center event logs. In order to view the logs on the management center, you would need to do one of two things:

Explicitly turn on logging for rule 269

Enable log overrides for a particular group

Once the denied rule shows up in the management center event logs, the denied events can be viewed on the management center. This helps with the troubleshooting process.

The problem is that rule 269 blocks all network traffic not explicitly allowed by another rule. Since rule 269 applies to the "CSA MC Network Security Module", it only affects the management center. This is why WSUS updates work fine with the pre-configured server and desktop rules. In those policies, there is no rule explicitly blocking network traffic. The default action is to allow traffic, so the WSUS update traffic is allowed for desktops and servers.

There are a number of ways to fix the problem for the management center. The easiest method is to use the Wizard in the event log entry for rule 269. The Wizard provides a method of easily creating an exception rule for the specific traffic that was blocked.

The first step is to locate the rule 269 event log entry and click on the Wizard link. This is shown in the red oval in the diagram below

The next step is to click on the "Allow Operation" radio button, provide a justification and click "Finish". This is shown below.

After "Finish" is clicked, the necessary variables and rule are created. The next step is to generate the policy to install the rules. The diagram below shows the variables and rules that will be generated.

After the rule generation, there should now be an exception rule that allows access to the WSUS server to get Microsoft updates. This is shown below.

A closer inspection of the rule shows that it is a granular rule only allowing executable "svchost.exe -k netsvc" to talk to the WSUS server, as a client, on port 80/tcp. This is shown below.

To verify that the rule is really working, you can temporarily turn on logging for the exception rule. This is shown below.

A reboot of the management center should kick off the WSUS update check again. Once this is completed, something similar to the following should be in the event log

After verifying the success of the exception rule, make sure to turn off logging on the exception rule and any other logging that was turned on for troubleshooting purposes above.

Cisco CSA 6.0 Upgrade Note

2008-12-15T16:36:00.011-05:00

When upgrading to CSA 6.0, most of the effort is concentrated on upgrading the management center. Cisco provides fairly well documented instructions in their installation guide. The part they don't talk about enough is the CSA Agent upgrade to 6.0.

One problem I ran into has to do with upgrading clients running Windows XP SP3. According to the CSA 5.2 release notes, CSA 5.2 only supports Windows XP SP 0, 1, or 2. Of course, you can surmise that they just forgot to update their release notes with XP SP3 support, since the documentation is dated 4/2/07 and Windows XP SP3 came out on 5/6/08. Unfortunately, that is not true. This can be seen when viewing the "Host Identification" information under "Systems > Hosts > [hostname]". I've shown an example below. The "unsupported" information is shown in red.

Despite this screen, the CSA 5.2 Agent works fine after the upgrade to XP SP3. The big problem comes when the Management Center is upgraded to version 6.0 and you try to do the scheduled software update to upgrade all the agents to 6.0. This does not work. The normal process is to

Access "Systems > Software Updates > Scheduled Software Updates"
Create a new Item that schedules the update for a particular group
The agents check in with the Management Center, download the update, and install

The problem is that the agents never download the update. The "System > Hosts > [hostname]" page always shows that the software version as "Agent is running the latest software" instead of "Update Available". Both of these screenshots are shown below.

The only workaround is to create a new CSA 6.0 agent kit and push the new agent kit to all the users via your normal application installation mechanism (Altiris, SMS,...)

Cisco NAC Manager HA log files

2008-12-07T21:40:00.007-05:00

The NAC Manager documentation provides a number of logs that can be viewed to troubleshoot various issues. The only problem is that when a problem occurs it would be really nice to have a reference showing what good log output looks like. That's what I'd like to share here. Hopefully this will help someone troubleshooting an issue.

The log files below show the log files on a standby NAM when it becomes active
/perfigo/control/tomcat/logs/localhost_log..txt
2008-11-19 14:22:58 StandardHost[localhost]: Removing web application at context path /admin
2008-11-19 14:22:58 StandardHost[localhost]: Removing web application at context path
2008-11-19 14:23:03 WebappLoader[/admin]: Deploying class repositories to work directory /perfigo/control/tomcat/work/Standalone/localhost/admin
2008-11-19 14:23:03 WebappLoader[/admin]: Deploy JAR /WEB-INF/lib/jsf_hack_tld.jar to /perfigo/control/tomcat/webapps/admin/WEB-INF/lib/jsf_hack_tld.jar
2008-11-19 14:23:04 ContextConfig[/admin]: Configured an authenticator for method NONE
2008-11-19 14:23:04 PersistentManager[/admin]: Seeding random number generator class java.security.SecureRandom
2008-11-19 14:23:04 PersistentManager[/admin]: Seeding of random number generator has been completed
2008-11-19 14:23:04 PersistentManager[/admin]: No Store configured, persistence disabled
2008-11-19 14:23:22 StandardWrapper[/admin:default]: Loading container servlet default
2008-11-19 14:23:22 StandardWrapper[/admin:invoker]: Loading container servlet invoker
2008-11-19 14:23:22 HostConfig[localhost]: Deploying web application directory ROOT
2008-11-19 14:23:22 StandardHost[localhost]: Installing web application at context path from URL file:/perfigo/control/tomcat/normal-webapps/ROOT
2008-11-19 14:23:22 WebappLoader[]: Deploying class repositories to work directory /perfigo/control/tomcat/work/Standalone/localhost/_
2008-11-19 14:23:22 StandardManager[]: Seeding random number generator class java.security.SecureRandom
2008-11-19 14:23:22 StandardManager[]: Seeding of random number generator has been completed
2008-11-19 14:23:23 StandardWrapper[:default]: Loading container servlet default
2008-11-19 14:23:23 StandardWrapper[:invoker]: Loading container servlet invoker
2008-11-19 14:23:23 HostConfig[localhost]: Deploying web application directory upload
2008-11-19 14:23:23 StandardHost[localhost]: Installing web application at context path /upload from URL file:/perfigo/control/tomcat/normal-webapps/upload
2008-11-19 14:23:23 WebappLoader[/upload]: Deploying class repositories to work directory /perfigo/control/tomcat/work/Standalone/localhost/upload
2008-11-19 14:23:23 StandardManager[/upload]: Seeding random number generator class java.security.SecureRandom
2008-11-19 14:23:23 StandardManager[/upload]: Seeding of random number generator has been completed
2008-11-19 14:23:23 StandardWrapper[/upload:default]: Loading container servlet default
2008-11-19 14:23:23 StandardWrapper[/upload:invoker]: Loading container servlet invoker
2008-11-19 14:23:23 HostConfig[localhost]: Deploying web application directory wlan
2008-11-19 14:23:23 StandardHost[localhost]: Installing web application at context path /wlan from URL file:/perfigo/control/tomcat/normal-webapps/wlan
2008-11-19 14:23:23 WebappLoader[/wlan]: Deploying class repositories to work directory /perfigo/control/tomcat/work/Standalone/localhost/wlan
2008-11-19 14:23:23 ContextConfig[/wlan]: Configured an authenticator for method NONE
2008-11-19 14:23:23 StandardManager[/wlan]: Seeding random number generator class java.security.SecureRandom
2008-11-19 14:23:23 StandardManager[/wlan]: Seeding of random number generator has been completed
2008-11-19 14:23:23 StandardWrapper[/wlan:default]: Loading container servlet default
2008-11-19 14:23:23 StandardWrapper[/wlan:invoker]: Loading container servlet invoker
2008-11-19 14:23:23 HostConfig[localhost]: Deploying web application directory packages
2008-11-19 14:23:23 StandardHost[localhost]: Installing web application at context path /packages from URL file:/perfigo/control/tomcat/normal-webapps/packages
2008-11-19 14:23:23 WebappLoader[/packages]: Deploying class repositories to work directory /perfigo/control/tomcat/work/Standalone/localhost/packages
2008-11-19 14:23:23 StandardManager[/packages]: Seeding random number generator class java.security.SecureRandom
2008-11-19 14:23:23 StandardManager[/packages]: Seeding of random number generator has been completed
2008-11-19 14:23:24 StandardWrapper[/packages:default]: Loading container servlet default
2008-11-19 14:23:24 StandardWrapper[/packages:invoker]: Loading container servlet invoker
2008-11-19 14:23:24 HostConfig[localhost]: Deploying web application directory download
2008-11-19 14:23:24 StandardHost[localhost]: Installing web application at context path /download from URL file:/perfigo/control/tomcat/normal-webapps/download
2008-11-19 14:23:24 WebappLoader[/download]: Deploying class repositories to work directory /perfigo/control/tomcat/work/Standalone/localhost/download
2008-11-19 14:23:24 StandardManager[/download]: Seeding random number generator class java.security.SecureRandom
2008-11-19 14:23:24 StandardManager[/download]: Seeding of random number generator has been completed
2008-11-19 14:23:24 StandardWrapper[/download:default]: Loading container servlet default
2008-11-19 14:23:24 StandardWrapper[/download:invoker]: Loading container servlet invoker

/var/log/ha-log
heartbeat: 2008/11/19_14:22:58 info: Received shutdown notice from 'camanager1'.
heartbeat: 2008/11/19_14:22:58 info: Resources being acquired from camanager1.
heartbeat: 2008/11/19_14:22:58 info: acquire all HA resources (standby).
heartbeat: 2008/11/19_14:22:58 info: No local resources [/usr/lib64/heartbeat/ResourceManager listkeys camanager2] to acquire.
heartbeat: 2008/11/19_14:22:58 info: Acquiring resource group: camanager1 x.x.x.x controlsmart
heartbeat: 2008/11/19_14:22:58 info: Running /etc/ha.d/resource.d/IPaddr x.x.x.x start
heartbeat: 2008/11/19_14:22:58 info: /sbin/ifconfig eth0:0 x.x.x.x netmask 255.255.255.0 broadcast 172.31.31.255
heartbeat: 2008/11/19_14:22:58 info: Sending Gratuitous Arp for x.x.x.x on eth0:0 [eth0]
heartbeat: 2008/11/19_14:22:58 /usr/lib64/heartbeat/send_arp -i 1010 -r 5 -p /var/lib/heartbeat/rsctmp/send_arp/send_arp-x.x.x.x eth0 x.x.x.x auto x.x.x.x ffffffffffff
heartbeat: 2008/11/19_14:22:58 info: Running /perfigo/control/bin/controlsmart start
heartbeat: 2008/11/19_14:23:02 info: all HA resource acquisition completed (standby).
heartbeat: 2008/11/19_14:23:02 info: Standby resource acquisition done [all].
heartbeat: 2008/11/19_14:23:02 info: Running /etc/ha.d/rc.d/status status
heartbeat: 2008/11/19_14:23:04 info: Taking over resource group x.x.x.x
heartbeat: 2008/11/19_14:23:04 info: Acquiring resource group: camanager1 x.x.x.x controlsmart
heartbeat: 2008/11/19_14:23:04 info: Running /perfigo/control/bin/controlsmart start
heartbeat: 2008/11/19_14:23:04 info: /usr/lib64/heartbeat/mach_down: nice_failback: foreign resources acquired
heartbeat: 2008/11/19_14:23:04 info: mach_down takeover complete.
heartbeat: 2008/11/19_14:23:04 info: mach_down takeover complete for node camanager1.
heartbeat: 2008/11/19_14:23:14 WARN: node camanager1: is dead
heartbeat: 2008/11/19_14:23:14 info: Dead node camanager1 gave up resources.
heartbeat: 2008/11/19_14:23:14 info: Link camanager1:eth1 dead.

These logs show the story when a NAM starts as the active NAM and then "service perfigo stop" is entered to turn off the NAC service
/perfigo/control/tomcat/logs/localhost_log..txt
2008-11-19 14:22:55 StandardHost[localhost]: Removing web application at context path /admin
2008-11-19 14:22:55 StandardHost[localhost]: Removing web application at context path /upload
2008-11-19 14:22:55 StandardHost[localhost]: Removing web application at context path /download
2008-11-19 14:22:55 StandardHost[localhost]: Removing web application at context path /packages
2008-11-19 14:22:55 StandardHost[localhost]: Removing web application at context path /wlan
2008-11-19 14:22:55 StandardHost[localhost]: Removing web application at context path

/var/log/ha-debug
heartbeat: 2008/11/19_14:22:54 info: Heartbeat shutdown in progress. (4516)
heartbeat: 2008/11/19_14:22:54 info: Giving up all HA resources.
heartbeat: 2008/11/19_14:22:54 info: Releasing resource group: camanager1 x.x.x.x controlsmart
heartbeat: 2008/11/19_14:22:54 info: Running /perfigo/control/bin/controlsmart stop
heartbeat: 2008/11/19_14:22:58 info: Running /etc/ha.d/resource.d/IPaddr x.x.x.x stop
heartbeat: 2008/11/19_14:22:58 info: /sbin/route -n del -host x.x.x.x
heartbeat: 2008/11/19_14:22:58 info: /sbin/ifconfig eth0:0 down
heartbeat: 2008/11/19_14:22:58 info: IP Address x.x.x.x released
heartbeat: 2008/11/19_14:22:58 info: All HA resources relinquished.
heartbeat: 2008/11/19_14:22:59 info: killing HBREAD process 4521 with signal 15
heartbeat: 2008/11/19_14:22:59 info: killing HBFIFO process 4519 with signal 15
heartbeat: 2008/11/19_14:22:59 info: killing HBWRITE process 4520 with signal 15
heartbeat: 2008/11/19_14:22:59 info: Core process 4519 exited. 3 remaining
heartbeat: 2008/11/19_14:22:59 info: Core process 4520 exited. 2 remaining
heartbeat: 2008/11/19_14:22:59 info: Core process 4521 exited. 1 remaining
heartbeat: 2008/11/19_14:22:59 info: Heartbeat shutdown complete.

This show the status after "service perfigo start" is entered with another NAM active
[root@camanager1 logs]# service perfigo start
Starting High-Availability services:
[ OK ]
Please wait while bringing up service IP.
Heartbeat service is running.
Service IP is up on the peer node.
Stopping postgresql service: [ OK ]
Starting postgresql service: [ OK ]
DROP DATABASE
CREATE DATABASE
DROP DATABASE
CREATE DATABASE
Database synced
[root@camanager1 logs]#

/perfigo/control/tomcat/logs/localhost_log..txt
2008-11-19 14:30:14 WebappLoader[/admin]: Deploying class repositories to work directory /perfigo/control/tomcat/work/Standalone/localhost/admin
2008-11-19 14:30:14 WebappLoader[/admin]: Deploy JAR /WEB-INF/lib/jsf_hack_tld.jar to /perfigo/control/tomcat/webapps/admin/WEB-INF/lib/jsf_hack_tld.jar
2008-11-19 14:30:14 ContextConfig[/admin]: Configured an authenticator for method NONE
2008-11-19 14:30:14 PersistentManager[/admin]: Seeding random number generator class java.security.SecureRandom
2008-11-19 14:30:14 PersistentManager[/admin]: Seeding of random number generator has been completed
2008-11-19 14:30:14 PersistentManager[/admin]: No Store configured, persistence disabled
2008-11-19 14:30:15 StandardWrapper[/admin:default]: Loading container servlet default
2008-11-19 14:30:15 StandardWrapper[/admin:invoker]: Loading container servlet invoker
2008-11-19 14:30:15 HostConfig[localhost]: Deploying web application directory ROOT
2008-11-19 14:30:15 StandardHost[localhost]: Installing web application at context path from URL file:/perfigo/control/tomcat/admin-webapps/ROOT
2008-11-19 14:30:15 WebappLoader[]: Deploying class repositories to work directory /perfigo/control/tomcat/work/Standalone/localhost/_
2008-11-19 14:30:15 StandardManager[]: Seeding random number generator class java.security.SecureRandom
2008-11-19 14:30:15 StandardManager[]: Seeding of random number generator has been completed
2008-11-19 14:30:16 StandardWrapper[:default]: Loading container servlet default
2008-11-19 14:30:16 StandardWrapper[:invoker]: Loading container servlet invoker

/var/log/ha-log
heartbeat: 2008/11/19_14:27:25 info: **************************
heartbeat: 2008/11/19_14:27:25 info: Configuration validated. Starting heartbeat 1.2.5
heartbeat: 2008/11/19_14:27:25 info: heartbeat: version 1.2.5
heartbeat: 2008/11/19_14:27:26 info: Heartbeat generation: 44
heartbeat: 2008/11/19_14:27:26 info: ucast: write socket priority set to IPTOS_LOWDELAY on eth1
heartbeat: 2008/11/19_14:27:26 info: ucast: trying to bind: eth1

heartbeat: 2008/11/19_14:27:26 info: ucast: bound send socket to device: eth1
heartbeat: 2008/11/19_14:27:26 info: ucast: try binding receive socket to device: eth1
heartbeat: 2008/11/19_14:27:26 info: ucast: could bind receive socket to device: eth1:fe00a8c0.
heartbeat: 2008/11/19_14:27:26 info: ucast: started on port 694 interface eth1 to 192.168.0.253
heartbeat: 2008/11/19_14:27:26 notice: Using watchdog device: /dev/watchdog
heartbeat: 2008/11/19_14:27:26 info: pid 19899 locked in memory.
heartbeat: 2008/11/19_14:27:26 info: Local status now set to: 'up'
heartbeat: 2008/11/19_14:27:27 info: pid 19902 locked in memory.
heartbeat: 2008/11/19_14:27:27 info: pid 19903 locked in memory.
heartbeat: 2008/11/19_14:27:27 info: pid 19904 locked in memory.
heartbeat: 2008/11/19_14:27:27 info: Link camanager2:eth1 up.
heartbeat: 2008/11/19_14:27:27 info: Status update for node camanager2: status active
heartbeat: 2008/11/19_14:27:27 info: Local status now set to: 'active'
heartbeat: 2008/11/19_14:27:27 info: remote resource transition completed.
heartbeat: 2008/11/19_14:27:27 info: remote resource transition completed.
heartbeat: 2008/11/19_14:27:27 info: Local Resource acquisition completed. (none)
heartbeat: 2008/11/19_14:27:27 info: Initial resource acquisition complete (T_RESOURCES(them))
heartbeat: 2008/11/19_14:27:27 info: Running /etc/ha.d/rc.d/status status

Essential Cisco NAC deployment tools

2008-12-04T09:42:00.009-05:00

When deploying NAC there are a number of very useful tools that can help with implementing and troubleshooting. Here's a list of tools I've found useful

Wireshark - This is an open source network protocol analyzer that allows you to see exactly what traffic is going across the wire. There's a "Follow TCP Stream" feature that allows you to see the entire stream of traffic for a session. One place this can be used is when looking for certificate CRL information being sent from a client to a CA. You'll be able to see the exact URL that is being used in an easy to read manner.

LDAP Browser - This tool allows you to browse the LDAP tree to help determine what entries you should match on.

Kerbtray - This is a one tool in a set of Microsoft resource kit tools that is meant for Windows 2003, but also works for Windows XP. This tool provides information about Kerberos authentication. This is invaluable for troubleshooting AD SSO issues.

Camstudio - This is an open source video creation tool that you can use to create short video tutorials showing how NAC works. It can create an AVI or Flash file of your screen while you're demonstrating different NAC features. This can be a great tool for providing a visual representation of the NAC login process during end user training

Irfanview - This is a great tool for editing screenshots

Solution for Slow Cisco NAC WSUS Requirement Check

2008-11-10T22:46:00.004-05:00

Slow NAC posture validation can be one of the biggest stumbling blocks for a successful NAC deployment. One of the biggest reasons for slow posture validation is the time it takes for the WSUS Requirement check. I've come up with a list of troubleshooting steps to try to reduce the time it takes for the WSUS Requirement checks

Troubleshooting Option 1: Use the Latest version of Windows Update Agent
The latest version of Windows Update Agent includes new features that speed up the WSUS check process. First, make sure that Windows Update Agent 3.0 release is being used on the client. Also, the KB927891 patch must be installed if you are running XP SP2. You can verify the version by looking at the version of the c:\WINDOWS\System32\wuaueng.dll file. The version should be 7.2.6001.784 as shown in the picture

Based on the links listed below, this Windows Update Agent release is backwards compatible with WSUS release 2.0.

Because of the major changes that have been made with the new Windows Update Agent, this troubleshooting step should be done before any other troubleshooting is done. In addition to the faster checks, this latest version includes a number of fixes that controls the CPU utilization. Below are two links explaining the changes

http://blogs.technet.com/wsus/archive/2007/04/28/update-on.aspx
http://blogs.technet.com/wsus/archive/2007/05/15/srvhost-msi-issue-follow-up.aspx

The first link actually starts off with the following statement

In addition to the next week’s WSUS 3.0 release, we are making the new client portion available via the following plan to our customers who continue to experience performance issues like UI hang and long scan times.

In one instance, I saw a 90 second scan time go down to 5 seconds. I used the PT (Protocol Tracker) lines of the c:\windows\WindowsUpdate.log file to verify this. Below are screenshots with long time with Windows Update Agent 2.0 followed by the short time after the Windows Update Agent 3.0 upgrade.

This first screen shows the version, start time and end time in bold. You'll notice that updates take 98 seconds to complete.

This second screen also shows the version, start time and end time in bold. You'll notice that updates takes 2 seconds to complete

Troubleshooting Option 2: Defragment datastore.edb
The c:\windows\SoftwareDistribution\DataStore\DataStore.edb file is a database file that stores the local information about Microsoft Updates. When the Windows Update Agent downloads the WSUS data store, it compares it with the local data store in the DataStore.edb database. I found the instruction for defragementing the database on a Microsoft Forum Link I've posted the relevant information below.

The detection scan hits DataStore.edb causing a buffer overflow.
One can run esentutl from a Command Prompt to defragment DataStore.edb
instead of deleting it in hopes that will resolve the issue -

esentutl /d %windir%\SoftwareDistribution\Datastore\datastore.edb

If that doesn't resolve the issue, attempt to Recover the file -

esentutl /r %windir%\SoftwareDistribution\Datastore\datastore.edb

[This command performs recovery, bringing all databases to a
consistent state]

The next to last resort is to attempt to Repair it -

esentutl /p %windir%\SoftwareDistribution\Datastore\datastore.edb

NOTE: MS recommends that if the system is imaged regularly that a new
system image be done after running ANY of the above operations

* On XP Home Edition, one must stop the Automatic Updates service PRIOR
to running the above. This wasn't the case when doing so on XP Pro *

Troubleshooting Option 3: Remove Corrupted Database
This troubleshooting step removes the database directory entirely. The downside of this solution is that you will lose any history of updates. I found this procedure on the following website: http://myitkb.net/category/windows-updates. I've posted the relevant information below

type in net stop wuauserv and then hit
then enter cd /d %windir%\SoftwareDistribution hit
rd /s DataStore
click Yes at prompt
and then type in net start wuauserv and hit

**Note: On one machine I was testing with, I corrupted something that was required for Windows Update Agent to start. I used the commands from the on a web forum to fix the problem:

sc sdset bits "D:(A;;CCLCSWRPWPDTLOCRRC;;;SY)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;BA)(A;;CCLCSWLOCRRC;;;AU)(A;;CCLCSWRPWPDTLOCRRC;;;PU)"

sc sdset wuauserv "D:(A;;CCLCSWRPWPDTLOCRRC;;;SY)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;BA)(A;;CCLCSWLOCRRC;;;AU)(A;;CCLCSWRPWPDTLOCRRC;;;PU)"

Use at your own risk

Microsoft WSUS Guide for Cisco NAC deployments

2008-11-07T09:29:00.005-05:00

Microsoft Windows Server Update Services (WSUS) provides a method for managing Microsoft updates for company computers. Within a company, there are one or more WSUS servers that gets updates from Microsoft. Computers, within the company network, check in with this WSUS server to get their Microsoft updates. There are a number of benefits for using a WSUS server. Some of the benefits are

Control when updates are installed - This allows companies to test updates before deploying them to the user community.
Lower internet bandwidth usage - Keep the bandwidth, used for downloading Microsoft updates, within the internal network. This would keep the internet connection from becoming overloaded by users downloading updates directly from Microsoft

From a security perspective, keeping current with the latest Microsoft updates is very important. Computers are vulnerable to attacks if they do not have they do not have the latest security updates installed. Cisco NAC can make sure computers have approved Microsoft updates by using a WSUS Requirement. This requirement uses the WSUS API, on the the end computer, to poll the WSUS server for an index of all approved Microsoft updates. The end computer then uses the local Windows Update Agent to compare the local index, called a data store, with the index received from the WSUS server. Any differences would cause the Cisco NAC remediation dialog box to appear and guide the end user through downloading and installing the Microsoft updates.

In theory this should be a seamless process that occurs quickly. In practice, there are a number of problems that can occur. Some common problems are problems connecting to the WSUS server and errors when connecting to the WSUS server. Below are some common tools to use for troubleshooting WSUS problems.

Common Troubleshooting Tools

wuauclt.exe /detectnow - This is a great command to initiate detection of the WSUS server manually. Without this command you need to wait for the Automatic Update process to kick off.
c:\WINDOWS\WindowsUpdate.log - This file provides invaluable logs regarding the status of the Windows update progress.
esentutl.exe - This command is a database utility that can recover and repair the database used, on the end computer. The database is stored in c:\WINDOWS\SoftwareDistribution\DataStore\DataStore.edb
WSUS Client Diagnostics Tool - This tool checks the basic settings required for WSUS to work. The link above provides access to the Microsoft website providing more information about the tool along with a link to download

Here's an example on how the first two tools would be used.

A user is having problems getting Microsoft updates from the WSUS server. You go to the users computer and check out the c:\WINDOWS\windowsupdate.log file. In the file, you notice the following error message

WARNING: WU client failed Searching for update with error 0x8024400e

You run "wuauclt.exe /detectnow" and check the windowsupdate.log file again to make sure the problem is still occurring. After verifying that it still occurring, you do a Google search on "error 0x8024400e" and find a link to a website describing a similar problem and offering a solution. You contact the WSUS team and have them implement the change to fix the problem.

While Google searches are excellent ways of obtaining information about WSUS, I've found a number of links to start your troubleshooting efforts with. Below are the best links I've found to start your research

Basic Cisco wireless setup with Cisco Supportwiki

2008-10-19T18:53:00.006-04:00

Cisco recently introduced wiki pages for their support. I've started using it as one more resource for research and general information. One link that I recently used was to setup basic wireless security for Cisco Access Point.

The instructions were fairly simple, but they got the job done. The instructions were as follows

To configure Wi-Fi Protected Access (WPA) on a Cisco Access Point (AP) without an authentication server, configure the AP with a pre-share key (WPA-PSK).

To configure the WPA-PSK, perform these steps using the GUI interface:

1. In the Encryption Manager window, select cipher TKIP and click Apply.
2. In the Service Set Identifier (SSID) Manager window, perform these steps:
Create an SSID.
Select Open Authentication.
Set the Key Management to Mandatory.
Check the WPA box.
Enter a WPA-PSK and click Apply.

To check out the Cisco Support Wiki Site, go to http://supportwiki.cisco.com

Fun with boolean expressions in Cisco NAC Appliance Rules

2008-10-19T14:05:00.005-04:00

This is an example of how to handle a tricky boolean expression with NAC Appliance rules. I'll lay out a scenario and focus on the rule creation. I'll cover the other aspects in future blog posts.

Let's set the stage. Say you're setting up NAC for remote access VPN users using L3 in-band virtual gateway. The VPN is handled by an ASA and authentication is handled by VPN SSO. There are three classes of users. The three classes of users each have a different VPN profile for connecting to the VPN. These profiles are provided with separate VPN pools on the ASA. This make separating the users into roles fairly straightforward as described below.

With VPN SSO, RADIUS accounting packets are sent from the ASA to the NAC Server (NAS). One RADIUS accounting attribute is called "Framed_IP". This attribute contains the VPN pool IP address of the user. This information is used to map the user into a particular role. In this scenario, the computers used by the remote access users also have registry keys that define which user class they are in.

Now comes the fun part. Most of the users have the correct VPN profile for their user class, but there are some users that have an incorrect VPN profile. We'll call the user classes CLASSA, CLASSB, and CLASSC. How do you block the users, using the incorrect profile, from the network and also provide them with the correct VPN profile?

From a 10,000 foot view, this is an easy task of completing the following steps
1. Define the unique registry keys for CLASSA, CLASSB, and CLASSC
2. Create checks for each registry key and value name
3. Create the rules based on the checks
4. Create the requirements
5. Tie the requirements to the rules to create requirement-rules
6. Tie the requirement-rules to the roles for the different classes of users(ie CLASSA, CLASSB, and CLASSC)

The devil is truly in the details. Steps 1 and 2 are fairly straightforward. Use regedit to find the registry keys and value names. Then create the checks in the NAC Manager (NAM) by navigating to "Device Management > Clean Access > Clean Access Agent > Rules".

Step 3 is where things get interesting. I'll take the rules for CLASSA as an example. The rules will be tied to requirements that ensure that the VPN user is using the correct VPN profile. I'll call the checks created for each class REGA, REGB, and REGC. I'll be creating two rules. Below is a sentence description of what the rules will accomplish

1. For CLASSA, if the registry key on the VPN user's computer matches REGA, then pass the rule. If the registry does not match REGA, but does match REGB, then fail the rule. On rule failure, the requirement tied to this rule will provide the user a download link to download the CLASSB VPN profile. If REGA does not exist and REGB does not exist, pass the rule. This will allow other users that do not match REGA and REGB to flow down to the next rule.
2. For CLASSA, if the registry key on the VPN user's computer matches REGA, then pass the rule. If the registry does not match REGA, but does match REGC, then fail the rule. On rule failure, the requirement tied to this rule will provide the user a download link to download the CLASSC VPN profile. If REGA does not exist and REGC does not exist, pass the rule. This will allow other users that do not match REGA and REGC to flow down to the next rule.
3. This rule blocks access if none of the REGA, REGB, or REGC checks were seen. This would be a catchall rule looking for rogue users trying to access the network with an unapproved computer.

I'll explain the boolean logic behind the first rule. The first thing to keep in mind is that the requirement-rule will only trigger on a failure of the rule. If the rule passes, the the requirement is deemed successful and no remediation is necessary. With that in mind, let's take a look at boolean logic for the first rule. The rule will actually be

REGA or (!REGA and !REGB)

Let's break this out into the individual parts.

Part1: REGA

The first REGA designates that if REGA is not found, then fail the rule.

Part2: (!REGA and !REGB)

The portion within the parentheses succeeds only if REGA is not found and REGB is not found. It is important to have the !REGA because we only want to Remember, what we are really looking for is failure scenarios. Failure occurs in three combinations:

REGA is found and REGB not found
REGA is found and REGB is found
REGA is not found and REGB is found.

The first two combinations will never occur because they would already have passed the expression in Part1. The last combination is the one we want to fail. This means that the requirement remediation dialog box will only appear if REGA fails, but REGB is found.

REGA or (!REGA and !REGB)

Putting it all together, if REGA is found, then the rule will succeed and remediation is not necessary. The immediate success is because of the "or" boolean expression right after the REGA. If REGA is not found, then the second part of the expression, within the parentheses, is evaluated. If this expression succeeds it means that REGB was not found. In this case we're doing this because only want to provide remediation if REGB is found and REGA is not found.

Below is the matrix form of the rule. 1 indicates that the check evaluates true. 0 indicates that the check evaluates to false. In the REGA and REGB columns a 0 indicates that the registry key does not exist and a 1 indicates that the registry key does exist.

REGA	REGB	REGA or (!REGA and !REGB)	Result
0	0	0 or (1 and 1)	1 (pass)
0	1	0 or (1 and 0)	0 (fail)
1	0	1 or (0 and 1)	1 (pass)
1	1	1 or (0 and 0)	1 (pass)

In summary, the most important thing to keep in mind is that you want the rule to fail in order to trigger the requirement remediation.

Limiting Operating Systems Allowed Through Cisco NAC Appliance

2008-09-21T22:41:00.001-04:00

In order to limit the operating systems that are allowed through the NAC Server, configure the "User Pages". "User Pages" are located at "Administration > User Pages", as seen below. This makes sense when the login page is used for user authentication. Users plug in their computers, try to access a webpage, and are then redirected to the login page for their operating system. If their operating system is not defined under "User Pages", then they are denied access to the network.

What is less obvious is how the "User Pages" affect single sign on (SSO) scenarios. With NAC Appliance, there are two widely used single sign methods. The first SSO method is VPN SSO. This is used mostly with remote VPN access where the VPN device sends the NAC Server a RADIUS accounting packet after successful authentication. This allows the NAC Server to accept sessions from the user as successfully authenticated. The second SSO method is AD SSO. This is used mostly for campus deployments. In this method, a user's AD login is recognized by the NAC Server using Kerberos tickets.

In both SSO methods, the login page is never displayed because authentication is handled by SSO. With this in mind, configuring the User Pages is not an intuitive step in the configuration process. In actuality, the User Pages are very important in the configuration of SSO. The "User Pages" still define which operating systems are allowed through the NAC Server. This means that, even if a user successfully completes SSO, they will not be allowed access, through the NAC Server, if their operating system is not defined in "User Pages". Thinking of it another way, this is still the recommended method of blocking unwanted operating systems even when using SSO.

Cisco NAC Appliance and Wildcard SSL Certificates

2008-08-28T12:07:00.007-04:00

The Cisco NAC Appliance 4.1.6 Server Configuration guide clearly states that wildcard SSL certificates are not supported. Below is the associated text that is also a link to the section in the guide

Cisco NAC Appliance does not support "wildcard" certificates.

What is not stated is exactly why this is the case. On the Miami of Ohio mailing list, Nate Austin, provided more detailed information about why wildcard certificates are not supported

Theres actually a valid reason. The client pulls the redirection information out of the certificate Common Name. So if the CN is *.domain.com, it will try to redirect you to that and obviously fail.

I have never personally tried it where the SAN in the cert was the cas name, so I don't know if we can pull the name from there as well, but my instinct says probably not.

Cisco NAC Appliance 4.1.6 upgrade notes

2008-08-15T10:03:00.009-04:00

I'll start by saying that it is imperative to read the release notes, cover to cover, before doing the upgrade. There are a couple of problems I ran into with my first two NAC upgrades. Both problems revolved around one big change in 4.1.6. That change requires the communication between the NAS and NAM to provide mutual SSL certificate authentication. This means that the CA root certificate for the NAS SSL certificate needs to exist on the NAM and the the CA root certificate for the NAM SSL certificate needs to exist on the NAS. Previously, the NAM only authenticated the NAS SSL certificate so you only had to make sure that the CA root certificate for the NAS existed on the NAM. With this new requirement, you also now have to make sure that the NAS SSL certificate supports both SSL server and SSL client attributes. Chris Evans does a pretty good explaining this on his Miami of Ohio Mailing List entry.

The first big problem was that SSL certificates on the NAS and NAM must support SSL client and SSL server attributes. On the Miami of Ohio Mailing List, Rand talked about that issue. I ran into that issue with an Entrust Standard SSL certificate. It turns out that you have to purchase the Entrust Advantage SSL certifcate to get the SSL client and SSL server attribute functionality.

Here's what an SSL public certificate with only the SSL server attribute enabled looks like

Here's what an SSL public certificate with SSL server and SSL client attributes enabled looks like. This is what you want to see.

The second problem I ran into had to do with corruption of the SSL certificate when doing the upgrade. I had a Verisign certificate, which uses an intermediate root CA certificate, on the NAS. I made sure I added the root and intermediate CA certificate onto the NAM. When I did the upgrade the NAS and NAM wouldn't talk. In the NAS and NAM logs there were complaints about invalid chaining certificate. I checked the Trusted Certifcate Authority on the NAS and the NAM and made sure the intermediate and root CA Verisign certificate existed on both. I ended up solving the problem by re-inputting the private key and CA-Signed Certificate on the NAS. Once I did that and rebooted everything worked fine. I also saw in the 4.1.6 NAS config guide that the cacerts file can get corrupted. That may have been what happened during the upgrade. The config guide recommends the following

If you check nslookup and date from the CAS, and both the DNS and TIME settings on the CAS are correct, this can indicate that the cacerts file on the CAS is corrupted. In this case, Cisco recommends backing up the existing cacerts file from /usr/java/j2sdk1.4/lib/security/cacerts, overriding it with the file from /perfigo/common/conf/cacerts, then performing “service perfigo restart” on the CAS.

How to remote control a computer connected through a VPN client connection

2008-07-14T10:36:00.010-04:00

I've run into numerous cases where a user is successfully connected via a Cisco VPN client and is having application problems. The helpdesk would like to get into the user's computer to diagnose the problem. Since they have a valid VPN tunnel, you'd think they'd just be able to remote desktop into the user's computer and take a look. Unfortunately, as soon as you remote desktop into their computer, you get a screen saying you'll have to kick them off. When you kick them off, you're also killing the VPN connection.

In order to get around that limitation, I know of two options. The first option is to have the user install VNC Server. This would allow the helpdesk to use a VNC client to remote into their computer. This option requires that the user has admin right on the computer in order to install VNC Server.

The second option is preferable because it does not require admin rights and is already built into Microsoft WinXP. The method involves using "Remote Assistance". The process is to

Have a user create a file
Email the file to the helpdesk
Have the help desk download the file from the email
Have the help desk double click on the file to open a connection to the users computer.

The remote assistance program is located at Start->All Programs->Remote Assistance, as seen below

This brings up the Remote Assistance wizard. The user should click on "Invite someone to help you"

In the next screen click Continue

The next screen gives the option of defining a password that the helpdesk has to input before being allowed remote access

Next the user saves the file

Finally, the user emails the file to the helpdesk. The helpdesk downloads the email attachment and double clicks on the file to launch it. After the file is launched, it opens a remote desktop session to the user's computer.

Adding static routes for Cisco NAC Manager and Profiler

2008-07-13T12:04:00.006-04:00

The NAC Manager and Profiler don't have a documented way of adding a static route, in addition to the default gateway. In most cases this is fine because all traffic follows the default gateway. When doing a pilot or setting up a lab environment there's a greater possibility of needing a static route to direct some traffic another direction than the default gateway.

Since the NAC products are built on Fedora Core, you can use the standard way of adding default routes within Fedora. Modifying the routing table requires root access, so make sure you are logged in as root or type "su -" to elevate to root privileges. Assuming eth0 is used for the traffic, you would create a file called "route-eth0" in the "/etc/sysconfig/network-scripts" directory. Here's an example of the contents of the file assuming you want to route the 192.168.0.0/16 subnet to 10.1.1.110

GATEWAY0=10.1.1.110
NETMASK0=255.255.0.0
ADDRESS0=192.168.0.0

As you probably figured out, you can add additional entries for GATEWAY1, NETMASK1, and ADDRESS1 to add additional static routes.

Once you've created this file, you can apply it in one of two ways. The safest way is to reboot the device with "shutdown -r now". The second way is to just restart the routing process with "service network restart"

Ports required for AD SSO

2008-07-08T21:06:00.004-04:00

When configuring NAC for AD SSO, the last place you'd think to look would be the documentation, right? We'll, this would be one time that it makes a lot of sense to RTFM. On page 9-7 of the PDF version of the 4.1.3 Clean Access Server Installation and Configuration Guide they have all the ports required for AD SSO.

Here are the TCP ports required, in the unauthenticated role, for AD SSO to work: 88, 135, 389, 445, 1025, and 1026.

The one thing that isn't listed in the documentation is that ICMP is also required. Part of the login process includes trying to ping the AD server. If this fails, then AD login doesn't work

CSA Basic Building Blocks

2008-07-07T16:38:00.000-04:00

CSA is a very powerful tool to enforce the security policy for a company. It has a very structured approach to create a security policy that is enforced through the CSA Agents. In order to optimize its use, it is important to understand the fundamental building blocks involved with turning the written security policy into an actionable enforcement tool.

I view the building blocks in two separate parts. The first part is creating the actions that will be used to enforce the security policy. The second part is defining the different types of computers, such as desktops and servers, that have the same type of characteristics. Once these two parts are created, they are linked together so that the correct actions are linked to the appropriate types of computers.

The first part involves creating three objects that build upon each other: rules, rule modules, and policies. The first object is called a rule. This is the basic if/then action that determines enforcement. An example would be, "if an application tries to open a cmd.exe shell, deny and log the access". In addition to denying access, there are a number of different actions that can be taken. The diagram below shows the different actions available. The diagram is important, because, in many places within CSA, the icons associated with the actions are shown without the actual names.

The second object is called a rule module. A rule module combines multiple rules together that all pertain to the same operating system and provide the same type of functionality. Rule modules are then combined into a policy. The policy should contain all aspects that cover the security policy for a particular group of computers (ie desktops or servers). Unlike the rule modules, the policies are not restricted to pertaining to a single operating system. That completes the first part.

The second part is defining the types of computers. CSA calls these groups. These groups break up the computers based on operating system and other logical criteria such as function and business group. Additionally, CSA parameters, such as polling interval, alerts, and events, can be defined for the group instead of for individual hosts.

The last step is to tie the policies created to the groups. This creates an enforceable security policy for the different types of computers in the network.

AVG 8.0 will be fully supported in NAC 4.1.6

2008-06-25T17:47:00.000-04:00

Currently NAC only supports installation checks for the paid version of AVG 8.0. The free version and definition file checks will be supported in version 4.1.6. From what I've been told, this version should be coming out sometime in July.

Resetting NAC Manager database

2008-06-17T11:35:00.000-04:00

I've been writing some NAC labs and I wanted to figure out the best way to clear out the database and start from scratch. I found the instructions in the /perfigo/dbscripts/README file on the NAC Manager. Here are the relevant commands to clear out the database and start from scratch

To remove perfigo database issue:
-----------------------------
su -l postgres -c "psql -h 127.0.0.1 -p 5432 controlsmartdb < /perfigo/dbscripts/pg_droptable.sql"
su -l postgres -c "dropdb -h 127.0.0.1 -p 5432 controlsmartdb"
To install perfigo database issue:
-----------------------------
su -l postgres -c "createdb -h 127.0.0.1 -p 5432 controlsmartdb"
su -l postgres -c "psql -h 127.0.0.1 -p 5432 controlsmartdb < /perfigo/dbscripts/pg_createtable.sql"

*Note: Running the commands will remove the license file as well, so make sure you have the NAC Manager and Server license files before running the commands

Solution to slow CAM login

2008-06-14T08:34:00.002-04:00

I just saw this in the 4.1(1) release notes. It's resolved caveat CSCsi23228. I haven't had to use it but it may be useful someday if I run into slow CAM login time
http://www.cisco.com/en/US/docs/security/nac/appliance/release_notes/411/411rn.html

CAM database performance degraded over time

Clean Access Manager performance degrades over time, users may experience slowness during login process and CAM web administration interfaces. The slowness may start to exhibit itself after an extensive number of database delete/insert/modify operations.

There are three workarounds for this issue which can be applied under different conditions.

Workaround 1

This can be applied during maintenance window when CAM is not in service. Note that this may take up several minutes, please do not interrupt the process.

1. service perfigo stop
2. su -l postgres
3. vacuumdb -h 127.0.0.1 -a -f
4. exit
5. service postgresql restart
6. service perfigo start

Workaround 2

This can be applied when system is in service with light load. Note that this may take up several minutes, please do not interrupt the process.
1. su -l postgres
2. vacuumdb -h 127.0.0.1 -a -f
3. exit

Workaround 3: This can be added as system daily cron job to prevent the potential slowness.

1. Create a file named "db_vacuum.sh" under "/etc/cron.daily" with the following content:
#!/bin/sh
su - postgres -c "vacuumdb -h 127.0.0.1 -a -f"
2. cd /etc/cron.daily
3. chmod +x db_vacuum.sh

DMVPN with NAT

2008-06-13T08:47:00.003-04:00

It looks like Cisco has been fixing NAT issues with DMVPN. They fixed the NAT issue for spokes talking to the hub using NAT traversal. This is the same method that VPN clients use. It uses UDP port 4500 to send the IPSec traffic instead of IP protocol 50 (ESP) and IP protocol 51 (AH). Here's a link with more explanation.
http://www.cisco.com/en/US/docs/ios/security/configuration/guide/dmvpn_dt_spokes_b_nat.html

In versions after 12.4(6)T, the spoke-to-spoke traffic with NAT is supported. Take a look at this link for more information.
http://www.cisco.com/en/US/docs/ios/12_2t/12_2t13/feature/guide/ftgreips.html#wp1039515

Here's the important information from the link

In Cisco IOS Release 12.4(6)T or earlier, DMVPN spokes behind NAT will not participate in dynamic direct spoke-to-spoke tunnels. Any traffic to or from a spoke that is behind NAT will be forwarded using the DMVPN hub routers. DMVPN spokes that are not behind NAT in the same DMVPN network may create dynamic direct spoke-to-spoke tunnels between each other.

In Cisco IOS Release 12.4(6)T or later releases, DMVPN spokes behind NAT will participate in dynamic direct spoke-to-spoke tunnels. The spokes must be behind NAT boxes that are preforming NAT, not PAT. The NAT box must translate the spoke to the same outside NAT IP address for the spoke-spoke connections as the NAT box does for the spoke-hub connection. If there is more than one DMVPN spoke behind the same NAT box, then the NAT box must translate the DMVPN spokes to different outside NAT IP addresses. It is also likely that you may not be able to build a direct spoke-spoke tunnel between these spokes. If a spoke-spoke tunnel fails to form, then the spoke-spoke packets will continue to be forwarded via the spoke-hub-spoke path.

I tried this out in a Dynamips lab and it worked great.

Here's a diagram of the dynagen lab I created with the relevant config

How does Cisco NAC change your DHCP IP

2008-06-11T23:03:00.002-04:00

When implementing NAC you may wonder how it changes your IP when you move back and forth betwen the untrusted and trusted VLANs. Back in the olden days, the only way to do this was to bounce the switch port. This caused the link to go down on the connected computer which kicked off a new DHCP request. Nowadays there's a method that works better when the switch port has an IP phone and a computer on the same