Radius – Server 2008 R2 NPS

We’ve OLYMPUS DIGITAL CAMERA         been using NPS on Server 2008 for a while now and its been perfect for handling 802.1x authentication (EAPTLS) and radius auth from the HP WESM in the 5400zl. The radius setup for the HP Wireless Edge Services was pretty easy, it only needs radius clients for the Primary WESM and any Redundant WESM’s.

Now that we’re adding another 50-70 E-MSM422 AP’s for the MSM765 controller we need to add radius clients for each AP. After a conversation with Adam (@DJADSA) we worked out that we were going to hit the 50 radius client limit in Server 2008 Standard. Adam showed me a couple of neat tricks with their NPS configuration that would save us a tonne of time and are new additions to R2!

The first trick was adding a subnet range for Radius Clients instead of adding a radius client for AP individually. Adding the IP/CIDR and shared secret will let all devices in the range talk to the NPS server.

radiusclient

The next tip from Adam was with the Accounting in NPS. We’d tried to get SQL logging to behave in Server 2008 a few times and failed miserably. The NPS application in 2008 would connect to a SQL database but wouldn’t create the structure etc. There was a sql script on the web that would create it for you but we didn’t have any luck getting it all to work properly. 2008 R2 has a new wizard for setting up NPS accounting and the final stage of the wizard gives you the option of creating the SQL structure of the database. very tidy

accountingwiz

With SQL logging enabled it give us the option of writing a web part or two for SharePoint to let staff know which users are connected where, and lets us easily run scripts to find client/authentication problems.

MSM765 SNTP Time Sync

msm765

This week we’ve been reconfiguring our MSM765 wireless controller and adding some new features for Students and guests to the School. With our old ZL WESM we were able to have a VLAN on the wireless network with an HTML based login, which allowed the students to use their own machines on the wireless network with their AD credentials. We wanted to replicate this setup on the MSM by using HTML-based user logins and still use their AD logins. We hit a problem when we tried to configure the Active Directory Authentication on the controller because the time on the controller wasn’t in sync with the 5400 chassis or the domain!

When we check Controller –> Management –> System Time, we could see the the time was incorrect, but there was no option to change it or specify an NTP server. The command ling reference for the controller (MSM7xx-CLI-RG-May09-5992-5933.pdf) gave a few clues on how to set the SNTP server and get the controller connected to our Windows time server

Connecting the terminal to the 5400 with the controller (MSM is in Bay I)

BGSCore(config)# services  I 2
BGSCore(msm765-application-I)> enable
BGSCore(msm765-application-I)# conf
BGSCore(msm765-application-I)(config)#
BGSCore(msm765-application-I)(config)# ntp protocol sntp
BGSCore(msm765-application-I)(config)# ntp server 1 192.168.1.19
BGSCore(msm765-application-I)(config)# ntp server
BGSCore(msm765-application-I)(config)#

The time sync’d straight away and made the connection to AD without a hitch

image

ProCurve – Front-Panel Security & Authentication

I was looking for some ProCurve documentation on AAA security and stumbled across the Hardening ProCurve Switches White Paper and found a few nice things to add to our ProCurve config.

Password Clear Protection – Front-Panel Securitylogo_procurve_networking_by_hp
ProCurve devices utilize the Reset and Clear buttons on the front panel to help users reset the switch configuration to factory default or to reset the console password. This capability creates a security risk anywhere it’s impossible to  prevent physical access to the switch. ProCurve makes it possible to disable this functionality to protect from malicious use of these features.

There are two components to front-panel security: “password clear” and “factory reset.” Both must be disabled to fully secure the device.

In the switch’s default mode, a malicious user can utilize the front-panel clear button to reset a console password stored locally on the switch. To disable this feature, issue the command:

ProCurve Switch(config)# no front-panel-security password-clear

The other capability built into ProCurve switches is the ability to reset the switch configuration to the factory default mode:

ProCurve Switch(config)# [no] front-panel-security factory-reset

Executing this command prevents reset of the switch configuration by use of the front-panel Reset and Clear buttons.

It’s critical to understand that disabling these features severely restricts administrator options if the password is lost or forgotten. Before making these changes, users are strongly encouraged to review all considerations outlined in the Access and Security Guide for your model.
wireless_edge_services_zl_module
Authentication – Server-Supplied Privilege Level
Login privilege level instructs the switch to accept the authenticating user’s command level (manager or operator) that is supplied by the server. This allows manager-level users to skip the login context and proceed immediately to enable context, thus eliminating the need for a manager-level user to login twice.

To allow the switch to accept the privilege level provided by the server, use the following configuration command:

ProCurve Switch(config)# aaa authentication login privilege-mode

To supply a privilege level via RADIUS, specify the “Service-Type” attribute in the user’s credentials.
• Service-Type = 6 allows manager-level access
• Service-Type = 7 allows operator-level access
• A user with Service-Type not equal to 6 or 7 is denied access
• A user with no Service-Type attribute supplied is denied access when privilege mode is enabled

– The Radius Authentication for switch access sounds interesting. If our Staff are using their network credentials to access the switch config, or contractors that are working on the network, we can easily enable/disable their access to the switches without hassle and letting everyone know the Manager/Operator passwords

Mic Check // Oh Wait a minute now…

Over the last 12 months since we’ve installed our J9051A ZL Wireless Edge Services Module (WESM), we’ve had some intermittent issues with some of our wireless notebooks causing the WESM to freak out and run at 100% CPU and boot all of our wireless stations of the Wireless network. These notebooks work fine on the WLAN 99% of the time but every now and then they freak out and cause problems with the TKIP integrity check.

The notebooks that we’ve had trouble with have had either an Intel or Broadcom wireless NIC:

  • Acer 3230 with WLAN: integrated Intel® PRO/Wireless 2200ABG
  • HP 1100 Tablet with WLAN: Intel PRO/Wireless LAN 2100 3B Mini PCI
  • Motion Computing Tablet with WLAN: Broadcom Wireless

We have many other 3230’s on the WLAN and one other Motion tablet that are identical to the machines that we’ve had issues with, but no other machines have caused the TKIP failure. We’ve updated drivers for the wireless NICs and installed all Windows updates and still haven’t been able to correct the problem?

When the TKIP failure occurs the CPU on the WESM hits 100%, see below, while it tries to perform the TKIP Countermeasures. TechDuke has a great explanation of the TKIP Message Integrity Check (MIC), and explains that when a wireless station fails the MIC, or Michael, hash check twice within 60 seconds then all wireless stations are booted off the wireless network for a minute and forced to reconnect/re-authenticate. Zack de la Rocha was way ahead of his time when he wrote Mic Check, he explains the MIC failure perfectly…

Rage Against The Machine – Mic Check (The Battle of Los Angeles (1999))
Mic Check
Oh Wait a minute now
Ha ha ha
Come on
Wait a Minute Now
Check

WESM_CPU_Usage
The Diagnostic page on the WESM showing that the CPU had been running at 100%, as soon as we disabled Dial-in access in Active Directory for the offending notebook we broke the Radius authentication and the WESM went back to running as normal.

Wireless Edge Service Log

   1: Feb 06 11:27:46 2009: %CC-4-TKIPCNTRMEASSTART: TKIP countermeasures started on wlan 1
   2: Feb 06 11:28:15 2009: %MGMT-4-OTHERREQQUED: request queued in delegated requests
   3: Feb 06 11:28:46 2009: %CC-4-TKIPCNTRMEASEND: TKIP countermeasures ended on wlan 1
   4: Feb 06 11:28:47 2009: %CC-4-TKIPMICCHECKFAIL: TKIP message integrity check failed in frame on wlan 1
   5: Feb 06 11:28:47 2009: %KERN-3-ERR: mic check failure <00-13-CE-04-FB-4A>. (pkt_len 360 prio: 0) rx: <2B-2B-02-DC-00-FF> calc: <A7-CE-2B-B8-45-C6>.
   6: Feb 06 11:28:51 2009: %CC-4-TKIPMICCHECKFAIL: TKIP message integrity check failed in frame on wlan 1

Checking the message log on the WESM, we could identify which machine was failing the MIC and which WLAN they were connected to. Line 5 shows the MIC check failure and the MAC address of the offending machine.

dhcp

We traced the offending MAC address back to its owner via the DHCP console and disabled Dial-in access for the computer and on the user account of its owner. This causes WLAN Radius authentication to fail the EAP-TLS auth because a valid certificate and dial-in access are required for access to that particular WLAN.

Currently we’re still running the original firmware wt.01.03 that came with the ZL module, but will update to wt.01.15 shortly and test these machines on the WLAN to see if the updated firmware can handle integrity check failure with a little more grace than the original firmware.

HP Case Study – Ballarat Grammar

thm_ballarat_grammerHewlett Packard have just release a Case Study on the hard work that we’ve done over the last year or two with them, and our HP reseller Trident. Check it out here.

The Case Study explains some of the challenges that we had with our previous network, servers and workstations, and how HP and Trident helped us find a solution that would help us overcome these challenges.

As well as the HP Case Study, earlier this year HP ProCurve released a Case Study on our migration from Cisco/Alloy switches to ProCurve, Wireless Edge Services and 802.1x port security. The Press Release Foundersis here on CIO. With a lot of help from Lisa and Fotios we’ve been able to develop our network, provide a 10Gb fibre backbone and much improved services for Staff and Students, including our Boarders on the wireless network, using the Wireless Edge Services Module (WESM). Expect to see some How-To’s posted soon on some of the funky things that we setup with our HP gear, especially the 802.1x, the WESM and our guest wireless network.

All of the work discussed in these case studies is obviously a team effort, and wouldn’t have been possible without the hard work and planning from Des, Nathan B, Nathan H, Leon and Rian.

* Photo of Ballarat Grammar by Rob Olston

Scripting Switch Configuration Backup

Here’s is a short VBS script which telnet’s into your ProCurve switch and sends a config backup to your TFTP Server. The code can easily be changed to telnet into pretty much any device that supports configuration via telnet.

This code is a modified version of a snippet posted on the VBForums

Dim objShell
Dim objNetwork

Set objNetwork=CreateObject("WScript.Network")

strTitle="Telnet Demo"
strDefaultServer="Server01"
strDefaultUser=objNetwork.UserDomain & "\" & objNetwork.UserName
strDefaultPassword="P@ssw0rd"

strComputer=InputBox("What server or device do you want to connect to?",_
strTitle,strDefaultServer)
If Len(strComputer)=0 Then WScript.quit



strPassword="password"

Set objShell=CreateObject("wscript.shell")
'Start Telnet
objShell.Run "Telnet " & strComputer
'Give app a chance to get started
WScript.Sleep 5000
objShell.AppActivate "Telnet " & strComputer

'Send login credentials
objShell.SendKeys strUsername & "~"
WScript.Sleep 2000
objShell.SendKeys strPassword & "~"
WScript.Sleep 2000

'Send commands
WScript.Sleep 200
objShell.SendKeys "~"
WScript.Sleep 200
objShell.SendKeys "copy startup-config tftp 172.16.1.15 "& strComputer &"_config.txt"
WScript.Sleep 1000
objShell.SendKeys "~"

'give lengthy commands time to finish
'WScript.Sleep 10000

'make sure we get window again
objShell.AppActivate "Telnet " & strComputer
'run another command
'objShell.SendKeys "net share"
'WScript.Sleep 200
objShell.SendKeys "~"

'Close session
'make sure we get window again
objShell.AppActivate "Telnet " & strComputer
objShell.SendKeys "exit"
WScript.Sleep 200
objShell.SendKeys "~"
objShell.SendKeys "exit"
WScript.Sleep 200
objShell.SendKeys "~"
WScript.Sleep 200
objShell.SendKeys "y"
objShell.SendKeys "~"
WScript.Sleep 200
objShell.SendKeys "~"

Save this text in a .vbs file and run via “cscript switchbak.vbs“ from the command line

If your modifying the script to run on something other than a ProCurve switch ,you may have to tweak/add/remove the Sleep and SendKeys ”~” . Using the SendKeys and the tilde will send a carriage return to the telnet session

Also, if you’d like to run this script you will need a TFTP server, free download from Solawinds

To make the back script a useful tool I’ve set it to run and query an MS SQL database to get the addresses for our ProCurve switches and have scheduled it to run regularly. I will post an update with how I’ve set that up sometime soon. The Script is essentially the same but I’ve removed the prompt for the device address and added the database connection query, will post the details shortly

If you have setup the ProCurve Manager then you may find this post redundant, but, I’ve found it to be a handy script to have, and send backup config files to a server where they are easily accessible in a disaster, hopefully…

Daylight Saving, ProCurve and Time Synchronisation

Setting the new start and end dates for day light saving on your switches before daylight saving started would have been excellent planning and forward thinking. However, if your like me and didn’t remember until you rolled into work on Monday morning, three weeks ago, then here are the commands to get your switches back under control.

If you have a time server on your network you can set your switches to sync with it, otherwise you can set the switches to sync with an external time server if your firewall allows.

switch# conf

switch# sntp server ntp.cs.mu.OZ.AU

switch# timesync sntp

switch# sntp unicast

switch# time daylight-time user-defined begin-date 5/10 end-date 5/4

switch# wr mem

If you have a time server on your network, substitute ntp.cs.mu.OZ.AU from line 2 with the name or IP of your time server

Error tracking can be tedious at the best of times, but if the time on your network and switches is correct and consistent across devices then the job is easier for you and your team.