Cisco LAP1042 and WLC-2106

I was called into a client where their WLC-2106 had failed.  They had no documentation other than passwords.  The Radios where LAP1042.  I installed a new WLC-2106 and I accessed the management console.  None of the LAP1042 show up in the access points.  I have been trying to figure this out for a couple hours with no luck.

I knew the management IP address was 10.1.3.2 and the virtual interface was 1.1.1.1  However, I did not know the AP-Interface and I tried to guess what it might be,  The network has VLANs and the APs are on VLAN3.  Is my problem with the AP-Interface.  I tried assigning it an IP address on the 10.1.3.0 network.

Any help would be appreciated
gmerilliAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Ken BooneNetwork ConsultantCommented:
Well the APs connect to the AP-Interface so having that right is key.  You can try a few things here:
Console in to one of the APs and reboot it.  While it is rebooting you can watch the console messages to see what it is doing.  You should see the IP address that the AP is trying to establish a session with.

Also they APs might figure out the WLC via DNS. Check and see if there is a host entry for cisco-capwap-controller - i think there is another variation to that.  Check that.

Also here is  a doc that goes over troubleshooting APs not connecting:

http://www.cisco.com/c/en/us/support/docs/wireless/5500-series-wireless-controllers/119286-lap-notjoin-wlc-tshoot.html
Istvan KalmarHead of IT Security Division Commented:
Hi,

Did you set the NTP or local time on WLC? Correct time is mandatory for registration.
If the AP-s not came up please set mac authorization on WLC for AP-s:

http://www.cisco.com/c/en/us/support/docs/wireless/4400-series-wireless-lan-controllers/98848-lap-auth-uwn-config.html
gmerilliAuthor Commented:
Reset to factory.  still nogo..not seeing any APs.  using local time not NTP.  Message log has a number of errors(see below).  looking for where to set mac authorization(not sure).

messages in message log:
Oct 26 12:34:02.704 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'cliWebInitParms.cfg'
Oct 26 12:34:02.362 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'rfidInitParms.cfg'
Oct 26 12:34:02.220 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'dhcpParms.cfg'
Oct 26 12:34:02.175 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'bcastInitParms.cfg'
Oct 26 12:34:01.107 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'rrmInitParms.cfg'
Oct 26 12:34:00.875 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'apfInitParms.cfg'
Oct 26 12:34:00.511 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'mmInitParms.cfg'
Oct 26 12:34:00.420 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'aaaapiInitParms.cfg'
Oct 26 12:34:00.416 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'pemInitParms.cfg'
Oct 26 12:34:00.383 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'dot1xInitParms.cfg'
Oct 26 12:33:50.211: phy.c 155: link on port 1 change to up
Oct 26 12:33:50.205: phy.c 155: link on port 0 change to up
Oct 26 12:33:49.111 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'sshpmInitParms.cfg'
Oct 26 12:33:49.110 cnfgr.c:1823 CNFGR-3-INV_COMP_ID: Invalid Component Id : Unrecognized (36) in cfgConfiguratorInit.
Oct 26 12:33:49.108 spam.c:449 LWAPP-2-SEM_CREATE_ERR: Could not create semaphore for notifying AP registration
Oct 26 12:33:49.097 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'spamInitParms.cfg'
Oct 26 12:33:49.040 acl.c:244 ACL-3-ENTRY_DONOT_EXIST: Unable to find an ACL by name "".
Previous message occurred 2 times.
Oct 26 12:33:49.039 sim.c:272 SIM-3-INVALID_PORT: Using invalid port number. Port out of range. Port # 0
Oct 26 12:33:49.037 acl.c:244 ACL-3-ENTRY_DONOT_EXIST: Unable to find an ACL by name "".
Oct 26 12:33:49.034 acl.c:244 ACL-3-ENTRY_DONOT_EXIST: Unable to find an ACL by name "".
Oct 26 12:33:49.033 acl.c:244 ACL-3-ENTRY_DONOT_EXIST: Unable to find an ACL by name "".
Oct 26 12:33:48.893 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'simInitParms.cfg'
Oct 26 12:33:48.892 acl_cfg.c:309 ACL-3-LOAD_CONF_FAILED: Acl config is corrupted. Resetting Acl config to default.
Oct 26 12:33:48.770 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'dot1dInitParms.cfg'
Oct 26 12:33:48.751 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'fdbInitParms.cfg'
Oct 26 12:33:47.838 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'snmpInitParms.cfg'
Oct 26 12:33:47.701 sim_util.c:1273 SIM-3-FILE_READ_FAIL: Failed to read config file /proc/ide/ide1/hdd/capacity.
Oct 26 12:33:47.672 sim_util.c:2131 SIM-3-QOS_ACM_DOT1BG_FAIL: Unable to set uranium Admission Control (ACM)
Oct 26 12:33:47.671 sim_util.c:2124 SIM-3-QOS_ACM_DOT1A_FAIL: Unable to set uranium Admission Control (ACM)
Oct 26 12:33:47.671 sim_util.c:2131 SIM-3-QOS_ACM_DOT1BG_FAIL: Unable to set platinum Admission Control (ACM)
Oct 26 12:33:47.670 sim_util.c:2124 SIM-3-QOS_ACM_DOT1A_FAIL: Unable to set platinum Admission Control (ACM)
Oct 26 12:33:47.669 sim_util.c:2131 SIM-3-QOS_ACM_DOT1BG_FAIL: Unable to set gold Admission Control (ACM)
Oct 26 12:33:47.669 sim_util.c:2124 SIM-3-QOS_ACM_DOT1A_FAIL: Unable to set gold Admission Control (ACM)
Oct 26 12:33:47.668 sim_util.c:2131 SIM-3-QOS_ACM_DOT1BG_FAIL: Unable to set bronze Admission Control (ACM)
Oct 26 12:33:47.668 sim_util.c:2124 SIM-3-QOS_ACM_DOT1A_FAIL: Unable to set bronze Admission Control (ACM)
Oct 26 12:33:47.667 sim_util.c:2131 SIM-3-QOS_ACM_DOT1BG_FAIL: Unable to set silver Admission Control (ACM)
Oct 26 12:33:47.667 sim_util.c:2124 SIM-3-QOS_ACM_DOT1A_FAIL: Unable to set silver Admission Control (ACM)
Oct 26 12:33:47.636 sim_config.c:194 SIM-3-INTFGET_GIG_ETH_FAIL: Failed to get the Interface number of the Gigabit Ethernet Port
Oct 26 12:33:47.625 sim_config.c:194 SIM-3-INTFGET_GIG_ETH_FAIL: Failed to get the Interface number of the Gigabit Ethernet Port
Oct 26 12:33:47.619 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'simInitParms.cfg'
Oct 26 12:33:46.569 nvstore.c:304 SYSTEM-3-FILE_READ_FAIL: Failed to read configuration file 'dtlInitParms.cfg'
SolarWinds® IP Control Bundle (IPCB)

Combines SolarWinds IP Address Manager and User Device Tracker to help detect IP conflicts, quickly identify affected systems, and help your team take near instantaneous action. Help improve visibility and enhance reliability with SolarWinds IP Control Bundle.

Ken BooneNetwork ConsultantCommented:
So show us the messages you get while its booting up.  Console in to the AP, start logging in your terminal program, and then physically power off the AP and then power it back on.  It would be helpful to see the initial boot up messages.  That is where it initially tries to contact the WLC and we can see what methods it is using etc..
gmerilliAuthor Commented:
How do you access the AP.  I can ping the IP addresses but no telnet or http.
Ken BooneNetwork ConsultantCommented:
You need to use a console cable and connect to it from your serial port or your usb port with a usb to serial adapter.
gmerilliAuthor Commented:
Pulled AP off ceiling and consoled into AP.

This is what is repeatedly happening:
*Mar  1 00:18:16.561: %CAPWAP-3-ERRORLOG: Invalid event 38 & state 2 combination.
*Mar  1 00:18:16.683: %DHCP-6-ADDRESS_ASSIGN: Interface GigabitEthernet0 assigned DHCP address 10.1.3.100, mask 255.255.255.0, hostname AP1-1cdf.0fb5.25b4

*Mar  1 00:19:05.559: %CAPWAP-3-DHCP_RENEW: Could not discover WLC using DHCP IP. Renewing DHCP IP.
Not in Bound state.
*Mar  1 00:19:14.066: %CAPWAP-3-ERRORLOG: Invalid event 38 & state 2 combination.
*Mar  1 00:19:14.188: %DHCP-6-ADDRESS_ASSIGN: Interface GigabitEthernet0 assigned DHCP address 10.1.3.100, mask 255.255.255.0, hostname AP1-1cdf.0fb5.25b4

cisco-capwap-controller is resolving


AP1-1cdf.0fb5.25b4>Ping cisco-capwap-controller

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.3.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
Ken BooneNetwork ConsultantCommented:
So I see where you can resolve the cisco-capwap-controller to 10.1.3.2.   In the boot up sequence, there are several methods the AP uses to find the controller.    Do you ever see in the messages where the AP trys to look up CISCO-CAPWAP-CONTROLLER after it gets its DHCP ip address assignment?
Ken BooneNetwork ConsultantCommented:
Also what is the device that is handling DHCP for the APs?   Is it a windows dhcp server or is it a cisco device?
gmerilliAuthor Commented:
This may be a DHCP issue.  I have two switches(10.1.1.2 and 10.1.1.11).  DHCP server defined on 10.1.1.2:

ip dhcp excluded-address 10.1.3.1 10.1.3.99
ip dhcp excluded-address 10.1.3.111 10.1.3.255
!
ip dhcp pool AP
   network 10.1.3.0 255.255.255.0
   default-router 10.1.3.1
   dns-server 10.1.1.20
   domain-name net2-pc.local
   option 60 ascii "Cisco AP c1040"
   option 43 hex f104.0a01.0302
!

10.1.1.11 :

!
ip dhcp-server 10.1.1.2
vtp mode transparent
!
!
interface GigabitEthernet1/0/1
 description Uplink-2-APs
 switchport access vlan 3
 switchport mode access
 mls qos trust dscp
!
interface GigabitEthernet1/0/2
 description Uplink-2-APs
 switchport access vlan 3
 switchport mode access
 mls qos trust dscp
!
interface GigabitEthernet1/0/3
 description Uplink-2-APs
 switchport access vlan 3
 switchport mode access
 mls qos trust dscp
!
Access points connected to interfaces are not getting IP addresses.  Any Ideas as to what I am missing?
gmerilliAuthor Commented:
Oh..I set up an internal DHCP server on the WLC-2106.  That is the only way to get the APs to get IP addresses.  But I do not know how to set up option 60 and 43.
Ken BooneNetwork ConsultantCommented:
Ok so it looks like you were setup for dhcp on the switch initially.  That part looks good.  The internal DHCP server on the WLC-2016 is normally used for clients - not APs.  So if you have a scope there for the APs I would remove it.  

From your initial logs it looks like they were getting an IP.. I see 10.1.3.100 assigned to the device.

Was this configured like this already?
ip dhcp pool AP
   network 10.1.3.0 255.255.255.0
   default-router 10.1.3.1
   dns-server 10.1.1.20
   domain-name net2-pc.local
   option 60 ascii "Cisco AP c1040"
   option 43 hex f104.0a01.0302

Cause option 43 is what gives the AP the WLC IP address.  What is configured for option 43 is 10.1.3.2.  So that means 10.1.3.2 should be the AP-Management address.  If you are using something different for the AP-Management address then you need to have that reflected in option 43.  The IP address is the last 4 bytes in option 43.. 0a01.0302 = 10.1.3.2.
gmerilliAuthor Commented:
Originally the only thing that was in the DHCP pool was the network and default router.I added the dns-server; domain name; and options 60 and 43.  10.1.3.2 is the management interface.  The APs did not get an IP address until I added the scope to the WLC-2106.  I have since disabled this trying to get the AP to get its IP address from the Cisco Switch and priming it with option 43.  The AP come back and say that they could not get an IP address.  I would be OK with just hard assigning everything in the AP. But the AP console does not use the same CISCO IOS commands and I am at a loss on how to do this.  I even set up a separate DHCP pool to hard assign 10.3.1.100 to the MAC address of the AP and that did not work.
Ken BooneNetwork ConsultantCommented:
So when you posted this:

Pulled AP off ceiling and consoled into AP.

This is what is repeatedly happening:
*Mar  1 00:18:16.561: %CAPWAP-3-ERRORLOG: Invalid event 38 & state 2 combination.
*Mar  1 00:18:16.683: %DHCP-6-ADDRESS_ASSIGN: Interface GigabitEthernet0 assigned DHCP address 10.1.3.100, mask 255.255.255.0, hostname AP1-1cdf.0fb5.25b4

*Mar  1 00:19:05.559: %CAPWAP-3-DHCP_RENEW: Could not discover WLC using DHCP IP. Renewing DHCP IP.
Not in Bound state.
*Mar  1 00:19:14.066: %CAPWAP-3-ERRORLOG: Invalid event 38 & state 2 combination.
*Mar  1 00:19:14.188: %DHCP-6-ADDRESS_ASSIGN: Interface GigabitEthernet0 assigned DHCP address 10.1.3.100, mask 255.255.255.0, hostname AP1-1cdf.0fb5.25b4


That was after you configured DHCP on the WLC?

We need to figure out why the AP is not getting dhcp from the switch.  Can you post the layer 3 interface for vlan 3 and the ports that connect the two switches config?
gmerilliAuthor Commented:
OK...not sure what is wrong with core switch DHCP.  tested and core switch DHCP not giving out IP addresses.  Probably restart would fix problem.  But restart takes everything down.  Put DHCP server on same switch as APs.  APs are now getting IP address. used option 60 and 43 as above.  Can Ping APs from controller and vice versa.  APs still not registering with controller.  Wanted to download Firmware for controller...but Cisco requires that I purchase a service contract to download firmware(argh!!!!!!).   Controller has same messages as listed above on start up.  I tried to look up the error messages with and did not get a lot of information,  AP tries to connect to controller about 5 times and then asks DHCP for a new address.

Anyone know where I could download firmware for wlc2106 without buying a service contract from cisco...it also looks like this Controller may have old firmware(V4) and the APs have newer firmware(v7).
Ken BooneNetwork ConsultantCommented:
Sorry - not gonna be able to get the firmware without maintenance.
gmerilliAuthor Commented:
OK..gotta love Cisco...called to get a service agreement on WLC2106 so that I could download the latest firmware.  They would not sell me the service agreement nor would they let me download the latest firmware.  Only option that they had was to buy CT2504 and new service agreement for it.   all I need is the latest firmware for the WLC2106.  Anyone have a solution for this?  Very frustrating dealing with them.
gmerilliAuthor Commented:
Here is the console from the AP.  repeats.  10.1.3.2 and 10.1.3.112 can ping each other.  any ideas about Invalid event 38 & state 2 combination?  What does it mean?


.
*Mar  1 00:22:09.097: %DHCP-6-ADDRESS_ASSIGN: Interface GigabitEthernet0 assigned DHCP address 10.1.3.112, mask 255.255.255.0, hostname AP1cdf.0fb5.25b4

Translating "CISCO-CAPWAP-CONTROLLER"...domain server (255.255.255.255)
*Mar  1 00:22:13.096: %CAPWAP-5-DHCP_OPTION_43: Controller address 10.1.3.2 obtained through DHCP

*Mar  1 00:23:03.098: %CAPWAP-3-DHCP_RENEW: Could not discover WLC using DHCP IP. Renewing DHCP IP.
Not in Bound state.
*Mar  1 00:23:14.067: %CAPWAP-3-ERRORLOG: Invalid event 38 & state 2 combination.
gmerilliAuthor Commented:
was able to download ios from and internet source and successfully upgraded the controller.  The controller came with V4.2 and after upgrading to 7.0, all of the access points came up.  The access points were also on 7.0.  So it appears that the error above indicates the controller and access point versions do not match.

Because of this issue with Cisco we are looking at replacing the wifi solution with Aruba.  In our instance Cisco Policy has lost themselves business.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Ken BooneNetwork ConsultantCommented:
Please keep in mind you are dealing with a Cisco product that had its end of life announcement in 2011 - 4 years ago, and apparently your customer did not keep up the maintenance.   So that is pretty normal in this industry.
gmerilliAuthor Commented:
fixed problem
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Network Management

From novice to tech pro — start learning today.