how do you find a seemingly rogue dhcp server with no traffic related to the IP address displayed in DHCP Find utility?

I had this question after viewing DHCP Find is picking up a rogue server.

I am experiencing the exact same issues as the gentleman in the posted link. I'm seeing DHCP offers from a 112.112.108.116 IP address (which when looked up originates from China) Here's the caveat...The offers are IP addresses in the range configured in our regular DHCP server. There is no traffic captured in Wireshark from the IP address and there are no ARP entries anywhere with that IP address. I also have that IP blocked at the firewall. The accepted answer to that post was to see if there was a BLU phone connected to the network somewhere. There was no confirmation that this was actually the case. Anyone out there have any thoughts or ideas that they can send my way? I'm at a loss here...
Rob WombleSenior Network AdministratorAsked:
Who is Participating?
 
atlas_shudderedConnect With a Mentor Sr. Network EngineerCommented:
Set up DHCP snooping on the switches inside the affected subnet/vlan to kill dhcp advertisements from any port other than the one define as being connected to the dhcp server.  This will help you prevent rogue offers onto the network.
switch(conf)#ip dhcp snoop
switch(conf)#interface fast/gig x/x/x
switch(conf-int)#ip dhcp snooping trust

Open in new window


Line 1 above enables snooping on the switch globally.
Line 2 jump to interface that trusted DHCP server resides on, or any layer 2 uplinks (trunks/port-channels) between switches looking from far end back toward the trusted DHCP server
Line 3 configure interface as a trusted interface locally, which permits dhcp offers received on that interface to be processed

This will isolate rogue dhcp offers from the LAN.

If you want to track down the DHCP host do the following:

1.  From an host that has received an address from the invalid host, run ipconfig /all and capture the IP of the issuing server
2.  On the same host, run arp -a and cross reference the IP from step 1 to it's associated MAC.  (It may be necessary to ping the IP to capture arp information)
3.  Beginning from your core switch, run the following:
switch# show mac add | inc *mac address from step 2 above*

Open in new window


capture the interface listed beside the MAC address in this table

switch# show cdp ne

Open in new window


match the interface captured above with the next hop in the CDP neighbor table displayed.  If there is no neighbor, then interface shown should be direct connected to your rogue server

Hope that helps
0
 
Dr. KlahnPrincipal Software EngineerCommented:
There is no traffic captured in Wireshark from the IP address

No, there wouldn't be; DHCP uses UDP.

You might try this:

Configure Wireshark to catch UDP packets of DHCP format.  Temporarily split the network in two at the router.  Connect Wireshark to one half and bring up a system or systems to see if any DHCP responses are caught from the rogue server.  If not, connect to the other half and check again.  Continue splitting at each hub or switch until you find the segment that the offending server is on.  Then go find it.  But note that this will work only if the device is wired.  If it's wireless, about all you can do is capture its MAC and make a guess at what it might be.
0
 
Dirk KotteConnect With a Mentor SECommented:
use wireshark and try to get an IP from rogue dhcp server.
set filter to "bootp" (without quotes)
take a look to the MAC address from DHCP offer.
(This should be the device running the server)
now check the mac-address tables from your switches to see where (which port) this device is connected.
0
Improve Your Query Performance Tuning

In this FREE six-day email course, you'll learn from Janis Griffin, Database Performance Evangelist. She'll teach 12 steps that you can use to optimize your queries as much as possible and see measurable results in your work. Get started today!

 
Rob WombleSenior Network AdministratorAuthor Commented:
Thank you guys for your responses. I'm looking into your suggestions now. I've already done a lot of what was already suggested. This issue seems to be affecting Mac's more than anything. Windows machines are getting addresses and Mac's are just not getting addresses and ending up with self assigned addresses.
0
 
Rob WombleSenior Network AdministratorAuthor Commented:
Update: This issue seems to only affect Apple products. iPhones and MacBooks are getting addresses but no apple products. Also I see in the DHCP logs that there are several iphones that are trying to renew their addresses. There will be 4-5 renew requests in a row in the log per device and then cycles through with renew requests from each device over and over. Not sure what to think about this. Windows machines are not affected and receiving IP addresses as they should be.
0
 
atlas_shudderedSr. Network EngineerCommented:
4 questions:

1.  Is the network population the same as it was prior to posting this question?  In other words, have the Mac products received addresses previously without incident?
2.  You mention iPhones, I have to assume you are seeing those on wireless network.  Are the other affected devices attempting to connect via wireless?  Are the windows machines wireless or fixed port?
3.  Are you using DHCP reservations?
4.  What is the utilization of addresses in your DHCP database?
0
 
Rob WombleSenior Network AdministratorAuthor Commented:
Thanks Atlas..

1.  Is the network population the same as it was prior to posting this question?  In other words, have the Mac products received addresses previously without incident?
Yes the network population is the same and the Mac products have previously received addresses. They are not receiving addresses either wired or wirelessly
2.  You mention iPhones, I have to assume you are seeing those on wireless network.  Are the other affected devices attempting to connect via wireless? Not that I know of..windows laptops are connecting just fine both wired and wirelessly.  Are the windows machines wireless or fixed port? Most Windows machines are fixed port. Some are laptops that have no trouble wired or wirelessly.
3.  Are you using DHCP reservations? No
4.  What is the utilization of addresses in your DHCP database? Currently at 74%
0
 
atlas_shudderedSr. Network EngineerCommented:
Now worries Rob.  I'll have to default back to my original recommendation of implementing snooping to attempt to find nullify your rogue and ping/arp to trace him down.  The update your provided gave me some hope that it may be some type of media problem for a minute but you have put that through the proverbial wood chipper.  There may still be some weird protocol issue at work.  You'd be back to a packet capture on that front.  Sorry
 
One other thing that you can do is run a debug ip dhcp after you have implemented snooping.  If he is out there, debug will show his drops in the logging.

Just to be clear on the snooping.  By setting up the trusted port you are telling the switch(es) what ports they are to expect to see dhcp ack/offers from.  Any port that you do not specifically place in trust mode will kill any traffic they see.  This does not impact dhcp request packets.
0
 
Rob WombleSenior Network AdministratorAuthor Commented:
I've enabled dhcp snooping trusted the ports to the dhcp server only. Still being plagued by DHCP issues. To me...it looks like there's just a flood of DHCP offers coming from an external DHCP server...although I have that address blocked at the firewall and there is no traffic with that address as a source or destination address. here is what is showing up in DHCP find.
dhcpfind1312018.png
0
 
atlas_shudderedSr. Network EngineerCommented:
Rob - I am posting the config notes from cisco on dhcp snooping configuration.  Looking back through my notes, I can see at least on cli comment that I did not include.  (ip dhcp snooping vlan "x" - where x is the vlan you want snooping to be activated).  

The link also includes directions for setting up source-guard but I would advice against this one unless absolutely necessary for resolution.  Implementing can have consequences that take your problem from bad to worse if not properly planned out and understood before input.

Have a look at the document at the link.  If you configure and still seeing the problem, I would suggest spanning your dhcp server port and sniffing the traffic to determine if it is pushing the rogue packets itself before considering source-guard.

https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst4500/12-2/25ew/configuration/guide/conf/dhcp.html
0
 
Rob WombleSenior Network AdministratorAuthor Commented:
Thank you Atlas,
 I really do appreciate your wilingness to provide suggestions. I've temporarily shut down the Windows DHCP server and enabled DHCP on another device The traffic has immediately calmed down from 7% of the traffic down to .4% so I'm thinking there's something going on with the Windows server so I'll dig into that next. Thanks again for all your help
0
 
atlas_shudderedSr. Network EngineerCommented:
No worries Rob.  Good luck with the rest.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.