identifying obfuscated IP addresses used by commercial proxy services

How can we track the provenance of an IP address?  Is there a magic RBL that lists the known IPs of commercial proxy services?

Having a problem with an "American" company who leases IPs from datacenters in U.S./Canada.  But, they have no control over the IP space.  They sub-rent the addresses to another company that uses it for anonymous proxy service.

What do you know about middlemen or shell companies that SWIPe addresses so they can lease them out in an obfuscated manner?

The users of the VPN/proxy service can get services from North America that are normally blocked by IP geo-fencing.

The users also hack U.S.-based networks who normally don't block their polite neighbors to the north.

Does ARIN or any other NRO actually require that you control your IP space?  Or is it generally accepted that the registrant might just be a shell company?
LVL 32
aleghartAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Dr. KlahnPrincipal Software EngineerCommented:
I have a somewhat similar problem on an Apache server.  Periodically a single request comes in for "/index.html" and the browser ident field is always identical.  Clearly it's a robot that moves around the net.  Chasing the IP back to the source, it it always goes back to a commercial hosting company.

I've been blocking hosting companies over a year now, and the problem recurs at new addresses every week.  I now have over 600 lines of CIDR IP blocks in the iptables block list and it gets bigger at about 10 blocks a week.

(This is after the address has been vetted by mod_spamhaus, mod_honeypot and mod_torcheck!)

There are several "internet defense" companies producing "do not want" lists, but I'm cheap and am content so far to add more iptables entries as the problem recurs.  The problem is thinning out as the number of unblocked CIDR blocks is dropping.  (If you want a copy of my host's iptables block list, I'll be glad to supply it.)

But the downside of this is that blocking, say, a /18 CIDR block at a commercial hosting company also blocks every legitimate company that hosts there.

To answer your questions:

What do you know about middlemen or shell companies that SWIPe addresses so they can lease them out in an obfuscated manner?

There are quite a few of them and imo, they're all shady.

Does ARIN or any other NRO actually require that you control your IP space?

No.  Your address block will get a bad reputation (e.g., spamhaus or Project Honeypot) if you don't, but there is no requirement.

Or is it generally accepted that the registrant might just be a shell company?

Yes.  Nobody who owns one of the shady hosting/proxy companies wants their name on the registration.  By way of similar example, look at the DNS registrars that now offer anonymous registration to anybody.
aleghartAuthor Commented:
Hey Dr. Klahn,

Thanks for the input.  I'm hoping to find some guidelines on control of IP addresses that are registered.  Any official docs on whether it is or is not even considered.  They usage ratios are often thrown at customers as a means of withholding IP address distribution...wondering why actual use is different than sub-leasing to a third party.

Regarding your connection annoyance:  A single hit for /index.html is not a hack.  At worst, it's a mapping project.  Similar things happen at firewall levels with ICMP packets.

I'm looking at thousands of hits for known exploits run against a single host.  The browser user-agent is a known reconnaissance tool, and the traffic is coming from another country through a private VPN designed to hide the identity of the hacker.

I tried connecting to the URL listed in your profile. It serves up permission errors.  Not sure if this is related to your random /index.html connections, or if it's a transient problem.

Sorry to get OT inside my own question...
gheistCommented:
bot herders already have their zombie network. You will not be first if they hack your server. Thats were IPs come from.
What is NRO?
Powerful Yet Easy-to-Use Network Monitoring

Identify excessive bandwidth utilization or unexpected application traffic with SolarWinds Bandwidth Analyzer Pack.

aleghartAuthor Commented:
NRO = Number Resource Organization

ARIN = American Registry for Internet Numbers

Although, with IPv6...not just dealing in those numbers any more.
gheistCommented:
Never heard of Number Resource Organisation.

ARIN is one of RIRs
They dont ask to identify each IP with a person like KGB
aleghartAuthor Commented:
ARIN is one of RIRs
They dont ask to identify each IP with a person like KGB

Not looking to identify a person.  ARIN limits the IP space and will deny applications in order to preserve the limited numbers left.

It seems that people/organizations are taking the IP space and renting it out for use in other countries, without maintaining any sort of control.

I'm looking for information on whether ARIN or any other NRO specifically addresses control of the IP space and their use.
gheistCommented:
Any webserver gets scans from botnets. That is part of internet noise. Since you identify them by user agent and hit on index.html you can block those IPs for good. They are victims infected and will continue to scan the internet on behalf of their masters.
aleghartAuthor Commented:
gheist, I'm not discussing random sweeps from botnets.

The user-agent does not indicate an infected computer...it's a hacker using pen-testing tools.

I'm looking for information about IP usage guidelines, not how to block an user-agent.
gheistCommented:
whois should point you to abuse contact of particular network. though one http request will not rise them from slumber.
aleghartAuthor Commented:
again...not looking at one incident.  This is just an example.
Not looking for ISP's NOC.
Not looking to block zombies.

Looking for information on IP address usage by ARIN.

This thread is so OT, it's ridiculous.
gheistCommented:
Ask ARIN how allocated IPs are being used, dont get offensive.

Internet is a cooperative venture and you need to contact site that attacks you. It has not changed in last 20 years I sit in front of screen with internet.
aleghartAuthor Commented:
gheist,  I'm getting frustrated because the only advise is drive-by advice regarding a single bot scan.  It's not really relevant, and not really good advice.  Just ignore the noise...

I'm looking for usage guidelines or requirements from ARIN and their counterparts who control IP address space.

There's nothing co-operative about sites that obfuscate their true operators.    That's the root problem.

I contacted the IP space owner, and they flat-out admitted that they rent it out for use by a VPN service on another continent so those users can fake that they are in the U.S. or Canada.

I sat in front of a green screen before there was an 'Internet'...with a 300 baud modem.  It has changed a lot in just the last 10 years...you don't need to go back 20+ years.

Operators nowdays are concerned about protecting their revenue, even when their users are behaving badly.  Some companies specialize in assisting their users to behave badly.

But again...OT.
giltjrCommented:
Let me see if I understand what you are looking for.  It sounds like you are looking to see if any of the RIR's have a policy that  that would prevent me from setting up a proxy server and allowing you to use it for the purpose of allowing you to hide who you are and where you are located.  That I am aware of there is no such policy by any of the RIRs.


ARIN has this page which has links to some of their various policies:

https://www.arin.net/policy/nrpm.html

Which has links to their various policies.
btanExec ConsultantCommented:
Doubt you can find one central authority having such list of VPN/Proxy list shared in public but such lawful interception and monitoring does exist for those intel agencies which is beyond the discussion here or neither worth exploring further. instead, if really interested to know the IP behind because it is doing unlawful (Fraud related) or anomalous  or "whatever" suspicious activities, WhatIsMyIP.com service online is a good place to find and reveal the existence and more info on the IP.
https://www.whatismyip.com/ip-address-lookup/
This include identify use of Tor exit node (very common for achieving anonymity thru Tor which malware even leverage on for callback to their C&C backend master controller) and of existence under DNSBL (Domain Name Blacklist).

There are tools to identify Tor Exit using the torproject own tool (https://exonerator.torproject.org/) and Exitmap tool (https://github.com/NullHypothesis/exitmap) that check for false negatives on the Tor Project's check service and to find malicious exit relay. This has gain traction and interest esp in underground dark web (protected using Tor too..mostly)

Regardless, many exploit free proxy instead - see below.
http://proxylist.hidemyass.com/ or http://www.publicproxyservers.com/proxy/list1.html or http://proxy-list.org/english/index.php
For commercial one, they will be the end to end VPN tunnel and unless you are LE (whom has reach into ISP), there is no mean of finding out more of the tunnel especially.
Fred MarshallPrincipalCommented:
I'd like to help but I'm not sure that there are "good" answers to the multiple questions.  But I'll try:
Does ARIN or any other NRO actually require that you control your IP space?  Or is it generally accepted that the registrant might just be a shell company?
I think this set of questions and issues needs to be parsed so that the pieces can be addressed in a more focused manner.  I don't mean to be picky but having clear ideas about what the issues are is important to avoid a rambling discussion.  Making assumptions about what someone else means isn't productive.  So....

- What is meant by "control IP space"?
- What sort of requirements might anyone apply to assure some level of control of IP space?
- What is a "shell company" in this context?

You mentioned "swiping IP addresses".  How does this relate to how routing is done on the internet in general?  Are you assuming ISP behavior that is contrary to good practice?  In general, stealing IP addresses seems not productive in the real world.  But then, I realize that there are multiple "real worlds"  :-)

Indeed there are proxy services.  Their purpose is to hide the actual users from the rest of the world.  What's your question re: this?

Here's a scenario that one can imagine:
- a nefarious user arranges for proxy service.
- then the nefarious user uses the proxy service interface for unwanted things.
- then you want to find out who the nefarious user is in order to ... block them?  Or, isn't it just as good to block the proxy address?  If not, why not?

One should probably assume that IP addresses are going to be dynamic no matter what.  The methods used might be automatic and they might be manually-implemented.  Every time a new address appears that's an example of a dynamic - whether the eventual use of that address is "good" or "bad" or "wanted" or "unwanted".  I would suggest that this is the orgin of real-time blacklists (RBLs).  That's why they're useful.

I'm looking for usage guidelines or requirements from ARIN and their counterparts who control IP address space.
 I would be careful you don't over-extend your use of "control" in this context.  It seems clear that ARIN and the others control the *assignment* of addresses.  But I don't see anywhere where they would purport to control the *use* of the addresses.  I could be wrong in this.  It's easy enough to read the ARIN web pages.

 
There's nothing co-operative about sites that obfuscate their true operators.    That's the root problem.
Well, they certainly cooperate with their customers.  It appears that you're looking for information that's not available on purpose - you would like them to cooperate with YOU in direct opposition to the service they are providing their customers.  Is that right? THAT would be a root problem unto itself it seems.

I contacted the IP space owner, and they flat-out admitted that they rent it out for use by a VPN service on another continent so those users can fake that they are in the U.S. or Canada.
I believe it and am not surprised.  Are you surprised?  This seems like nothing more than a statement of fact.  I can't really comment further without going all philosophical...
Steve BinkCommented:
Organizations like ARIN don't really care how you use your IP address.  Per ARIN's Number Resource Policy, their concerns are more directed at managing the IP pool, not worrying about if every denizen of the internet is complying with some sense of global morality.  Since RIRs are inherently multi-national organizations, whose morality would they be enforcing?

Running a proxy service is an entirely legitimate enterprise.  In America, Canada, and (AFAIK) virtually all other ARIN member countries, there is no bar or prerequisite to running a proxy service.  Even if you complained to ARIN about an IP block running a service, they would have nothing for you.  It does not violate their policies, since they have no policies on end-user utility or content.

Target your complaints to the LIR/ISP owning the offending IP.  If they are unwilling to listen or assist, well... you'll just have to live with that.  Not everyone on the intarwebs is bright and shiny, and the anonymity that gives the internet its robust flexibility is the same attribute that lets the bad guys annoy good and righteous people like yourself.  That's why god made routing black holes.
aleghartAuthor Commented:
@Fred Marshall - thanks for your questions.

- What is meant by "control IP space"?
--actively use it
--actively used by the org/person requesting it
--ARIN "justification" form requires 50-75% claimed usage, including listing existing host/IP address before issuance...why bother if they don't actually care why you're using the space?

- What sort of requirements might anyone apply to assure some level of control of IP space?
--able to monitor and control usage

- What is a "shell company" in this context?
--a company with 'domestic' business address per the RIR's footprint
--registering IPv4 space for the domestic company
--renting the IP addresses for use in another country to bypass IP geo-fencing
--renting the IP addresses for use in another country to hide the source network

A shell company is not using the IP space.  They are renting it out to orgs/people who do not qualify for the IP addresses because of the jursidiction of the RIR.
aleghartAuthor Commented:
@Steve Bink - clarification below.

-Since RIRs are inherently multi-national organizations, whose morality would they be enforcing?
--Unknown.  That's why my question...what/where does ARIN have any usage guidelines?  While it might seem like a philosophical question, I don't think it's arbitrary.

If the RIR can say yes/no to an applicant based on their internal guidelines, than what happens when the applicants circumvent the process by using a shell?  

-Not everyone on the intarwebs is bright and shiny, and the anonymity that gives the internet its robust flexibility is the same attribute that lets the bad guys annoy good and righteous people like yourself.  That's why god made routing black holes.

I'm not flying the righteous flag here, nor needing a lesson in anonymity.  Was looking for ARIN guidelines on usage.  If they don't care...then a complaint will have no immediate action.  If they do care, I'm looking for citation or a link.  Their site, along with the other RIR sites are not conducive for

Second request was for an RBL of known proxy services that are used for hiding user location.

Wikipedia uses some sort of anonymity detection to find the proxy users.  To reduce abuse from hidden IP addresses, users must login with a valid account before making any edits.    I don't think they are "righteous", as you say.  A genuine concern for abuse.

I use proxy services for research.  I understand their value.  But, I would also like to know who else is banging on my perimeter in a manner that can be identified and blocked.

Routing to a black hole requires that you have a source address list.  Hand-curating a list would seem inefficient.  The biggest problem is finding the subnets that are currently performing these services.  Like everything else on the internet...it will change from year to year or hour to hour.  An RBL would help.

It's not just from a blocking perspective.  Doing a one-off DNS query to an RBL can save many minutes or hours of investigative work.
gheistCommented:
There is no limit on how you use your IP address space.
You need to prove 50% usage to acquire IPv4 space now. 10 years ago you just had to write prefix number.
Now you jump other horse. First question was about proxy services of commercial nature, now you want to fight EFF and TOR too.
giltjrCommented:
I would think anything that is even close to what you are looking for from ARIN would be in the Registration Services Agreement:

    https://www.arin.net/resources/agreements/rsa.pdf

Or the Legacy Registration Services Agreement :

     https://www.arin.net/resources/legacy/index.html
giltjrCommented:
"Wikipedia uses some sort of anonymity detection to find the proxy users.  To reduce abuse from hidden IP addresses, users must login with a valid account before making any edits."

I would think that to perform update that you would need to login with a valid account.  I should not matter if you behind a proxy or not.  Your statement implies that at some level I would not need to login because Wikipedia knows who I am based on my IP address.  I doubt that very much.  I would make the assumption that they make you login no matter what.
Steve BinkCommented:
>>> --Unknown.  That's why my question...what/where does ARIN have any
>>> usage guidelines?  While it might seem like a philosophical question, I
>>> don't think it's arbitrary.

That's one of the points I was getting at before - ARIN does not have a dog in this fight.  They don't care WHY you are using an IP, only if you are actually using it.  More correctly, they are concerned that you are using at least x% of your assigned block.  Their concern is limited to managing the IP space for their region.  They are not the least bit interested in knowing about, much less managing, the services available within that space.

>>> I'm not flying the righteous flag here, nor needing a lesson in anonymity.

I was trying to be funny, not snippy.  Moving on...

>>> Wikipedia uses some sort of anonymity detection to find the proxy users.  
>>> To reduce abuse from hidden IP addresses, users must login with a valid
>>> account before making any edits.

As pointed out by giltjr above, I would think it likely that Wikipedia is requiring an account regardless of the source of your IP.  In any case, you can see some of their relevant info at:

https://en.wikipedia.org/wiki/Wikipedia:Blocking_policy
https://en.wikipedia.org/wiki/Template:Blocked_proxy
https://en.wikipedia.org/wiki/Wikipedia:WikiProject_on_open_proxies

That last one is of real interest here - it shows that Wikipedia maintains its own black-/block- list for open proxies via community reports.  Some of the links from that page (look under "See Also" for subpages and related pages) imply there is at least some level of automation for the checking/detection, but it looks like much of the work is done through community curation.  Could be they make their list accessible to the public, but it looks like they are in the process of redesigning it.

When I worked at a hosting company, we used to check in-/out-bound mail against a variety of RBL/SBL providers.  Anything found suspect was quarantined, or otherwise dealt with a fair bit of prejudice.  A quick search for open proxy blacklist gave me a lot of open relay lists, but nothing promising for open proxies.  Setting up your own internal list with DNS would not be that hard, though managing it could still be a bit of a time sink.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
aleghartAuthor Commented:
giltjr - Wikipedia doesn't track you by IP address.  They do block you by IP address when you are on their internal list.  I'm looking for a similar resource, preferably a publicly-queryable (is that a word?) RBL that can be hit via DNS.

Developing and managing a list internally is not efficient, as any one site (except for perhaps Wikipedia or google.com) would receive very little traffic compared to all proxies in use.

Wikipedia allows anonymous edits on most pages, unless the page is in protection mode or unless your IP is blocked.  In this specific case, it is a commercial VPN-based proxy, _not_ an open proxy as described in the block message.

anonymous edits allowed
proxy blocked
gheistCommented:
Hunting down Asian-serving services in ARIN IP service area is not Robin Hood, it is closer to Don Quijote...
aleghartAuthor Commented:
My reasons for this Q are not to block the world.  I'm looking for a scoring mechanism no different that the RBLs used for spam filtering or SMTP relay blocking.

It's nothing against any one nation or region.  But, there is a flag when you obfuscate your IP address by region-hopping.

It's a tiny flag, considering the volume of non-nefarious traffic that traverses proxies.  Many are people trying to get uncensored content that's not available in their own country.

Same goes for tor traffic.  Can I flag for analysis?  Yes.  Does it mean the user is evil?  No.  Just a data point.

If there is an RBL I can query, it would help in analysis.  I don't need to block everyone based on a binary decision for 'proxied = yes/no'.

It took a couple of hours of work to identify and make a decision on one IP address.  Would like to reduce that to a few seconds, if possible.  Had the intermediary company been uncooperative, I would still be doing research, or would have given up by now.

Human time is not a plentiful resource.  If there's a dynamic list out there, I'm sure it would save a lot of other people some time too.
aleghartAuthor Commented:
Hunting down Asian-serving services in ARIN IP service area is not Robin Hood, it is closer to Don Quijote...

They said the same about spam scoring software and 'restraint-of-trade' complaints.  Yet show me one mail service that does not query an RBL.

It's not an impossible dream.  There are some organizations who already have lists of open and commercial proxy server networks.
gheistCommented:
Make it happen - invent protocol more widely deployed than DNS, and serve your private blacklist over it...
aleghartAuthor Commented:
They say invention is born from laziness.

I'm one step before invention...I just want to ride on the backs of others who are smarter or more industrious than me.  (And, I'm honest about it.)

Maybe next week I'll find all that copious spare time we all have...
btanExec ConsultantCommented:
blocking by IP is neither the best means but depending on proxy as mentioned is not a good approach either. the granularity to geolocate and find out the actual true ip behind those proxy will be worthy for consideration. eventually it is just to say do we want to allow such party to the resource you own or guard....and with ipv6, those block assignment if exist doesnt matters if the whole idea is for attributing the source coming in ...

also several people I get to know or hear of who've gotten IPs or requested clarification from ARIN, it's rather clear that they don't have any hard and fast rules for what kinds of usage are acceptable or not. Basically if you follow their rules regarding utilization percentages, you can probably get more.

Game servers, VPS servers, VPN servers, proxies, SSL, even occasionally email, all sorts of things have a valid use for multiple IP addresses. Some hosts will give you varying numbers of IPs for varying intended purposes, but ARIN does not have any hard and fast rules as to what is valid technical justification and what is not. We have our own IP policies that are supposedly more strict than ARINs policies. But in general, ARIN follows their own guidelines as closely as they can, and their own guidelines don't specify what precisely is considered reasonable technical justification and what does not, it simply states that the IPs must be in use.

I doubt then there is a list of IP block truly worth find out unless we go targeted base on use case or have some service like Norse or LookingGlass advising the good and bad ip - if we have the reach to them...

Pardon for the adds..which may not be helping as much
aleghartAuthor Commented:
Thanks for the discussion.  It's mostly a search for policy...not an easy thing to find or parse.  Ended up a philosophical question, so the grad is "A" for the relevant discussion.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Networking

From novice to tech pro — start learning today.