Weighing Options for Penetration Testing

McKnife:  Yes, I understand that risk.  However, there are practical considerations to be considered.  Learning is one of them.  Cost is another.  Acceptable results is another.  And, all this is somewhat tangential to my original question.

No matter how one might approach this, in-house or contracted, the fundamental question remains one worth knowing the answer.

vulnerability scans use rules to test against, and have to be continually updated.

Pen Tester's have no rules they just attack

There are no web servers involved....

pen testers don't need a web server

My comment about the webservers was directed to the folks who were kind enough to point out that they might be targets.... in this case there's no such target and, thus, not those types of vulnerabilities is all.  I can well imagine that a vulnerability scan might not be so good facing one of those targets.
The situation that I'm concerned about has a firewall and a couple of VPN routers with public addresses.  Vulnerability scans yield nothing at this point.
So, I'm scratching my head trying to decide how to best attack them - or to ask someone to attack them.
I don't see why one would spend money to attack something that reveals no vulnerabilities.
I can understand that a pen tester could attack in the blind.  But should one spend the money to do that under the circumstances described?
It's fine to say: "of course!" unless you also have to answer "why?".  "Just because it's a good idea" isn't likely to cut it.  Not that it isn't a good idea of course.
Examples are always good.

That's fair. I would say yes because you'd rather understand your security posture and what you can do to better defend yourself. Tactics, techniques, and procedures evolve all the time. It's better to discover and learn threats out there than to remain in the dark. You're obviously not going to suddenly learn all of the zero days and so on in existence, but you can get at least get a better idea.

masnrock:  OK.  I guess this then raises one of my concerns.    You say:

From there, you're going to have to work on your penetration tests.

What I don't yet understand is how one works on penetration tests *without* vulnerability scans to guide.  Or is it just "cut and try" from start to finish?  Surely a pen tester can run a vulnerability test.  Does this suggest that they don't?  If they don't, why?  If they don't because it would break some "rule", where do they start?  If they can't find any vulnerabilities then how do they decide what to do - AND preserve time and money?

What sort of "potential attack vectors"?

Actually, it's not that hard.  The easiest vector is always through a human.  Call up and ask questions and people tend to give you answers.  Phishing.  Much easier than any of the technical scans and attacks.

I'm beginning to get the picture I think.
Since "anyone" can do a vulnerability scan then those activities are discounted.  And the timeliness of the information that the scan is based on would certainly be an issue.
But, when a *PEN TESTER* does a vulnerability scan then that's part of the blessed pen testing process and has different meaning without added basis.

If I were to keep things neat then I'd keep them separate.
My notion is this:
Somebody is going to do a vulnerability scan.  It doesn't really matter who does it.
Somebody is going to do a pen test.  Given that the vulnerability scan information is available or is re-run then we can consider this phase "done" and should anticipate that it *will be* done as part of the pen test.

How does the pen tester bid a job when they are going to be "poking and prodding"?  How does the customer know when the "poking and prodding" has been adequate?  Please forgive my lack of experience with this community.
How much should one pay for an external pen test on a single IP address?

masnrock:
Thanks for bearing with me on this!  As you can see, I'm struggling a bit to figure out how people think and operate in this environment.

So, I guess there are "documented" vulnerabilities and there are "known but undocumented" vulnerabilities.  What the heck is that??  Sounds a bit odd.

From my limited experience, the vulnerability scanners do a reasonable job of fingerprinting hardware (or OS).  So, again, what's the difference?

Also, I have read that the pen testers should not have knowledge of the network configuration.  Why not?  Would this not give them an advantage in doing their attacks?  What's the disadvantage?

I'm not trying to grind any axe, just understand.

David Johnson:  All good points.  It makes me realize that the objective I'm actually pursuing is really more limited in scope in comparison to how people define penetration testing and those things you mention.  What I'm looking for, specifically, is external network penetration testing of public IP addresses at the edge of a private network.  Things like physical security and social engineering are outside the scope of this *one* objective.  There are other objectives of course.... So, it's just a piece of the pie.

So, to be a pen tester, one has to have private knowledge to be any good.
How are we to know?

Think of a security guard versus a thief. One will go based on the known and follow rules. The other aims has no problem going against the rules.

serialband: I understand your concern re: the human element and you and others have  provided a clear perspective on it.   At the moment, and the crux of this question, is separate from that.  I can see how that may seem myopic.  I am NOT assuming it's irrelevant - but, simply, that it's not what I'm asking about.  Indeed, we expect to be dealing with the human element but I'm not the decision-maker relative to HOW that will be done or by who.  However, *I* have to do the rest of it and help make the decisions how to split it up .. if "splitting it up" is even operative in the end. Perhaps the split is "conventional" and we do the vulnerability testing and someone else does conventional penetration testing.

All:  There is one question that remains unanswered:

WHY should the pen tester start with not knowing the internals (whatever that means)?  I don't think we are testing THEM but that THEY are supposed to be testing our systems and environment.  (Not that it would be provided but) I'm still curious how their knowing "internal" things would affect the prioritization of the results?  After all, that's the bottom line isn't it?  Or, does this refer to things like who is who in the zoo? what companies provide support?  etc. leading to the human engineering part.  So, what ARE the "internals" by definition?

David Johnson:
So, it's as I suggested, we are testing THEM to a degree.  
"Can a well-qualified pen tester get into our system without any foreknowledge?" is a test of the pen tester.
vs.
"Is our system robust to attack (even if there is insider information provided)?" is more a test of the system.

But, you have helped define what "inside" information might contain.
Surely providing login credentials would be too much!
On the other hand, I've been considering insider knowledge more along the line of network topology.

Your original question asked about doing a Penetration Test in addition to a Vulnerability Scan.  We've defined that for you.

No system is robust to attack if you give insider credentials.  Once you have access, you have access, even if it's limited.  Unfortunately, in a real Pen Test, many people do provide login credentials.  That's why a network scan is wholly different from a pen test.

Network topology will be gained once they have credential access.  That's not something the average user will be capable of giving out.  Only the sysadimn/network admin will have that information.

serialband:  I may have misled you with my comment about credentials.  My point was about "inside" information and what that *is* and what it *is not*.  I tried to say that credentials *are surely not*.

I have still not received an answer regarding "why?" having inside information is discouraged.  My view is still that this alludes to testing the tester.  

So, it's as I suggested, we are testing THEM to a degree.  
 "Can a well-qualified pen tester get into our system without any foreknowledge?" is a test of the pen tester.
 vs.
 "Is our system robust to attack (even if there is insider information provided)?" is more a test of the system.

Adam Brown:  Actually I am such a neophyte that what may seem like an answer to an expert practitioner isn't necessarily an answer for me.  So I respectfully disagree that my questions have already been answered.  Part of the reason for that is that answers have generated more questions.  

Anyway: great answer!

The purpose of a penetration test is to act as a legally sanctioned simulation of an unauthorized hacker attempting to breach the security of the environment.

That's what was missing for me.  Now that you've said it, I guess it's a bit obvious.  We are trying to simulate an unauthorized hacker.  And, the outcome / report, in view of it's purpose of "what information can be obtained?" could be contaminated by starting with *any* inside information.  

We have to assume that a hired hacker is as good as an unknown hacker, eh?

This also gets to "what is information" in this definition.

1) For networking people, it could be things like make and model and OS version of a device - which could lead to access to customer information either today (and in the report) or tomorrow (so the technical information is reported but not with customer data resulting).  And, if we assume that there's a moving target, not all of this might be found in a vulnerability test.  But then we get wound around "what is public information" axle IF we expect pen testing to somehow be "better".

2) For operations people, it could be focused on customer data AND they may overlook #1 as important.

Presumably, experienced penetration testers will know what to report.  But it's best when the customer knows what to ask for and what to expect.

Pen tests should also be done by a different team of people than those granted access to scan known networks.

serialband:

Pen tests should also be done by a different team of people than those granted access to scan known networks.

Yes, I've been given this input a few times; either here or in my own reading.  This led me to ask "WHY?".  This led me to ask: "Doesn't this suggest a test of the tester?"  Adam Brown provided the best, most succinct, answer:

The level of information obtained during a penetration test is what the testing organization will report on after the process is complete. Giving *any* information about the environment beyond pertinent domain names defeats the purpose of a true penetration test, since an unauthorized attacker will (hopefully) not be able to obtain that information without breaching security in some way.

and I added the notion that the resulting report could/would be thus contaminated.  

Adam Brown: sez:

It's important to note that without physical access to the network (or wireless access), there are a lot of limitations to what can be done to attack an environment. Social engineering attacks like phishing and the introduction of malware are used to gain remote access, but without those in place, only the level of access explicitly allowed for remote use is accessible. Penetration tests will, as I mentioned, focus on flaws in the applications and services that are available publicly. This includes Websites, VPNs, mail servers, and any other system that requires public Internet connectivity to function.

I think I'm getting it now.  Thanks all for the introduction to the language, terms and perspectives!

Thanks all!!  Very helpful!