Hyper-V VMs physically on separate subnets - howto

I'm setting up Windows Server 2016 Standard.
My objective is to have 2 VMs that are connected into their own subnet for a testbed.
I've read about the internal switches, etc. but don't see how to make a physical connection to each of the subnets AND be independent.
(Perhaps the testbed objectives are a bit different than a typical production network).
I am imagining adding a couple of NICs so there will be:
A NIC for the Hyper-V manager.
A NIC for each of the subnets.

A nudge in the right direction would help a lot!
LVL 28
Fred MarshallPrincipalAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Coolie SheppardSystems EngineerCommented:
The ports that you're going to use for VM access, physically plug those ports into your switch and trunk those ports with the respective VLANs that are in different subnets.

Next, create the VMs and in the network adapter settings, enable the VLAN option and enter in the tag that's on the switch (and router or firewall, whatever you're using for routing) and that's it.
Cliff GaliherCommented:
As I've been following your other questions, I kinda know what you want to do.

In the real world, your machines will be in different sites and those sites are (presumably) linked by routers. Either VPN or MPLS, etc.

To emulate that. Youll want each VM on its own switch. And also on each switch, you'd want a router.  I'd use a free Linux distro for this, like untangle.

Those "router" VMs will each have two virtual NICs (like a real router would have two ports... One LAN and one WAN.)

Put the Lan vnic on the virtual switch with your virtual DC for that "site."

Create a third virtual switch to act as the "internet" network. Plug the WAN vnic from both routers into that switch.  Choose a subnet for the WAN that is unique from the other subnets so the router can... Route.

Then declare static routes on the routers or set up a fake VPN and you've effectively created a two-site test platform..

Personally I think it's overkill, as testing across subnets probably isn't necessary for your goal of moving from a workgroup to a domain. But it'll definitely allow you to do such a test if you feel so inclined.
Fred MarshallPrincipalAuthor Commented:
Cliff:  Thanks!  
Well, I understand the words well enough but because I have so little experience with the VMs, I'd rather prefer to keep things as similar to the real world as possible.  I have plenty of routers, can set up a physical MPLS connection between physical subnets, etc.  And, I do plan to have computers on each of the subnets that will join the domain in their respective sites.

Let me respond to your suggestions to improve communication:

In the real world, your machines will be in different sites and those sites are (presumably) linked by routers. Either VPN or MPLS, etc.
Correct.

To emulate that. Youll want each VM on its own switch. And also on each switch, you'd want a router.  I'd use a free Linux distro for this, like untangle.
I believe you mean virtual switches here. I'm unclear about the router part.  I don't want to mess with Linux as, for me, it's always a learning process and a time commitment.  (I did set up Kali for Pentesting and that was sorta OK but still a bit tedious).  Anyway, I don't understand the Linux reference here.

Those "router" VMs will each have two virtual NICs (like a real router would have two ports... One LAN and one WAN.)
I don't see how I can attach other computers to a virtual NIC.

Put the Lan vnic on the virtual switch with your virtual DC for that "site."
Understood but probably not meeting my objectives for the testbed.

Create a third virtual switch to act as the "internet" network. Plug the WAN vnic from both routers into that switch.  Choose a subnet for the WAN that is unique from the other subnets so the router can... Route.
Also understood.  Use the same internet for both DCs on a vnic just like a physical "internet switch".  But how does it get an internet connection into that switch?

Then declare static routes on the routers or set up a fake VPN and you've effectively created a two-site test platform..

Personally I think it's overkill, as testing across subnets probably isn't necessary for your goal of moving from a workgroup to a domain. But it'll definitely allow you to do such a test if you feel so inclined.
The simulation is to move from two site workgroups to a domain with a DC in each site, keep the subnets that exist and show that communication and file sharing still work and then learn how to do other things in this architecture.

Being an engineer, I've always been more comfortable with physical functional blocks and often approach system architecture by partitioning things.  For example, in setting up a site-to-site VPN (and having plenty of public IP addresses), I've set up VPN routers separate from the gateway routers - in order to avoid making the gateway routers configurations more complicated (because they were complicated enough and key elements in the operation).  Just a choice.

Perhaps a diagram would help:
DC-Architecture-and-Simulation.pdf
CompTIA Network+

Prepare for the CompTIA Network+ exam by learning how to troubleshoot, configure, and manage both wired and wireless networks.

Cliff GaliherCommented:
I suppose first, I'll admit I find your diagram a bit confusing.   In the "Planned"  the way you stacked the boxes implies a flow of data that doesn't seem real.   I find using a tool like Visio to diagram the network works better.  You really should include "switches" in the diagram as they help clear up confusion, not just for others, but often light up a lightbulb as to how you'd simulate it.

If your planned sites would each have a physical switch to connect domain-connected workstations to servers then you'd want virtual switches in your simulation for example, and you'd more easily visualize that.

Secondly, in your "simulation" you kept MPLS in the diagram, but then have a DC in subnet A as a VM across the MPLS link....which seems off from your goal.  Both figuratively and literally.  Unless you plan on spinning up multiple Hyper-V servers in real sites..which is certainly an option...then you wouldn't have MPLS "in" Hyper-V, so why that's in the simulation is a bit confusing.  Again, Visio or similar tools work better fo this than mere free-hand drawing tools.   You can't follow a plan if you don't really have one.

Finally, as to how linux got in the conversation...

I was under the impression you wanted to simulate the entire environment on a single Hyper-V host.  In order to do that, you still need "routers" on each end of each (virtual) site you'd create.   *MOST* small business routers or UTM devices run a variant of linux.  From SonicWALL to WatchGuard to Fortigate, to those old Linksys black-and-blue boxes, they are running linux with a custom GUI or tools layered on top.  That's why free projects like Tomato existed that could easily replace the built-in OS on consumer routers.

So to simulate two sites connected by routers on a single Hyper-V host, it makes sense to run a linux-based router fo each site in Hyper-V.  The other option would be to use additional physical NICs to have traffic flow out of one virtual site, just to flow back into the other virtual site on the same server.  For someone unfamilar with router distros, that might be easier, but would require having the routers on hand and the right tools to connect the WAN side of each...not always easy of the routers have integrated non-ethernet WAN connections.  The linux suggestion is the fastest way to simulate the actual connection between sites, and linux router distros have the same benefits as their physical counterparts.  They have a specialized GUI (so you don't see yourself working on linux), ...it just happens to be a router that is running as a VM on Hyper-V.

As for the bit on what I meant for virtual NICs, virtual switches, etc, it sounds like you may not fully understand what Hyper-V does for networking.

A virtual switch is exactly that.  It is a switch, just like a physical switch would be.  Multiple VMs can be "plugged into" a switch, but the switch is its own entity in Hyper-V and is not directly associated with a VM.

A VM, just like a real machine, can have multipe Virtual NICs.  And each virtual NIC can be connected to the same *or different* virtual switches.

Circling back to my "routers run linux" comment.  When you buy a watchguard UTM and it has 6 ports on the front, it is basically a computer, running linux, with six ethernet NICs integrated on the motherboard.  Each NIC can be assigned a different IP, subnet, etc.  Usually done throught the GUI, so you can do things like set up a WAN, a LAN, and one or more DMZs.  Each segment gets its own NIC.

And bringing it full circle, if you diagram your network with actual connections, you can visualize how a router VM would need two virtual NICs.  One for the LAN and one for the WAN.

And in a diagram, you'd usually connect two sites either by VPN, which connects via WAN through the internet (drawn as a cloud, because you don't control that part), or via MPLS  ...which is also an infrastructure you usually don't control...as it is controlled by whoever you lease the lines through.   So you diagram out the WAN connections on each end of the router going over MPLS into a generic switching fabric of some sort...again a cloud is a good representation.

For a simulation, you *have* to replace that cloud with something...or your sites would be totally alienated.  The easiest way to do that is with another virtual switch.  But you can't just yank out routing.   Hyper-V won't magically change basic networking constructs and allow two subnets to talk to each other without routing over layer 3.

Now, in theory, you could get rid of the "connecting" v-switch and one of the routers (routing VMs, and connect both subnets to a single router with multiple virtual NICs...one in each site subnet.  Objectively it'd accomplish the same goual and cut out the extra overhead.  But visually it won't align with any mental diagram you have, as you'd have the same router serving both sites in your simulation...and taht wouldn't be the case in your final planned topology.  That would bother some purists...and it seems you are wanting a purist approach to your simulation, as you explained in your final paragraph.  So my suggestion of two routers and a third v-switch to simulate the MPLS link was purely aesthetic in meeting that goal.


Hope that helps!
Fred MarshallPrincipalAuthor Commented:
Cliff:  Thanks!  We are closer on this than it might appear.
Rather than re-drawing the diagram, let me explain what it's supposed to be saying in words that support what's already drawn:

The Existing is pretty high level and doesn't show the switches.  It's a "cartoon" of the architecture from 30,000 feet and is supposed to emphasize what changes going to the Planned.  

The Planned is trying to show the workgroups replaced by a domain with 2 sites.  A Windows Server 2016/2019? Hyper-V system at each site with, most importantly, a DC VM at each site.  Then, with another VM available, perhaps a file server .. something like that and not decided nor important at this point.

The MPLS remains throughout albeit one all at one place in the testbed.

The Simulation is intended to emulate the Existing (in the small) and to introduce the DCs.
For convenience in the testbed, it uses:
- one workstation per subnet/site
- one internet gateway  per subnet/site
- one Hyper-V platform with two VM DCs - one for each site/subnet and introduces no other server VMs.

With this lashup, we should be able to see/experience:
- setting up Hyper-V management and VMs (once again).
- starting with two interconnected workgroups with file-sharing workstations, setting up the domain and the "sites" on the existing subnets - which emulate the physical sites and subnets.
- making clear the interplay/roles between the two DCs.
- (joining the workstations to the domain).
- controlling access of the shared files on the workstations (acting as a member servers?)
- experimenting with Group Policies and scripts e.g. firewall rules that have scope for both subnets.

I think that's a pretty good outline of my current objectives.  Obviously it's tailored to my LACK of knowledge and experience in doing this type of implementation and my wealth of knowledge knowing what works for this organization and in line with its disaster recovery plans.
Where some things may seem easy or normal to others aren't for me.  I appreciate suggestions to be sure.  But, I have to pick and choose amongst them to make best use of what I know and to avoid what would be big learning sinkholes for me.  So I may be emphasizing things that seem so easy as not to be worthy of much concern to others.  And, I will be avoiding other things that seem easy for others but wouldn't be efficient for me to tackle as a tangential matter.

I hope this clarifies.  

Thanks again!!
RAFAIT CONSULTANTCommented:
Create 2 vSwitch such that vSwitch=A is connected to NIC-A while vSwitch-B is connected to NIC-B. Then each NIC goes to the respective physical switch.

Then assign vNICs on the VMs to the respective vSwitch .

https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v-virtual-switch/hyper-v-virtua...

https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/get-started/create-a-virtual-...

....edit....

Unless you are very familiar with powershell, I would strongly recommend that you use Hyper-V manager.
Fred MarshallPrincipalAuthor Commented:
RAFA:  Got it.  Thanks!  Apparently, NIC-A can connect to a vSwitch but not to a vNIC.  Makes some sense.
The links didn't work but I was able to decipher enough to get the pages opened.

Thanks again!  Exactly what I was looking for.
RAFAIT CONSULTANTCommented:
Hello,

Very well, you can also do the following, it's another idea:

Either have two physical nics (or ports) connected to seperate switches or setup to v-switch and VLANs.

best regards,
Fred MarshallPrincipalAuthor Commented:
Well, that's what I had in mind.  I have plenty of switches available and they don't need *any* setup.  So then it's like this:?

VM to NIC to physical switch?
OR
VM to vSwitch to NIC to physical switch?

Any reason to not use USB-Ethernet adapters instead of PCI NICs for the VMs?
Fred MarshallPrincipalAuthor Commented:
I have the simulation set up and working now -  up to the point that it looks like the "Existing" network.

The plan is to introduce the DCs one at a time so it will be done incrementally with less operational impact, etc.

A new diagram is attached that may help better than earlier ones.

The VMs are set up, the Hyper-V computer has 3 physical NICs, two vSwitches are set up - one for each DC VM.
The Hyper-V platform is connected to an existing subnet in the testbed site.
The vSwitches are each connected to an External connection using one of the physical NICs to simulate each of the DCs connecting in their respective sites/subnets.

Rafa:  Both of the links you provided are dead ends....

The Hyper-V computer has 3 physical NICs.  So, I'm a bit confused by what I'm seeing there:Hyper-V network connectionsThis doesn't seem consistent so I can't glean the "rules" for this kind of setup (which I've never seen before). e.g.
Ethernet 2 AND vEthernet (Virtual Switch Server1) are on Network 4.
Ethernet 3 isn't showing a Network #
vEthernet (Virtual Switch Server 2) is showing on Network 5.
But, one might expect Ethernet 3 and vEthernet (Virtual Switch Server2) would be on the same Network.

I rather believe that I don't yet fully understand the context in which the platform and the VMs (and VSs and vNICs) interact with the physical NICs and the settings in Hyper-V and the VMs....
DC-Architecture-and-Simulation-Rev-1.pdf
Fred MarshallPrincipalAuthor Commented:
Ah!  I figured it out.  The procedure involves:
First set up vSwitches with each of the physical NICs.
Add Virtual MAC address of the VM on Hyper-V per:
https://www.xsofthost.com/help/assign-ip-on-virtual-machines-from-hyper-v-windows-server/amp/
So we assign the intended -NIC MAC addresses to the vSwitches in the Hyper-V platform.
That's one critical connection.
and then:
Configure the Network Adapter inside the VM “Windows Server”
but not what the referenced page says.  Instead, use the intended physical NIC IP address.
So we assign the IP, either static or by DHCP.  I'm using DCHP from the outside subnet router.
Seems to be working perfectly now!!

Now if I could just remember or figure out how the switches got aligned with the VMs!!  :-)  Ah!  it's the MAC address above.....
Cliff GaliherCommented:
NO NO NO!!!!!   You are going farther into the weeds from what I can tell.  You aren't getting closer...

At that point I need to reiterate previous advice that you should call someone in to help you. In person.  I get you are trying to do this yourself, save costs, and learn...but we don't just throw textbooks at children and tell them to learn.  We don't even do that to college kids.  Some concepts just need to be *taught* or they are difficult to grasp, or ask questions "in the moment" and it is even worse trying to spin up a lab, because if by a fluke something "happens" to work, it "teaches" you that is how it was supposed to work "when it wasn't"

You can probably make a car engine run on ethanol with enough experimentation...it wouldn't make the car highway safe.  But someone experimenting and hearing the engine turn over might not realize how dangerous a path they are on....

---

With that said, I suspect you'll ignore that advice, but I'll at least try to prevent you from driving off the edge of the cliff.  

You are tying together too many concepts that aren't meant to be tied together.  So delete what you've done (yes I consider this necessary after reading about assigning "intended MAC addresses to vSwitches, among other tangled yarns I don't think can be unraveled) and let's start over and establish some CORE concepts:

Lets start with the simple switch.  You go and buy a "dumb" Dell switch.  It has a few ports.  But it also runs software that maps MAC addresses of devices to the ports it saw that MAC address on. You don't set a MAC address or an IP address on these kinds of "dumb" switches and you have no control over how they behave really.  But they do run software (that can be broken if you create "loops" or broadcast storms), and that is what differentiates a switch from a hub.  

Okay.  A vSwitch in Hyper-V is no different.  It is a *SWITCH* and nothing more.  It is running in the Hypervisor.  It is not running in, on, or for a VM.  It is running that same kind of MAC-to-port mapping software that the Dell switch does.   Now, if you've ever run out of ports on a switch, you probably know you can just buy another switch and daisy chain them.  Or do a hub-spoke topology (often seen in large buildings) with core switches and edge switches.  And you just avoid aforementioned "loops" (connecting two switches with more than one cable) ...well that is important because....

In Hyper-V, when  you create an "external" vSwitch, you have to pick one of your physical NICs.   That NIC gets co-opted by Hyper-V and becomes a "port" on that particular vSwitch.  When you use a patch cable to plug that physical NIC into another switch, you are just hooking two switches together just like above.  Hyper-V does *everything* it needs to, and configured the physical NIC the way it needs for that to work.  You absolutely, unquestionably, under no circumstances, should be changing the MAC address of the NIC.  Nor the IP address (and in fact, Hyper-V disabled IPv4, so you couldn't without breaking things....)    DON'T CHANGE YOUR NIC PROPERTIES!!!!!!!

You can create more than one external vSwitch, and you can assign each one a different NIC.  This is no different than you having a switch in your house, and your neighbor having a switch in theirs.  Sure, they are both switches, but they don't see each other *AT ALL* ...Hyper-V keeps them separate.  The only way they'd see each other is if you plugged them into the same physical switch or into a router or similar and did configuration (outside of Hyper-V) to make them all part of one larger network.   The point is, however, that each vSwitch is *just a switch* and each NIC is *just a port* on each of those switches and they are, for all intents and purposes, separate entities.    

Now the final piece.  Your VMs.  In the Hyper-V Manager tool (it is JUST A TOOL!!! It is not a running environment), you can view the properties/settings of each VM.  In each VM, you can add a virtual NIC.  This appears inside the VM as if it were a NIC of a computer.   In Hyper-V Manager, if you view that NICs properties, you can select a switch, and you can assign a MAC address, among other things.   I *strongly* recommend leaving this as default which is to let Hyper-V generate  unique MAC address for that NIC, unless you *REALLY* need that to be a specific MAC address for some odd reason like DHCP reservations. The other part is important though.  When you select a vSwitch, you are effectively plugging a patch cable from the VM NIC to the vSwitch.  That's it.  You *aren't* assigning a MAC address to the vSwitch.  You aren't assigning an IP address to the vSwitch.  The *SWITCH* is unchanged.  All you are doing is plugging a (virtual) patch cable from that VM into that switch.  Then that switch is connected with a patch cable (using the physical NIC above), to the rest of your network.

This is important BECAUSE, you can create 10 VMs and assign them all to the same vSwitch.  *IT IS A SWITCH!*  Just like you can buy a 24 port switch, patch 23 machines into it, then use the 24th port to patch it to another switch with 23 more machines.....same concept here.

Your vSwitch can have MANY VMs connected to it with many ports, and then that physical NIC is just an upstream port to another switch or router or whatever.   Your VMs are *not* tied to a physical NIC.  At all.   They have their own "virtual adapter" that is plugged into a virtual switch.  And you could even change that setting in the VM to a different virtual switch...and it'd be just like unplugging a patch cable and plugging it into a different switch.  The virtual adapter's MAC address would not change, and if the IP is static, that would not change either.  IIf it is DHCP, it'd get an address from a DHCP server on that broadcast domain, like any other device.

Which brings me to the final point.  You assign the MAC address in Hyper-V manager (or leave it alone, as is) but everything else is done *IN THE VM!*  You log into the VM, and in the network settings you'll see a network adapter, just like you would a physical machine.  You can go into that network adapter and view its IPv4 and IPv6 settings and *that* is where you'd assign a static IPv4 address if you so choose.   Think of the VM just like a physical machine. The VM doesn't know or care whether it is plugged into a virtual switch, a physical switch, or no switch at all.  It isn't tightly coupled to the vSwitch, and it definitely isn't tied to any physical NIC.    It is a compartmentalized entity LOOSELY connected to the rest of the system by Hyper-V.

Hope that helps.
Fred MarshallPrincipalAuthor Commented:
Cliff:  Once more, I think we are more in agreement than not.  It's the NO! NO! NO!!! that seems a bit much though.
Everything that you've said seems to track with what I'm coming to understand.
Obviously, I'm a neophyte with *this* but not otherwise.
Granted, my terminology and language may be a bit rough at this point.
At least I have a system that's working as I'd intended it to for the testbed - and I find no discord with what you've said.

I appreciate the descriptions.
In the real world, most of this won't matter - but it does for the testbed as I've conjured it up.

I thought I'd posed a pretty simple situation and asked a pretty simple question.  That nobody answered it early on is "interesting".

Allow me to repeat it in the simplest terms possible:

I have a Hyper-V platform in my lab which will support 2 Windows Server 2016 Standard VMs.
I've set up a testbed that will make use of this.

In the testbed, I have 2 subnets / site simulation and I expected to introduce a VM DC in each.
The subnets exist in the real world and are simulated in the real world testbed.
How to introduce / connect the Server VMs to their respective physical subnets to go to work as DCs?
That was the question.
Cliff GaliherCommented:
I think the disconnect is your expectation is for a simple answer. But if you really understood the explanation of How this works then you would have and know the answer.  And since you don't have the answer, nsot experts here will conclude that you don't understand the underlying technology.

All of that comes together to create something of a catch-22. We can't make the answer "simple" because EVERY environment is different and, short of actually asking for physical pictures of every peice of equipment and every config file for every computer and router on the network, we can't account for your environment.  Plus that is definitely something most experts expect to get paid to do... Design in detail.

And when we answer in abstract, you don't find the answer simple enough, and aren't bridging the gap to fill in the details for your environment.

Thus.... A disconnect from what you expect and what most experts can and will offer.  I'm not sure how else to explain the state of affairs here. This seems/feels more like a terminology issue to me. It has the feel of sole fundamental knowledge gaps.

You may take affront to my "No no no" but based on what I read I saw some troubling statements that.... Historically..... Have been big issues.  It is not an exaggeration that I saw someone attempt to configure a physical NIC on a hyper-v server in a live environment with hundreds VMs on it and crashed the system. The cost to the business was significant and the rates a company charges for emergency service is astronomical.

When I see bad habits being formed early that I know can cost significantly down the road, "no no no" is a tually understated. At least compared to a six figure bill of service in American dollars.
Fred MarshallPrincipalAuthor Commented:
Cliff:  I respect your experience and value the inputs.  
If I may suggest a middle ground:
Sometimes a step-by-step set of instructions is needed and can be appropriate.
On the other hand, a 30,000-foot viewpoint description is all that's possible and is all that's needed.
Then, one tailors between the two extremes to suit their understanding of the question and their own capabilities.

I believe that "sole knowledge gaps" is a pretty good way to describe my current understanding of many things.
Before long, the gaps get filled.  But, if my learning style is uncomfortable then perhaps that's a challenge.

After having paraphrased my question in my last post, I take it that you believe the question unworthy of being answered in any possible way one might imagine; since you have not.  I appreciate the endeavor to not mislead.  But, my experience suggests that there is an answer that you can give that will meet all the criteria.  Something like:
"What you are describing would properly be called "xyz".   In general, this entails "abc".  Did you say if you were ......  ?"  
There are no step-by-step instructions here; simply guidance.  
Of course, if you feel like it would be talking into a vacuum then it's not going to be worth your time.

I am beginning to realize that my testbed implementation might be a bit odd for folks - as the two Server VMs are being used independently in some sense - at least through the first phases.  Maybe this makes my question harder to answer.  As before, I'm willing to bare my ignorance in order to improve communication.

I do appreciate the efforts to advise!
Cliff GaliherCommented:
It isn't that I think it is unworthy to be answered. It is that I've already answered it.  Giloing back to my very first comment several weeks agoz it laid out a fairly detailed way to implement the testbed that would mirror your live environment.

You seemed confused by those suggestions so subsequent replies have been an attempt to fill in the knowledge gap to clear up the confusion. But the original comment is STILL the answer you are looking for.  It didn't seem beneficial to repeat it as it's already there.... And if it is still confusing I am not sure how many ways to make it make sense.

*shrug*

You see it as me nti answering. I see it as you not understanding the answer provided.  That's a gap I don't know how to bridge.
Fred MarshallPrincipalAuthor Commented:
Cliff Galiher:  Where can I find the answer that specific answer that you refer to?  I looked but didn't find.....  And, I didn't find a question that I'd asked in that time frame on that topic.  There was one on file sharing but that's not the same topic and you responded but not in the context of this question.  So I'm a bit in the dark.

Also, if I'm not mistaken, this is the first time that I've asked about my plan for the testbed architecture.  It's different, somewhat, from the final production architecture.  So perhaps that's why I don't remember....  Splitting a Hyper-V platform's 2 Server VMs between two "sites" is only part of the testbed and not the final configuration as it's envisioned.

Thanks!
Cliff GaliherCommented:
I am talking about this question, not another question. My typos aside, my first few comments in THIS thread were on April 17th which is "a few weeks ago." And they were fairly detailed and addressed your initial ask.
Fred MarshallPrincipalAuthor Commented:
Cliff:  OK. Well, I was distracted by the assumption that the entire simulation would be on virtual machines and all the details in that context.
As I hope will be clear by now, there are no "router" VMs in the testbed - just DC VMs.
etc.

I don't see the answer to my question and don't see what I've done that's untoward in making this work.  I'd like to understand.  
This should be a very simple testbed implementation wtith 2 DC VMs on one Hyper-V platform and simulating DCs at separate sites.
Now, I thought I'd answered my own question in https://www.experts-exchange.com/questions/29143184/Hyper-V-VMs-physically-on-separate-subnets-howto.html#a42857204 (and edited it to indicate that I'd addressed the last line!!).

To paraphrase, I think this is what that says.  I'm not saying that this is "right" or what is "best", just that it works:
Given:
1) Start with one Hyper-V with 2 Server VMs.
2) Start with the hardware platform having 3 physical NICs.  One for the Hyper-V platform, one for each of the planned physical subnets that are separate from the first.
Remaining Steps:
3) Create 2 vSwitches - with the objective of allowing the use of the DC VMs on each of the 2 physical subnets.  
Associate one to one added physical NIC and one to the other physical NIC by selecting from the pulldown.
4) Also, in Hyper-V, in VM Settings, set the MAC address of each VM vNIC to match the physical NIC MAC address.
5) In each VM, set the vNIC settings for DHCP or with a static IP address as one would do with any NIC.

As above, this seems to work well for the current objectives.  And, I'm still interested in knowing how to do this "better" if there's a way.

If I wanted the VMs to be portable then I imagine that this kind of IP assignment would be unfortunate.  
Maybe fixing that is part of how to do this in a "better" way?

Thanks
Cliff GaliherCommented:
#4 is a bad idea.   Either leave the vNIC with a dynamic MAC address or set it to a MAC address that is unique.  The vNIC is, for all intents and purposes, another NIC.  Which means setting it to the same MAC address as the physical NIC (pNIC) means, by deifnition, it is *NOT* unique and that will come back to haunt you.  

Overall your steps look fine, if that is your question.  If there is another question in there, I missed it.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Fred MarshallPrincipalAuthor Commented:
Thanks!  For now, in this configuration, there is little concern that the vNIC isn't unique.  But I can see where it might be an issue in broader implementations.
I admit that I rather interpreted a more complex set of instructions for a more complex objective.
So, if I had left the vNIC MAC address as dynamic, this would still be working?  ... IP Address assignment, etc.

If I had done that then I don't see the connection between the VM and the pNIC (via the vSwitch)
So, I don't see the connection between the vSwitch and the VM.
The vSwitch knows the pNIC by virtue of the pulldown assignment - so that's the "wire" connecting them.
Where is the vNIC connection to the same vSwitch? Where do we "plug in the wire?"  :-)
Cliff GaliherCommented:
"For now, in this configuration, there is little concern that the vNIC isn't unique."

That should absolutely be a concern.  This is not Hyper-V or VM specific.  TCP/IP on Ethernet assumed uniqueness, and ARP has equired this to work properly for decades.   This is a very BASIC requirement and ,...though it may appear to work for now, something will inevitably break.  

"So, if I had left the vNIC MAC address as dynamic, this would still be working?  ... IP Address assignment, etc"

Yes.  Even DHCP reservations would work unless you move the VM.  In clustered scenarios where mobility is common, that's where you'd assign a static MAC (but it'd still be UNIQUE and not tied to the pNIC.)

"So, I don't see the connection between the vSwitch and the VM."

I actually did answer this.   If you scroll up to my "No no no" comment, scroll down to the paragraph that starts with "the final piece."   And I believe my comments several weeks ago also touched on this.  So its up there for sure.

The virtual adapter in the Hyper-V settings allows you to select which switch.  I've included a screenshot of one of my test VMs with highlights to show this setting.
vNIC.PNG
Fred MarshallPrincipalAuthor Commented:
So then this is what the steps would be:

Given:
1) Start with one Hyper-V with 2 Server VMs.
2) Start with the hardware platform having 3 physical NICs.  One for the Hyper-V platform, one for each of the planned physical subnets that are separate from the first.
Remaining Steps:
3) Create 2 vSwitches - with the objective of allowing the use of the DC VMs on each of the 2 physical subnets.  
Associate one to one added pNIC and one to the other pNIC by selecting from the pulldown.
4) Also, in Hyper-V, in VM Settings, set the Network Adapter / Virtual switch to the switch associated with the intended pNIC.
5) In each VM, set the vNIC settings for DHCP or with a static IP address as one would do with any NIC.

Do I now have it right?

Thanks!
Cliff GaliherCommented:
That sounds right to me, yes.

I definitely recommend taking a methodical approach to all of this though.  The steps aren't wrong, but I'd take the extra time to test individual things, especially given the learning curve.  For example:

*  Name EVERYTHING.  Before you create a vSwitch, take the time to identify each pNIC in the OS and rename that NIC.  Then creating a vSwitch will be easier because the NIC name is easily identifiable in the dropdown.   Name each vSwitch something unique and identifiable.  Then selecting that vSwitch in the virtual adapter dropdown is also easier.

* Create on VM and one vSwitch and test.  Make sure it works on its local subnet as expected.

* Create another VM and another vSwitch and test.   Make sure *it* works on its local subnet as expected.

* Then, and only then, connect the subnets and make sure you can access across them (file shares, ping, etc.)

* Finally, knowing the network is stable, promote your DCs and test replication.

...In other words, we've been focusing on the basics of Hyper-V networking and having that foundation was important, but we never *really* discussed the order of operations for your larger goal/plan.  Without the networknig component actually understood, it always seemed a bit "cart before the horse" to do so...after all, discussing DC replication is moot if the network doesn't work.

But by taking the time to name everything, troubleshooting or understanding what is happening gets a lot easier.  The network and configuration gets close to "self describing" as any other IT pro that needs to service the environment can do so relatively easily if the need arises.
Fred MarshallPrincipalAuthor Commented:
Cliff:  Good advice!  Barely different than what I've been doing.  So we definitely agree once more!
Oh yes, definitely the names!!
The only thing I did in different sequence was to create both vSwitches at once.
The subnets communicate as expected without the Servers.
Next, I turned off one subnet and concentrated on promoting the DC into in one of them.
Next, tested file shares and am addressing some minor issues there right now.
Next step is to connect the other subnet workgroup to see how they play together.
Next step is to promote the other DC in some appropriate fashion - associated with the 2nd subnet/site as much as that makes sense.  It will end up that way physically.

There are still many decisions to make and iterations to go through.
For example, might we be best off by having something like DFS.  Still evaluating the pros and cons of such an approach.  
In a sense, we have this sort of thing now for file backups but not for file serving.

Thanks!
Fred MarshallPrincipalAuthor Commented:
Thanks all!
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Server 2016

From novice to tech pro — start learning today.