?
Solved

Network setup for single ESX host with ASA5505

Posted on 2009-12-28
14
Medium Priority
?
1,207 Views
Last Modified: 2012-05-08
Hi,

I have single server running ESX 4.0. It has total of 7 network interfaces - 6 gigabit NICs and one network interface for Supermicro's SIMSO+ IPMI used for server board's remote management.

I got whole block of public IPs (/24) from my ISP (server is located in small datacenter). There is no firewall (except probably ISP's firewall that I do not have an access) and one 24-switch with about 8 empty ports I can use for my server if I have to (the switch is Netgear smart switch, I think it was GS724AT).

I was told that I should not rely on ESX's built-in firewall. I have a Cisco ASA5505 SEC+ firewall on hand that I hope will be suitable for my setup.


The host will consist of 15-18 VMs grouped in 4-5 groups - like Linux, Windows, test etc. Some of the virtual servers may need separate public IP, some of them can share one IP. I was thinking about separationg groups of servers and utilizing separate NICs for each group if possible. Servers in one group should talk to each other, but servers from different groups probably should stay separated



The problem is that I can't imagine how actually the network setup should looks like.

1. Let's start with ASA. I have 8 ports, one set as outside and second one (is it a good idea at all?) that can be set as DMZ or as backup outside interface. So my first question is about this second interface - what is the best and recommended use for it? I can also use it as inside interface - in this case if I use ASA as switch I will have all my seven NICs connected directly to firewall ports and I will not need a switch.


2. So, I have 6 (or 7) inside ports left. Let's say I will use ASA as switch in addition to firewall. How will I pass these public IPs to network cards? Should I use NAT and DHCP and have private IPs assigned to NICs or..? What is your recommendation about this part of setup?


3. ASA does not have Gigabit ports, just 100Mbit. All NICs are gigabit NICs. Do I need a gigabit switch in my scenario for any reason? I can have one 8- or16-port gigabit switch leaving ASA just with few used ports. What will be the setup in this scenario?


4. Is there any reason I should not use ASA5505? I've heard that ASA is not too good with routing.



Any help and advise will be greatly appreciated. Once we find the right way to go I will ask additional question in separate topic, but first let's find what the right setup is in general. The main goal is to secure the ESX host and particularly service console and to have an flexible option for public IPs.


Thank you!
0
Comment
Question by:MACROLEVEL
  • 6
  • 6
  • 2
14 Comments
 
LVL 32

Assisted Solution

by:nappy_d
nappy_d earned 1800 total points
ID: 26135496
  1. Yes create a DMZ for all of your servers that require a public IP.  All of the servers that require a public IP should be placed on the vswitch that has a NIC assigned to it.(Mydiagram to follow)
  2. Th public IPs are NEVER assigned to the VMNIC but rather ther virtual machines themselves. The ONLY VMNICs that will have an IP is the services consoles.  NEVER, EVER, put this in your DMZ and make it public facing.. (see my diagram to follow)
  3. The iternet is not gigE speed so 100Mb connectivity is absolutely fine.
  4. The AS should be more than ample, when configured correctly.
0
 
LVL 32

Assisted Solution

by:nappy_d
nappy_d earned 1800 total points
ID: 26135633
Here is my sample design for you.

BTW you have 6 NICs on the ESX box right?  Assign two NICs to the service console, two to the DMZ vswitch and the remaining two to the production vswitch.

The NICs assigned to the vswithces will not have any IP addresses.

You should create a port based vlan on your network switch, to which you will connect your NICs from the ASA and the NICs from the vswitch.

Picture-143.png
0
 
LVL 24

Expert Comment

by:ryder0707
ID: 26135802
1. Let's start with ASA. I have 8 ports, one set as outside and second one (is it a good idea at all?) that can be set as DMZ or as backup outside interface. So my first question is about this second interface - what is the best and recommended use for it? I can also use it as inside interface - in this case if I use ASA as switch I will have all my seven NICs connected directly to firewall ports and I will not need a switch.
You dont need all interfaces, 3/6(redundant pair) is good enough. one port/pair for Internal, one port/pair External & one port/pair for DMZ(if you need to publish internal servers to Internet)

2. So, I have 6 (or 7) inside ports left. Let's say I will use ASA as switch in addition to firewall. How will I pass these public IPs to network cards? Should I use NAT and DHCP and have private IPs assigned to NICs or..? What is your recommendation about this part of setup?
Yes NAT is the standard method in this case, you could do 1-1 NAT from each public IP add to each VM internal IP add


3. ASA does not have Gigabit ports, just 100Mbit. All NICs are gigabit NICs. Do I need a gigabit switch in my scenario for any reason? I can have one 8- or16-port gigabit switch leaving ASA just with few used ports. What will be the setup in this scenario?
No its not really needed, you could setup the following from ASA to internal core switch, and your esx connected to this core switch instead not directly to ASA, if your core switch has GigE ports, you should utilize them for connecting to esx hosts


4. Is there any reason I should not use ASA5505? I've heard that ASA is not too good with routing.
Its not to say ASA is not good, best practice is to offload routing to a dedicated layer 3 device and let the ASA do only packet filtering
Normally there is another layer 3 device in front of ASA to do layer 3 routing without any rules whatesoever connecting to ISP media converter
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:MACROLEVEL
ID: 26151771
Thank you for quick response, nappy_d &  ryder0707  :-)

So, I assume based on your comments that HW firewall (ASA5505 in my case) is mandatory in order to protect ESX box.

Now, let's go into details.

5. I think I can't have both second outside interface and DMZ at the same time. If so, what option is better I mean, should I really use DMZ or I can go with some other setup for servers like Exchange.

6. Is it good idea to use dedicated interface (and physical port) on ASA for management only, or it will share outside interface with WAN? I will access firewall (and ESX host) from remote location most of the time so everything that will provide better security and redundancy will be better.

7. My ASA's SEC+ license gives me 8 trunk ports, 20 VLANs and DMZ in addition to Dual ISPs.  I will drop Dual ISP functionality and will set DMZ instead, but can I somehow utilize these max. 20 VLANs I have to divide public IPs and assign range of them to each VLAN? I mean, can I create, say, 4 VLANs (Windows, Linux, Production and Test) and set Windows WLAN to forward all requests to  19x.x44.123.10...19 that will be linked to dedicated physical interface(s) on ASA connected to server's NIC(s) which will be a uplink for vSwitch created for all Windows VMs. Second VLAN on ASA, Linux, similarly will use a range 19x.x44.123.21...30, separate NIC and own vSwitch dedicated just for Linux virtual machines, and so on.

8. If I have DMZ interface set on ASA, it can be connected to one NIC on server box and this NIC can be assigned to just one vSwitch. In this case, if I have 3 different groups of servers (like Microsoft, Linux and, say, Test) connected each to its own vSwitch, how they will share this physical NIC? I mean, I may have servers in each group that may require to be in DMZ. If I use two switch ports on ASA for DMZ, I will have two NICs on server box dedicated for DMZ that I can link to one or two vSwitches. It is difficult for me to imagine at this moment the whole picture (and I am obviously not a vmware or Cisco engineer), but it seems that this should be done on vSwitch level somehow. Am I wrong? What is the right way?

9. As per Cisco, "Logical VLAN interfacesIn routed mode, these interfaces forward traffic between VLAN networks at Layer 3, using the configured security policy to apply firewall and VPN services. In transparent mode, these interfaces forward traffic between the VLANs on the same network at Layer 2, using the configured security policy to apply firewall services." What mode will I use for my setup and why?

Sorry for all these additional questions, but I do not have experience with ASA/ESX and advanced network setups. I have to put this server along with firewall in data center by the end of this week if possible. I will continue the setup remotely, but I should have all cables connected to right interfaces and everything accessible remotely. Sounds complicated? For me, surely. But I will try to do it with your help.

Thank you.
0
 

Author Comment

by:MACROLEVEL
ID: 26151773
ryder0707:

"1. Let's start with ASA. I have 8 ports, one set as outside and second one (is it a good idea at all?) that can be set as DMZ or as backup outside interface. So my first question is about this second interface - what is the best and recommended use for it? I can also use it as inside interface - in this case if I use ASA as switch I will have all my seven NICs connected directly to firewall ports and I will not need a switch.
You dont need all interfaces, 3/6(redundant pair) is good enough. one port/pair for Internal, one port/pair External & one port/pair for DMZ(if you need to publish internal servers to Internet)"

-> All right, let's say port 0/0 is Outside (WAN). 0/1 - dedicated management; 0/2 - IPMI interface; 0/3, 0/4 & 0/5 - inside interfaces connected to separated NICs; 0/6 and 0/7 - DMZs with their own NICs. This way one of server's NIC's will be not connected to firewall.


"2. So, I have 6 (or 7) inside ports left. Let's say I will use ASA as switch in addition to firewall. How will I pass these public IPs to network cards? Should I use NAT and DHCP and have private IPs assigned to NICs or..? What is your recommendation about this part of setup?
Yes NAT is the standard method in this case, you could do 1-1 NAT from each public IP add to each VM internal IP add"

-> Let say I have 19x.x44.123.10....19 I'd like to push via 0/3. If NAT is used, on ASA's LAN side I will have private IPs like 192.168.1.x, right? So how will the vSwitch/VMs get the public IP passed to them? Btw, do I need DHCP server on ASA in this case?


"3. ASA does not have Gigabit ports, just 100Mbit. All NICs are gigabit NICs. Do I need a gigabit switch in my scenario for any reason? I can have one 8- or16-port gigabit switch leaving ASA just with few used ports. What will be the setup in this scenario?
No its not really needed, you could setup the following from ASA to internal core switch, and your esx connected to this core switch instead not directly to ASA, if your core switch has GigE ports, you should utilize them for connecting to esx hosts"

-> What is the reason to use additional gigabit switch if there no shared storage and other hosts? From my understanding on a single ESX host all traffic between VMs will be on vSwitch level. The only thaffic that will travel via server's Gigabit NICs will be from adn to Internet which is limited by ISP anyway. If this is correct, the only reason for separate switch I see is that I am actually one port short which is motherboard's IPMI interface or ESX's second management console interface.


"4. Is there any reason I should not use ASA5505? I've heard that ASA is not too good with routing.
Its not to say ASA is not good, best practice is to offload routing to a dedicated layer 3 device and let the ASA do only packet filtering
Normally there is another layer 3 device in front of ASA to do layer 3 routing without any rules whatesoever connecting to ISP media converter"

-> So, should I put a router (or you meant layer 3 switch?) between ISP's switch and ASA firewall? I do not expect too much traffic/connections from outside.

Thank you.
0
 
LVL 32

Assisted Solution

by:nappy_d
nappy_d earned 1800 total points
ID: 26151865
5. I think I can't have both second outside interface and DMZ at the same time. If so, what option is better I mean, should I really use DMZ or I can go with some other setup for servers like Exchange.
  • keep your Exchange server on the LAN and just forward port 25, maybe 443 for OWA etc.
  • If you have webservers, I would suggest they go into your DMZ.
6. Is it good idea to use dedicated interface (and physical port) on ASA for management only, or it will share outside interface with WAN? I will access firewall (and ESX host) from remote location most of the time so everything that will provide better security and redundancy will be better.
  • NEVER EVER, make your firewall's management interface be accessible publicly from the Internet.  Always use a VPN connection if you need to manage it remotely.  I would suggest that you use the LAN interface to manage your ASA.
7. My ASA's SEC+ license gives me 8 trunk ports, 20 VLANs and DMZ in addition to Dual ISPs.  I will drop Dual ISP functionality and will set DMZ instead, but can I somehow utilize these max. 20 VLANs I have to divide public IPs and assign range of them to each VLAN? I mean, can I create, say, 4 VLANs (Windows, Linux, Production and Test) and set Windows WLAN to forward all requests to  19x.x44.123.10...19 that will be linked to dedicated physical interface(s) on ASA connected to server's NIC(s) which will be a uplink for vSwitch created for all Windows VMs. Second VLAN on ASA, Linux, similarly will use a range 19x.x44.123.21...30, separate NIC and own vSwitch dedicated just for Linux virtual machines, and so on.
  • YES!  Based on my diagram from above, you can creave serveral vswitches each connected to a port based vlan or you could configure VGT. But port based vlans is easier and faster to implement.
8. If I have DMZ interface set on ASA, it can be connected to one NIC on server box and this NIC can be assigned to just one vSwitch. In this case, if I have 3 different groups of servers (like Microsoft, Linux and, say, Test) connected each to its own vSwitch, how they will share this physical NIC? I mean, I may have servers in each group that may require to be in DMZ. If I use two switch ports on ASA for DMZ, I will have two NICs on server box dedicated for DMZ that I can link to one or two vSwitches. It is difficult for me to imagine at this moment the whole picture (and I am obviously not a vmware or Cisco engineer), but it seems that this should be done on vSwitch level somehow. Am I wrong? What is the right way?
  • The short answer is they can't.  A NIC can only be assigned to one vswitch at a time.  Now you could have a vswitch listening on several vlans but I think you will have to configure VTG.  Read more here:
0
 
LVL 24

Assisted Solution

by:ryder0707
ryder0707 earned 200 total points
ID: 26152003
-> Let say I have 19x.x44.123.10....19 I'd like to push via 0/3. If NAT is used, on ASA's LAN side I will have private IPs like 192.168.1.x, right? So how will the vSwitch/VMs get the public IP passed to them? Btw, do I need DHCP server on ASA in this case?
You need to assign static private address to each VM to do the 1-1 mapping, the ASA will manage this ofcoz

-> What is the reason to use additional gigabit switch if there no shared storage and other hosts? From my understanding on a single ESX host all traffic between VMs will be on vSwitch level. The only thaffic that will travel via server's Gigabit NICs will be from adn to Internet which is limited by ISP anyway. If this is correct, the only reason for separate switch I see is that I am actually one port short which is motherboard's IPMI interface or ESX's second management console interface.
For extra bandwidth ofcoz and depends on the load of internal traffic, gigabit will give you extra buffer just in case traffic/load increase
From time to time, you may want to copy out the vmdk to other machine for testing/backup purposes
Have you considered backup? most backup software recommends at least gigabit due to backup performance considerations
Perhaps you have only one host now, what if you add extra server in the future? vmotion needs gigabit as well
A lot of things you can do if you have the xtra bandwidth


-> So, should I put a router (or you meant layer 3 switch?) between ISP's switch and ASA firewall? I do not expect too much traffic/connections from outside.
Yeah its pretty common this way
0
 

Author Comment

by:MACROLEVEL
ID: 26154918
nappy_d:

6. Is it good idea to use dedicated interface (and physical port) on ASA for management only, or it will share outside interface with WAN? I will access firewall (and ESX host) from remote location most of the time so everything that will provide better security and redundancy will be better.

    * NEVER EVER, make your firewall's management interface be accessible publicly from the Internet.  Always use a VPN connection if you need to manage it remotely.  I would suggest that you use the LAN interface to manage your ASA.

-> how will I manage ASA remotely then?


7. My ASA's SEC+ license gives me 8 trunk ports, 20 VLANs and DMZ in addition to Dual ISPs.  I will drop Dual ISP functionality and will set DMZ instead, but can I somehow utilize these max. 20 VLANs I have to divide public IPs and assign range of them to each VLAN? I mean, can I create, say, 4 VLANs (Windows, Linux, Production and Test) and set Windows WLAN to forward all requests to  19x.x44.123.10...19 that will be linked to dedicated physical interface(s) on ASA connected to server's NIC(s) which will be a uplink for vSwitch created for all Windows VMs. Second VLAN on ASA, Linux, similarly will use a range 19x.x44.123.21...30, separate NIC and own vSwitch dedicated just for Linux virtual machines, and so on.

    * YES!  Based on my diagram from above, you can creave serveral vswitches each connected to a port based vlan or you could configure VGT. But port based vlans is easier and faster to implement.


-> still not clear. I will create vSwitches on ASA box. I will have several VLANs on ASA as well - DMZ and some for different groups of servers, right? Can you please give me more details about this part of setup?
0
 
LVL 32

Expert Comment

by:nappy_d
ID: 26155724
That is fine to assign an interface for management.  Just do not make accessible from the internet without a VPN connection.
0
 

Author Comment

by:MACROLEVEL
ID: 26186891
Here is what I collected so far as recommended solution:


Internet--ASA--pSwitch--Server--ESX


1. ASA will use Static IP on outside interface and will pass range of public IPs to inside/DMZ interfaces

2. I will put 8-port Netgear GS716T managed switch between ASA and server box for VMs that are not in DMZ. Do I need a second switch for DMZ? Or maybe I should use my Dell Power Connect 2716 managed switch instead? Do I have to pay attention for VLAN tagging and any other functionality that can be useful for my setup? I can actually create VLANs on the switch as well, and probably that's the right way to do it, am I correct? If I utilize VLANs on the switch, how will I then assign public IPs to virtual servers?

3. Here is ASA's port assigment plan, please correct me if I am wrong or you recommend better setup:

ASA port 0/0 - outside (WAN) [Public IP]
ASA port 0/1 - ASA management only [Public IP]
ASA port 0/2 - IPMI server hardware management module  [Connected to IPMI NIC on server box]
ASA port 0/3 - inside [Connected to Gigabit switch]
ASA port 0/4 - inside [Connected to NIC 0/3 on server box]
ASA port 0/5 -
ASA port 0/6 - ASA DMZ [Connected to NIC 0/4 on server box]
ASA port 0/7 - ASA DMZ [Connected to NIC 0/5 on server box]

If you recommend VLANs on the switch, then I will probably have most of ASAs ports connected to ports on the switch assigned to different VLANs. I still think about having one management interface for ESX connected directly to ASA, not through the switch so if the switch is down I will still have an access to management console.


3. IPMI interface: Should I put ASA port 0/2 (IPMI interface that is used strictly for physical server management) in DMZ VLAN or it is better to have it in dedicated VLAN on ASA with own public IP and proper rules set? Not sure what is recommended and can't find anything in manual (http://www.supermicro.com/manuals/other/AOC-SIMSO.pdf)

I found another information: "IPMI uses the same UDP port number (623 in decimal) with ASF (Alert Standard Forum) protocol. If the managed system is protected by a firewall, UDP port 623 must be opened."

I am going to wait for answers before splitting the general setup theme on some smaller question. Thanks!
0
 
LVL 32

Accepted Solution

by:
nappy_d earned 1800 total points
ID: 26186951
  1. Yes
  2. I would recommend that you go with port based vlans for connectivity.  Which ever one of those switches you referenced allows you to go this route would be my recommendation.  Yes it is the only way to create vLANs is on the switch.  You will assign Public IPs to the servers that are connected to the vSwitch which is setup for the DMZ and NOT to the NICs of the ESX host that is in the DMZ.
  3. ASA port 0/0 - outside (WAN) [Public IP]
    ASA port 0/1 - ASA management only [Public IP]
    ASA port 0/2 - IPMI server hardware management module  [Connected to IPMI NIC on server box]
    ASA port 0/3 - inside [Connected to Gigabit switch]
    ASA port 0/4 - inside [Connected to switch and assigned to vlan1
    ASA port 0/5 -
    ASA port 0/6 - ASA DMZ Connected to vlan 2
    ASA port 0/7 -
0
 

Author Comment

by:MACROLEVEL
ID: 26191903
nappy_d:

Based on your last answer, there will be no other cable connection between ASA and server box except the IPMI connection. All ASA's inside/DMZ interfaces will go through the physical switch to server via separated VLANs on it, right? I was thinking about having two ESX console interfaces connected via two separate NICs. We have two unused ports on ASA, so we can probably use one of them connected directly to servers's NIC as backup management console interface, and I assume the main management console will be running via (probably) separate VLAN on the switch, is that right?

I believe both switches support port-based VLANs, the only thing that bothers me is that one additional device is a potential point of failure in addition to network design complexity, that's why I was thinking of using ASA without a switch. So if you think I still can go without switch for my setup (and I can always add a switch later in case I will use shared storage or will add additional hosts), please let me know.

Thanks.
0
 
LVL 32

Expert Comment

by:nappy_d
ID: 26192133
I would still tell you to use a switch.  (Lucky me) I've yet to have a switch fail on me.

You only need ONE ESX ervice console and what you do for redundancy, is to add multiple NICs to the service console for failover.

I think you may be over-thinking your setup.
0
 

Author Comment

by:MACROLEVEL
ID: 26192389
nappy_d: you'll probably absolutely right about over-thinking my setup. I am going with solution/setup you recommend. I'll ask additional questions in separate topics once I do initial setup and have all devices connected.

Thank you both for your help!
0

Featured Post

Prepare for your VMware VCP6-DCV exam.

Josh Coen and Jason Langer have prepared the latest edition of VCP study guide. Both authors have been working in the IT field for more than a decade, and both hold VMware certifications. This 163-page guide covers all 10 of the exam blueprint sections.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this article, I am going to show you how to simulate a multi-site Lab environment on a single Hyper-V host. I use this method successfully in my own lab to simulate three fully routed global AD Sites on a Windows 10 Hyper-V host.
This article will explain How to fix Broken backup chain in Veeam Backup & Replication.
Teach the user how to configure vSphere clusters to support the VMware FT feature Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …
When cloud platforms entered the scene, users and companies jumped on board to take advantage of the many benefits, like the ability to work and connect with company information from various locations. What many didn't foresee was the increased risk…
Suggested Courses

807 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question