Clarification of Step Order Netscaler HA Setup with FIPS
Hello. We have a discrepency on the order of steps we need to enable FIPS in a HA setup for Netscaler MPX9700. These are running version 11.1.
From the Articles, such as https://docs.citrix.com/en-us/netscaler/12-1/ssl/fips/configure-fips-ha.html and https://docs.citrix.com/en-us/netscaler/12/getting-started-with-netscaler/configure-fips-first-time.html, it reads as though you start with the HSM/FIPS module and then the HA portion of the GUI. We are planning to use a WildCard for the certificate on the FIPS module and the URL's provided to users.
However; from research a co-worker insists that the HA portion through the GUI needs to be setup first, and then do the HSM/FIPS portion.
Any clarification from experience is appreciated.
From the Articles, such as https://docs.citrix.com/en-us/netscaler/12-1/ssl/fips/configure-fips-ha.html and https://docs.citrix.com/en-us/netscaler/12/getting-started-with-netscaler/configure-fips-first-time.html, it reads as though you start with the HSM/FIPS module and then the HA portion of the GUI. We are planning to use a WildCard for the certificate on the FIPS module and the URL's provided to users.
However; from research a co-worker insists that the HA portion through the GUI needs to be setup first, and then do the HSM/FIPS portion.
Any clarification from experience is appreciated.
Do more with
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
I found an old response, believe that this is a repeat conversation. I will try again using this advice as this conversation came around again as it was not working for us. - The advice "You really need to configure the FIPS and HSM before you do the HA or anything else. Do this first. Do FIPS, do the HA config, then configure storefront/gateway"
The secondary is that we need to use a WildCard certificate. If there is anything additional to add here, please let me know.
Thanks
The secondary is that we need to use a WildCard certificate. If there is anything additional to add here, please let me know.
Thanks
I've done several of these configurations, 12 Netscalers at one contract alone. 8 x 12000 series and 4 x 9700's. Perhaps I can assist.
Configuring a FIPS appliance involves configuring the HSM immediately after completing the generic configuration process. You then create or import a FIPS key. After creating a FIPS key, you should export it for backup AND you need this FIPS key to configure HA. Hence, first must complete steps below to initialize the HSM, which first requires basic configuration, then initialize the HSM, then generate FIPS key, then export key, then configure HA.
1. Complete the initial hardware configuration first, using:
https://docs.citrix.com/en-us/netscaler-hardware-platforms/mpx/netscaler-initial-configuration.html
2. Before initializing the HSM, you can upgrade to the latest build of the software on the Netscaler. (Optional)
Reference: https://docs.citrix.com/en-us/netscaler/12-1/upgrade-downgrade-netscaler-appliance.html
3. Change the SO password (see below)
Configuring HSM erases anything beyond the basic configuration. I would change the SO password during this time as well, 14 character max, no symbols. Why? Because if you don't it will retain the password set at time of HSM initialize. In other words, I would not want the initial password bound to the HSM. Changing it after HSM initialization won't matter if you reinitialize later on.
4. Initialize HSM module using CLI (I prefer CLI but can use GUI)
A. Logon to appliance as super user (assumes initial configuration completed)
Type the following commands to configure the HSM and verify the configuration:
show ssl fips
reset ssl fips
reboot
** Post Reboot
set ssl fips -initHSM Level-2 \<newSOpassword\> \<oldSOpassword\> \<userPassword\> \[-hsmLabel \<string\>\]
save ns config
reboot
** Post 2nd Reboot
show ssl fips
5. Create FIPS Keys
Option 1 - Using GUI
You must specify the key type (RSA or ECDSA) and specify the curve for ECDSA keys.
Create a FIPS key by using the GUI
Navigate to Traffic Management > SSL > FIPS.
In the details pane, on the FIPS Keys tab, click Add.
In the Create FIPS Key dialog box, specify values for the following parameters:
FIPS Key Name*—fipsKeyName
Modulus*—modulus
Exponent*—exponent
*A required parameter
Click Create, and then click Close.
On the FIPS Keys tab, verify that the settings displayed for the FIPS key that you just created are correct.
Create a FIPS key by using the CLI (Option 2)
create ssl fipsKey \<fipsKeyName\> -modulus \<positive\_integer\> \[-exponent ( 3 | F4 )\]
show ssl fipsKey \[\<fipsKeyName\>\]
Example:
1. create fipskey Key-FIPS-1 -keytype RSA -modulus 2048 -exponent 3
2. show ssl fipsKey Key-FIPS-1
3. FIPS Key Name: Key-FIPS-1 Key Type: RSA Modulus: 2048 Public Exponent: F4 (Hex: 0x10001)
6. Export FIPS Key
Option 1: CLI
export ssl fipsKey \<fipsKeyName\> -key \<string\>
Example: export fipskey Key-FIPS-1 -key Key-FIPS-1.key
Option 2: GUI
Navigate to Traffic Management > SSL > FIPS
In the details pane, on the FIPS Keys tab, click Export.
In the Export FIPS key to a file dialog box, specify values for the following parameters:
FIPS Key Name*—fipsKeyName
File Name*—key (To put the file in a location other than the default, you can either specify the complete path or click the Browse button and navigate to a location.)
*A required parameter
Click Export, and then click Close.
7. Repeat steps 1-4 on HA partner
8. Import SSL FIPS Key
Option 1: CLI
import ssl fipsKey \<fipsKeyName\> -key \<string\> -inform SIM -exponent (F4 | 3)
show ssl fipskey \<fipsKeyName\>
Example:
import fipskey Key-FIPS-2 -key Key-FIPS-2.key -inform SIM -exponent F4
show ssl fipskey key-FIPS-2
FIPS Key Name: Key-FIPS-2 Modulus: 2048 Public Exponent: F4 (Hex value 0x10001)
Option 2: GUI
Import a FIPS key by using the GUI
Navigate to Traffic Management > SSL > FIPS
In the details pane, on the FIPS Keys tab, click Import.
In the Import as a FIPS Key dialog box, select FIPS key file and set values for the following parameters:
FIPS Key Name*
Key File Name*—To put the file in a location other than the default, you can either specify the complete path or click Browse and navigate to a location.
Exponent*
*A required parameter
Click Import, and then click Close.
On the FIPS Keys tab, verify that the settings displayed for the FIPS key that you just imported are correct.
8. Now you can configure HA
https://docs.citrix.com/en-us/netscaler/12/ssl/fips/configure-fips-ha.html
Hope this helps.
Configuring a FIPS appliance involves configuring the HSM immediately after completing the generic configuration process. You then create or import a FIPS key. After creating a FIPS key, you should export it for backup AND you need this FIPS key to configure HA. Hence, first must complete steps below to initialize the HSM, which first requires basic configuration, then initialize the HSM, then generate FIPS key, then export key, then configure HA.
1. Complete the initial hardware configuration first, using:
https://docs.citrix.com/en-us/netscaler-hardware-platforms/mpx/netscaler-initial-configuration.html
2. Before initializing the HSM, you can upgrade to the latest build of the software on the Netscaler. (Optional)
Reference: https://docs.citrix.com/en-us/netscaler/12-1/upgrade-downgrade-netscaler-appliance.html
3. Change the SO password (see below)
Configuring HSM erases anything beyond the basic configuration. I would change the SO password during this time as well, 14 character max, no symbols. Why? Because if you don't it will retain the password set at time of HSM initialize. In other words, I would not want the initial password bound to the HSM. Changing it after HSM initialization won't matter if you reinitialize later on.
4. Initialize HSM module using CLI (I prefer CLI but can use GUI)
A. Logon to appliance as super user (assumes initial configuration completed)
Type the following commands to configure the HSM and verify the configuration:
show ssl fips
reset ssl fips
reboot
** Post Reboot
set ssl fips -initHSM Level-2 \<newSOpassword\> \<oldSOpassword\> \<userPassword\> \[-hsmLabel \<string\>\]
save ns config
reboot
** Post 2nd Reboot
show ssl fips
5. Create FIPS Keys
Option 1 - Using GUI
You must specify the key type (RSA or ECDSA) and specify the curve for ECDSA keys.
Create a FIPS key by using the GUI
Navigate to Traffic Management > SSL > FIPS.
In the details pane, on the FIPS Keys tab, click Add.
In the Create FIPS Key dialog box, specify values for the following parameters:
FIPS Key Name*—fipsKeyName
Modulus*—modulus
Exponent*—exponent
*A required parameter
Click Create, and then click Close.
On the FIPS Keys tab, verify that the settings displayed for the FIPS key that you just created are correct.
Create a FIPS key by using the CLI (Option 2)
create ssl fipsKey \<fipsKeyName\> -modulus \<positive\_integer\> \[-exponent ( 3 | F4 )\]
show ssl fipsKey \[\<fipsKeyName\>\]
Example:
1. create fipskey Key-FIPS-1 -keytype RSA -modulus 2048 -exponent 3
2. show ssl fipsKey Key-FIPS-1
3. FIPS Key Name: Key-FIPS-1 Key Type: RSA Modulus: 2048 Public Exponent: F4 (Hex: 0x10001)
6. Export FIPS Key
Option 1: CLI
export ssl fipsKey \<fipsKeyName\> -key \<string\>
Example: export fipskey Key-FIPS-1 -key Key-FIPS-1.key
Option 2: GUI
Navigate to Traffic Management > SSL > FIPS
In the details pane, on the FIPS Keys tab, click Export.
In the Export FIPS key to a file dialog box, specify values for the following parameters:
FIPS Key Name*—fipsKeyName
File Name*—key (To put the file in a location other than the default, you can either specify the complete path or click the Browse button and navigate to a location.)
*A required parameter
Click Export, and then click Close.
7. Repeat steps 1-4 on HA partner
8. Import SSL FIPS Key
Option 1: CLI
import ssl fipsKey \<fipsKeyName\> -key \<string\> -inform SIM -exponent (F4 | 3)
show ssl fipskey \<fipsKeyName\>
Example:
import fipskey Key-FIPS-2 -key Key-FIPS-2.key -inform SIM -exponent F4
show ssl fipskey key-FIPS-2
FIPS Key Name: Key-FIPS-2 Modulus: 2048 Public Exponent: F4 (Hex value 0x10001)
Option 2: GUI
Import a FIPS key by using the GUI
Navigate to Traffic Management > SSL > FIPS
In the details pane, on the FIPS Keys tab, click Import.
In the Import as a FIPS Key dialog box, select FIPS key file and set values for the following parameters:
FIPS Key Name*
Key File Name*—To put the file in a location other than the default, you can either specify the complete path or click Browse and navigate to a location.
Exponent*
*A required parameter
Click Import, and then click Close.
On the FIPS Keys tab, verify that the settings displayed for the FIPS key that you just imported are correct.
8. Now you can configure HA
https://docs.citrix.com/en-us/netscaler/12/ssl/fips/configure-fips-ha.html
Hope this helps.
Brian,
Thank you so much for the detailed steps, it is truly appreciated. One more question, as far as the FIPS key itself, when you generate this you are actually binding a certificate to this module? That's where I'm a little unsure if I understand what this is setting exactly. We need to use a WildCard cert and with that, confirming this is where I'd add that as well as where I'd update when the certificate is renewed.
Thanks in advance.
Thank you so much for the detailed steps, it is truly appreciated. One more question, as far as the FIPS key itself, when you generate this you are actually binding a certificate to this module? That's where I'm a little unsure if I understand what this is setting exactly. We need to use a WildCard cert and with that, confirming this is where I'd add that as well as where I'd update when the certificate is renewed.
Thanks in advance.
Best use case for your wildcard certificate would be for "Content Switch" or "SSL VIP" on the Netscaler HA Pair post configuration of HSM. The private key we generate on the HSM when creating the key is for that Netscaler device only but being you are implementing HA pair you can export that key and use on the other device.
Once you HA pair is configured you can then import the "wildcard" certificate to both appliances. This is separate from the HSM configuration.
Keeping in mind that the HSM configuration and that certificate is specific to that appliance or the HA pair only and where only the Super User account would access this component.
Your wildcard certificate would be more specific to a valid registered domain name that you want others to use to connect over TLS\HTTPS.
You get the most value out of the wildcard certificate using Content Switch where you can essentially write infinite rules to redirect traffic to virtual services defined down-level on the Netscaler.
Or, you can bind it to SSL VIP that would also use services.
For the internal only services where you want to maintain encryption you can use private certificates from internal CA as long as it is Netscaler to Netscaler communication OR Netscaler to internal defined services like StoreFront, XML, etc... and where the public/private key reside on both appliances using the same import process stated prior. Depending on how those certificates are generated you might need to convert but we can cross that bridge if we come to it.
Best to use public certificates for your Content Switch or SSL VIP otherwise you have to provide all down-level clients with both the public/private key and import that to each CSR. This is to use Load Balancing between services by placing an SSL/TLS VIP or content switch on the front end. The content switch, for example, can front end thousands of websites using a single content switch with a single wildcard certificate.
Regarding the wildcard certificate and updating.... Your public/private key from a public provider or internal CA is imported to both Netscalers using CLI or GUI and that key pair is held in the SSL directory which you can see using WinSCP. Using WinSCP, you copy the key pair to both Netscalers. You then import the key pair. Then you bind it to Content Switch, or SSL VIP.
Additional reference on SSL: https://docs.citrix.com/en-us/netscaler/12/ssl/how-to-articles.html
Content Switch configuration: https://docs.citrix.com/en-us/netscaler/12/ssl/how-to-articles/config-secure-cs-vserver.html
The content switch does require advanced level configuration which we can work through if you wish.
The simpler configuration is SSL VIP where you simply want to manage traffic back to your Citrix StoreFront servers, for example, and does not require a wildcard but simply a public/private key pair bound to both Netscalers, copy the key pair over - might require conversion - followed by importing the key pair AND intermediate chain on each appliance, followed by binding that to your CS or SSL VIP.app
Reference this concept but traffic is encrypted: https://docs.citrix.com/en-us/netscaler/12/getting-started-with-netscaler/load-balancing.html
This process for the key pair, intermediate chain and linking appears to be adequately documented here:
https://docs.citrix.com/en-us/netscaler/12/ssl/ssl-certificates/add-group-certs.html
In addition, I recommend one of my articles that goes in more depth regarding SSL:
https://www.experts-exchange.com/articles/25021/Citrix-SSL-TLS-Vulnerabilities-and-Operating-System-Hardening.html
Let me know if this helps or you have any other questions.
Additional links of interest:(for later configs - not much we cannot achieve with Netscaler) Here are a few examples:
https://www.experts-exchange.com/articles/25020/Monitor-Internal-MSSQL-Database-with-Citrix-Netscaler-Advanced-Monitor.html
https://www.experts-exchange.com/articles/25019/Citrix-NetScaler-HTTP-Compression-Feature.html
Once you HA pair is configured you can then import the "wildcard" certificate to both appliances. This is separate from the HSM configuration.
Keeping in mind that the HSM configuration and that certificate is specific to that appliance or the HA pair only and where only the Super User account would access this component.
Your wildcard certificate would be more specific to a valid registered domain name that you want others to use to connect over TLS\HTTPS.
You get the most value out of the wildcard certificate using Content Switch where you can essentially write infinite rules to redirect traffic to virtual services defined down-level on the Netscaler.
Or, you can bind it to SSL VIP that would also use services.
For the internal only services where you want to maintain encryption you can use private certificates from internal CA as long as it is Netscaler to Netscaler communication OR Netscaler to internal defined services like StoreFront, XML, etc... and where the public/private key reside on both appliances using the same import process stated prior. Depending on how those certificates are generated you might need to convert but we can cross that bridge if we come to it.
Best to use public certificates for your Content Switch or SSL VIP otherwise you have to provide all down-level clients with both the public/private key and import that to each CSR. This is to use Load Balancing between services by placing an SSL/TLS VIP or content switch on the front end. The content switch, for example, can front end thousands of websites using a single content switch with a single wildcard certificate.
Regarding the wildcard certificate and updating.... Your public/private key from a public provider or internal CA is imported to both Netscalers using CLI or GUI and that key pair is held in the SSL directory which you can see using WinSCP. Using WinSCP, you copy the key pair to both Netscalers. You then import the key pair. Then you bind it to Content Switch, or SSL VIP.
Additional reference on SSL: https://docs.citrix.com/en-us/netscaler/12/ssl/how-to-articles.html
Content Switch configuration: https://docs.citrix.com/en-us/netscaler/12/ssl/how-to-articles/config-secure-cs-vserver.html
The content switch does require advanced level configuration which we can work through if you wish.
The simpler configuration is SSL VIP where you simply want to manage traffic back to your Citrix StoreFront servers, for example, and does not require a wildcard but simply a public/private key pair bound to both Netscalers, copy the key pair over - might require conversion - followed by importing the key pair AND intermediate chain on each appliance, followed by binding that to your CS or SSL VIP.app
Reference this concept but traffic is encrypted: https://docs.citrix.com/en-us/netscaler/12/getting-started-with-netscaler/load-balancing.html
This process for the key pair, intermediate chain and linking appears to be adequately documented here:
https://docs.citrix.com/en-us/netscaler/12/ssl/ssl-certificates/add-group-certs.html
In addition, I recommend one of my articles that goes in more depth regarding SSL:
https://www.experts-exchange.com/articles/25021/Citrix-SSL-TLS-Vulnerabilities-and-Operating-System-Hardening.html
Let me know if this helps or you have any other questions.
Additional links of interest:(for later configs - not much we cannot achieve with Netscaler) Here are a few examples:
https://www.experts-exchange.com/articles/25020/Monitor-Internal-MSSQL-Database-with-Citrix-Netscaler-Advanced-Monitor.html
https://www.experts-exchange.com/articles/25019/Citrix-NetScaler-HTTP-Compression-Feature.html
Ok, thank you for the explanation. I'm going to give your steps a try and see if I can get a successful FIPS/HA going. We've been struggling with one set of devices in particular. Looks from your steps, I was doing similar, but thinking I missed something.
Will comment here if I need further help with the WildCard. We do have one working HA/FIPS (though the individual that got this working couldn't confirm what steps he did). I tried to update the Renewed WildCard cert, and it gave me errors indicating it was not a FIPS compliant cert. Therefore why all my questions around that.
Thanks again for your help.
Will comment here if I need further help with the WildCard. We do have one working HA/FIPS (though the individual that got this working couldn't confirm what steps he did). I tried to update the Renewed WildCard cert, and it gave me errors indicating it was not a FIPS compliant cert. Therefore why all my questions around that.
Thanks again for your help.
Brian,
One more question. I see here after the primary is done, you move to the secondary for these steps:
7. Repeat steps 1-4 on HA partner
8. Import SSL FIPS Key
Option 1: CLI
import ssl fipsKey \<fipsKeyName\> -key \<string\> -inform SIM -exponent (F4 | 3)
show ssl fipskey \<fipsKeyName\>
Example:
import fipskey Key-FIPS-2 -key Key-FIPS-2.key -inform SIM -exponent F4
show ssl fipskey key-FIPS-2
FIPS Key Name: Key-FIPS-2 Modulus: 2048 Public Exponent: F4 (Hex value 0x10001)
My Question is are you renaming the FIPS key to include the key-FIPS-2, so that each HA has an individual identity or is this supposed to be the same name as you are importing the one from the Primary Device?
Thanks
One more question. I see here after the primary is done, you move to the secondary for these steps:
7. Repeat steps 1-4 on HA partner
8. Import SSL FIPS Key
Option 1: CLI
import ssl fipsKey \<fipsKeyName\> -key \<string\> -inform SIM -exponent (F4 | 3)
show ssl fipskey \<fipsKeyName\>
Example:
import fipskey Key-FIPS-2 -key Key-FIPS-2.key -inform SIM -exponent F4
show ssl fipskey key-FIPS-2
FIPS Key Name: Key-FIPS-2 Modulus: 2048 Public Exponent: F4 (Hex value 0x10001)
My Question is are you renaming the FIPS key to include the key-FIPS-2, so that each HA has an individual identity or is this supposed to be the same name as you are importing the one from the Primary Device?
Thanks
The name is merely for you to distinguish this as the FIPS HSM certificate and you do not need to modify anything to import on other appliance. Actually, you want to keep them identical due to the HA synchronization process. If you rename it, you would then have two names in the /SSL directory where you only want the one.
Regarding the other ciphers question and compatibility you can also reference:
https://docs.citrix.com/en-us/netscaler/12-1/media/cipher-support-on-a-citrix-mpx-9700-fips-appliance-with-firmware-2-2.pdf
Regarding the other ciphers question and compatibility you can also reference:
https://docs.citrix.com/en-us/netscaler/12-1/media/cipher-support-on-a-citrix-mpx-9700-fips-appliance-with-firmware-2-2.pdf
Hello. I've tried following your detailed steps a few times now and end up with the same result. I am able to generate the key and follow the steps, where it errors is the import on the secondary device. I get the same error each time.
So, not sure what I'm missing here. It also takes several minutes to do the reboot on the device after the set ssl fips -initHSM step.
So for clarity, I've provided my steps (removed passwords)
show ssl fips
reset ssl fips
reboot
set ssl fips -initHSM Level-2 <New Password> so12345 <New Password> -hsmLabel wild_fips_cw
save ns config
reboot
show ssl fips
From GUI - Traffic Management/SSL/FIPS
Keys - ADD
create fipskey wild_fips_cw -keytype RSA -modulus 2048 -exponent F4
show ssl fipsKey wild_fips_cw
Should display
FIPS Key Name: wild_fips_cw Key Type: RSA Modulus: 2048 Public Exponent: F4 (Hex: 0x10001)
CLI -
export fipskey wild_fips_cw -key wild_fips_cw.key
*Also do from GUI to ensure a good backup
Traffic Management/SSL/FIPS, select key export - *Ensure you use a different file path so you don't overwrite the keys
*Copy local to my device using WinSCP and then use WinSCP to copy to secondary device*
Go to secondary Node
show ssl fips
reset ssl fips
reboot
set ssl fips -initHSM Level-2 <New Password> so12345 <New Password> -hsmLabel wild_fips_cw
save ns config
reboot
CLI
import fipskey wild_fips_cw -key wild_fips_cw.key -inform SIM -exponent F4
(Says can’t find the key and then I ended up trying to import via the GUI) - That’s when I get the “internal error”
show ssl fipskey wild_fips_cw
FIPS Key Name: wild_fips_cw Modulus: 2048 Public Exponent: F4 (Hex value 0x10001)
Any additional thoughts are appreciated.
Thanks
FIPS_Internal_Import_error.png
So, not sure what I'm missing here. It also takes several minutes to do the reboot on the device after the set ssl fips -initHSM step.
So for clarity, I've provided my steps (removed passwords)
show ssl fips
reset ssl fips
reboot
set ssl fips -initHSM Level-2 <New Password> so12345 <New Password> -hsmLabel wild_fips_cw
save ns config
reboot
show ssl fips
From GUI - Traffic Management/SSL/FIPS
Keys - ADD
create fipskey wild_fips_cw -keytype RSA -modulus 2048 -exponent F4
show ssl fipsKey wild_fips_cw
Should display
FIPS Key Name: wild_fips_cw Key Type: RSA Modulus: 2048 Public Exponent: F4 (Hex: 0x10001)
CLI -
export fipskey wild_fips_cw -key wild_fips_cw.key
*Also do from GUI to ensure a good backup
Traffic Management/SSL/FIPS, select key export - *Ensure you use a different file path so you don't overwrite the keys
*Copy local to my device using WinSCP and then use WinSCP to copy to secondary device*
Go to secondary Node
show ssl fips
reset ssl fips
reboot
set ssl fips -initHSM Level-2 <New Password> so12345 <New Password> -hsmLabel wild_fips_cw
save ns config
reboot
CLI
import fipskey wild_fips_cw -key wild_fips_cw.key -inform SIM -exponent F4
(Says can’t find the key and then I ended up trying to import via the GUI) - That’s when I get the “internal error”
show ssl fipskey wild_fips_cw
FIPS Key Name: wild_fips_cw Modulus: 2048 Public Exponent: F4 (Hex value 0x10001)
Any additional thoughts are appreciated.
Thanks
FIPS_Internal_Import_error.png
I tried to copy using WinSCP post reboot and pre reboot. So this would go here:
Go to secondary Node
show ssl fips
reset ssl fips
Tried here (Copy FIPS via WinSCP to secondary node)
reboot
Copy FIPS Key via WinSCP to secondary node (Tried here)
set ssl fips -initHSM Level-2 <New Password> so12345 <New Password> -hsmLabel wild_fips_cw
save ns config
reboot
Go to secondary Node
show ssl fips
reset ssl fips
Tried here (Copy FIPS via WinSCP to secondary node)
reboot
Copy FIPS Key via WinSCP to secondary node (Tried here)
set ssl fips -initHSM Level-2 <New Password> so12345 <New Password> -hsmLabel wild_fips_cw
save ns config
reboot
Okay, my apologies for not being more specific when configuring both nodes. I think the issue might be a time constraint that is part of FIPS requirement. Let me rehash the steps here for you although you might need to begin from step 1, which is okay... must be done correct.
This is going to take me some time but I will add notes as well and post all steps momentarily. Plus, these steps will be useful for future questions regarding FIPS HA configuration.
Next post will be more thorough and streamlined as I looked back over my post and they can be optimized. Apologies for confusion. Will post momentarily.
I'm using CLI so I will add steps to compensate for not using GUI. GUI automatically syncs the SSL files whereas CLI does not.
This is going to take me some time but I will add notes as well and post all steps momentarily. Plus, these steps will be useful for future questions regarding FIPS HA configuration.
Next post will be more thorough and streamlined as I looked back over my post and they can be optimized. Apologies for confusion. Will post momentarily.
I'm using CLI so I will add steps to compensate for not using GUI. GUI automatically syncs the SSL files whereas CLI does not.
Please review this extensive list of steps before performing any steps and let's discuss any questions or concerns.
Must complete all steps in six minutes, or less. Both appliances must have the basic configuration.
Assumption: Steps assume using Putty with PSCP installed or binary files copied to C:\Putty.
Download: https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html
[Complete on Primary and Secondary Nodes]
Step A: Complete initial configuration on Primary and Secondary:
Notes: Initial Configuration Reference: http://bit.ly/2H3qoHX
[Primary Node - assumes CLI]
--------------------------
Step 1: Reset SSL FIPS on Primary
CLI: reset ssl fips
Note: This command resets the FIPS card to the default password.
Step 2: Save Netscaler Configuration
CLI: save ns config
Step 3: Reboot Primary Netscaler
CLI: reboot
[Secondary Node]
------------------------
Step 4: Reset SSL FIPS on Secondary
CLI: reset ssl fips
Note: This command resets the FIPS card to the default password.
Step 5: Save Netscaler Configuration on Secondary
CLI: save ns config
Step 6: Reboot Secondary Netscaler
CLI: reboot
[Primary Node]
--------------------
Step 7: Set FIPS SO password versus default on Primary Node.
CLI: set ssl fips -initHSM Level-2 \<newSOpass\> \<oldSOpass\> \<userPass\> \[-hsmLabel \<string\>\]
Example; set ssl fips -initHSM Level-2 newpass oldpass -hsmLabel my_hsm_label
Step 8: CLI: save ns config
Step 9: CLI: reboot
[Secondary Node]
------------------------
Step 10: Set FIPS SO password versus default on Secondary Node.
CLI: set ssl fips -initHSM Level-2 \<newSOpass\> \<oldSOpass\> \<userPass\> \[-hsmLabel \<string\>\]
Example; set ssl fips -initHSM Level-2 newpass oldpass -hsmLabel my_hsm_label
Step 11: CLI: save ns config
Step 12: CLI: reboot
[Primary Node] - Configure SIM
--------------------
Reference: https://support.citrix.com/article/CTX118684
Step 13: Make sure the following files are NOT on the Primary or the Secondary. Remove if they exist:
/nsconfig/ssl/source.cert
/nsconfig/ssl/target.key
/nsconfig/ssl/target.secre
/nsconfig/ssl/source.secre
Step 14: Initialize FIPS system from Primary Node
Primary> init fipsSIMsource /nsconfig/ssl/source.cert
Step 15: Use SCP to copy “source.cert” output file from Primary to Secondary Node
{From Netscaler Primary GUI using Putty SSH Session}
Primary> scp /nsconfig/ssl/source.cert nsroot@<Secondary_IP>:/nsc
[Secondary Node] Initialize SIM
------------------------
Step 16: Initialize FIPS on Secondary Node
Secondary> init fipsSIMtarget /nsconfig/ssl/source.cert /nsconfig/ssl/target.key /nsconfig/ssl/target.secre
Step 17: This is to copy the Secondary “target.secret” to Primary
Secondary> scp /nsconfig/ssl/target.secre
[Primary Node] - Enable SIM
--------------------
Step 18: Enable SIM
Primary> enable fipsSIMsource /nsconfig/ssl/target.secre
Step 19: Copy Source Secret from Primary to Secondary
Primary> scp /nsconfig/ssl/source.secre
[Secondary Node]
Step 20: Enable SIM on Secondary
Secondary> enable fipsSIMtarget /nsconfig/ssl/target.secre
[Primary Node] - Create the SSL FIPS Key on Primary Node
--------------------
Step 21: Create FIPS Key Primary Node
Primary> create fipskey Key-FIPS-1 -keytype RSA -modulus 2048 -exponent F4
Primary> show ssl fipsKey Key-FIPS-1
Step 22: Export the FIPS Key
Primary> export fipskey Key-FIPS-1 -key Key-FIPS-1.key
[From Windows Workstation Running PSCP or WINSCP]
Step 23: Use SCP or WINSCP to copy Key-FIPS-1.key to Secondary
Using WINSCP, copy “Key-FIPS-1.key” from physical hard drive to Secondary /nsconfig/ssl directory.
[Secondary]
----------------
Step 24: Import SSL Key on Secondary
Secondary> import fipskey Key-FIPS-1 -key Key-FIPS-2.key -inform SIM -exponent F4
Secondary> show ssl fipsKey Key-FIPS-1
Must complete all steps in six minutes, or less. Both appliances must have the basic configuration.
Assumption: Steps assume using Putty with PSCP installed or binary files copied to C:\Putty.
Download: https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html
[Complete on Primary and Secondary Nodes]
Step A: Complete initial configuration on Primary and Secondary:
Notes: Initial Configuration Reference: http://bit.ly/2H3qoHX
[Primary Node - assumes CLI]
--------------------------
Step 1: Reset SSL FIPS on Primary
CLI: reset ssl fips
Note: This command resets the FIPS card to the default password.
Step 2: Save Netscaler Configuration
CLI: save ns config
Step 3: Reboot Primary Netscaler
CLI: reboot
[Secondary Node]
------------------------
Step 4: Reset SSL FIPS on Secondary
CLI: reset ssl fips
Note: This command resets the FIPS card to the default password.
Step 5: Save Netscaler Configuration on Secondary
CLI: save ns config
Step 6: Reboot Secondary Netscaler
CLI: reboot
[Primary Node]
--------------------
Step 7: Set FIPS SO password versus default on Primary Node.
CLI: set ssl fips -initHSM Level-2 \<newSOpass\> \<oldSOpass\> \<userPass\> \[-hsmLabel \<string\>\]
Example; set ssl fips -initHSM Level-2 newpass oldpass -hsmLabel my_hsm_label
Step 8: CLI: save ns config
Step 9: CLI: reboot
[Secondary Node]
------------------------
Step 10: Set FIPS SO password versus default on Secondary Node.
CLI: set ssl fips -initHSM Level-2 \<newSOpass\> \<oldSOpass\> \<userPass\> \[-hsmLabel \<string\>\]
Example; set ssl fips -initHSM Level-2 newpass oldpass -hsmLabel my_hsm_label
Step 11: CLI: save ns config
Step 12: CLI: reboot
[Primary Node] - Configure SIM
--------------------
Reference: https://support.citrix.com/article/CTX118684
Step 13: Make sure the following files are NOT on the Primary or the Secondary. Remove if they exist:
/nsconfig/ssl/source.cert
/nsconfig/ssl/target.key
/nsconfig/ssl/target.secre
/nsconfig/ssl/source.secre
Step 14: Initialize FIPS system from Primary Node
Primary> init fipsSIMsource /nsconfig/ssl/source.cert
Step 15: Use SCP to copy “source.cert” output file from Primary to Secondary Node
{From Netscaler Primary GUI using Putty SSH Session}
Primary> scp /nsconfig/ssl/source.cert nsroot@<Secondary_IP>:/nsc
[Secondary Node] Initialize SIM
------------------------
Step 16: Initialize FIPS on Secondary Node
Secondary> init fipsSIMtarget /nsconfig/ssl/source.cert /nsconfig/ssl/target.key /nsconfig/ssl/target.secre
Step 17: This is to copy the Secondary “target.secret” to Primary
Secondary> scp /nsconfig/ssl/target.secre
[Primary Node] - Enable SIM
--------------------
Step 18: Enable SIM
Primary> enable fipsSIMsource /nsconfig/ssl/target.secre
Step 19: Copy Source Secret from Primary to Secondary
Primary> scp /nsconfig/ssl/source.secre
[Secondary Node]
Step 20: Enable SIM on Secondary
Secondary> enable fipsSIMtarget /nsconfig/ssl/target.secre
[Primary Node] - Create the SSL FIPS Key on Primary Node
--------------------
Step 21: Create FIPS Key Primary Node
Primary> create fipskey Key-FIPS-1 -keytype RSA -modulus 2048 -exponent F4
Primary> show ssl fipsKey Key-FIPS-1
Step 22: Export the FIPS Key
Primary> export fipskey Key-FIPS-1 -key Key-FIPS-1.key
[From Windows Workstation Running PSCP or WINSCP]
Step 23: Use SCP or WINSCP to copy Key-FIPS-1.key to Secondary
Using WINSCP, copy “Key-FIPS-1.key” from physical hard drive to Secondary /nsconfig/ssl directory.
[Secondary]
----------------
Step 24: Import SSL Key on Secondary
Secondary> import fipskey Key-FIPS-1 -key Key-FIPS-2.key -inform SIM -exponent F4
Secondary> show ssl fipsKey Key-FIPS-1
Awesome thank you. Will give this a try today and cross my fingers :) I appreciate your extra effort to document this further for me.
Is there to be a certificate created prior to this step?
init fipsSIMsource /nsconfig/ssl/source.cert
I'm not sure where to find the source.cert or if this generates it for me.
I've attached what is currently on the Primary Netscaler minus the wild_fips_cw that I generated or is that what you're referring to? I only ask as there is no extension my files.
Thanks
NetscalerSourceCertquestion.png
init fipsSIMsource /nsconfig/ssl/source.cert
I'm not sure where to find the source.cert or if this generates it for me.
I've attached what is currently on the Primary Netscaler minus the wild_fips_cw that I generated or is that what you're referring to? I only ask as there is no extension my files.
Thanks
NetscalerSourceCertquestion.png
Nadda. Make sure you don't use that certificate for now. We are initializing the SIM to setup HA and the FIPS key will be generated further down. This would make more sense with GUI but I prefer CLI. Everything will be stored, when done, in the /nsconfig/ssl/ which you can see using WINSCP or using the appropriate OpenBSD commands logged in as nsroot.
You probably have not run the previous commands but just to be sure see Step 13. Use WINSCP to check /nsconfig/ssl/ directory for any existing certificates and remove them prior to continuing to Step 14.
Once you complete all steps HA will be configured and your good to go. I can post the GUI steps if you wish but CLI is better learning experience and keep in mind I don't have a FIPS Netscaler in front of me right now so I'm going off my notes in OneNote. Plus, as we are going through this I can update my initial post with the steps to add notes as needed to help alleviate future questions.
You probably have not run the previous commands but just to be sure see Step 13. Use WINSCP to check /nsconfig/ssl/ directory for any existing certificates and remove them prior to continuing to Step 14.
Once you complete all steps HA will be configured and your good to go. I can post the GUI steps if you wish but CLI is better learning experience and keep in mind I don't have a FIPS Netscaler in front of me right now so I'm going off my notes in OneNote. Plus, as we are going through this I can update my initial post with the steps to add notes as needed to help alleviate future questions.
Reading the article, looks like the reference to source.cert is really just using this syntax. This is creating the cert specific to the Netscaler. So I should just leave as is:
init fipsSIMsource /nsconfig/ssl/source.cert
I guess I'm still not understanding completely though where the 'target' key and secret get generated from or where I define them. Or is this really using an existing certificate?
Can you please clarify this area.
Thanks
Jennifer
init fipsSIMsource /nsconfig/ssl/source.cert
I guess I'm still not understanding completely though where the 'target' key and secret get generated from or where I define them. Or is this really using an existing certificate?
Can you please clarify this area.
Thanks
Jennifer
Yes, I did it this way to keep things simple and standardized. I recommend leaving those lines, where I provided full syntax, identical.
Where you do need to change some of the syntax would be where I have <>. Example, <Primary_IP>, where you would take out the greater than and less than symbols and replace that section with IP Address of your NSIP relative to Primary and Secondary nodes.
Where you do need to change some of the syntax would be where I have <>. Example, <Primary_IP>, where you would take out the greater than and less than symbols and replace that section with IP Address of your NSIP relative to Primary and Secondary nodes.
Which step does the timer actually start? Is it with the CLI: set ssl fips -initHSM Level-2 \<newSOpass\> \<oldSOpass\> \<userPass\> \[-hsmLabel \<string\>\] step or after that?
The reboot time alone is taking almost 5 minutes.
Thanks
Jennifer
The reboot time alone is taking almost 5 minutes.
Thanks
Jennifer
Ok, I have followed the steps twice, and get this when I try to import
ERROR: Input file(s) not present or not accessible in current partition
If I try via the GUI, I get an internal Error
If I look via WinSCP to verify the file exists, it is in nsconfig/ssl
So, guessing I'm timing out... but not sure as the command is like it doesn't see it. Is the import looking in a different directory than the nsconfig/ssl?
Thanks
Jennifer
ERROR: Input file(s) not present or not accessible in current partition
If I try via the GUI, I get an internal Error
If I look via WinSCP to verify the file exists, it is in nsconfig/ssl
So, guessing I'm timing out... but not sure as the command is like it doesn't see it. Is the import looking in a different directory than the nsconfig/ssl?
Thanks
Jennifer
This happens when SIM is initialized on primary and not secondary.
For now, disable HA on the Netscaler and repeat these steps: (We need to initialize SIM on both primary and secondary)
Disable HA sync and propagation on both the nodes.
Run the following command on the source appliance:
init fipsSIMsource /nsconfig/ssl/source.cert
Run the following command on the target appliance:
init fipsSIMtarget /nsconfig/ssl/source.cert /nsconfig/ssl/target.key /nsconfig/ssl/target.secre
Run the following command on the source appliance:
enable fipsSIMsource /nsconfig/ssl/target.secre
Run the following command on the target appliance:
enable fipsSIMtarget /nsconfig/ssl/target.key /nsconfig/ssl/source.secre
For now, disable HA on the Netscaler and repeat these steps: (We need to initialize SIM on both primary and secondary)
Disable HA sync and propagation on both the nodes.
Run the following command on the source appliance:
init fipsSIMsource /nsconfig/ssl/source.cert
Run the following command on the target appliance:
init fipsSIMtarget /nsconfig/ssl/source.cert /nsconfig/ssl/target.key /nsconfig/ssl/target.secre
Run the following command on the source appliance:
enable fipsSIMsource /nsconfig/ssl/target.secre
Run the following command on the target appliance:
enable fipsSIMtarget /nsconfig/ssl/target.key /nsconfig/ssl/source.secre
Ok, I'll go through and just follow these steps when I get to this section as if I try now, says the file already exists. Thanks for sticking with me, feel like we're close, but I'm definitely not understanding or missing something.
Nevermind my last comment, I deleted it. I am doing all and getting the right result except the SSL Key Step on the Secondary. I'll try one more time in the event I'm doing this too slow.
Thanks for your help - no worries on the meeting, I appreciate your help.
Thanks for your help - no worries on the meeting, I appreciate your help.
Just a bit of info - Did a true timer on the reboot itself, that alone is taking 6 minutes 30 seconds for a response. I have been using Putty, but maybe I'll work over a console connection to see if this reduces the time.
Hm. Well, the 6 minute timer has to do with the SIM key transfer. Are you still getting the error posted previously?
We can translate these steps to the GUI. You would simply type in the same certificate names I provided earlier but in the GUI. This replicates the files automatically.
It will take me some time to put together those new steps, however.
After the last reboot, we are only doing the initialize SIM steps?
Otherwise, it should force you at 6 minutes... has this happened? Those last few steps? Or did you still get that "not found" error?
We can translate these steps to the GUI. You would simply type in the same certificate names I provided earlier but in the GUI. This replicates the files automatically.
It will take me some time to put together those new steps, however.
After the last reboot, we are only doing the initialize SIM steps?
Otherwise, it should force you at 6 minutes... has this happened? Those last few steps? Or did you still get that "not found" error?
Okay. Can you just double check on the Primary first (using WINSCP for example) and look in the /nsconfig/ssl directory and check for any "sub-directories". It might read like "sl" or "ls"... don't remember.
Just make sure there are no subs and if there are, check for any files mentioned above...particularly the .secret.
And, if not on Primary, check the secondary and see if somehow the process moved it to a sub-directory.
We might need to just move on to GUI being it is more automated and less steps.
Just make sure there are no subs and if there are, check for any files mentioned above...particularly the .secret.
And, if not on Primary, check the secondary and see if somehow the process moved it to a sub-directory.
We might need to just move on to GUI being it is more automated and less steps.
Thanks -
From WinSCP - I have been clearing out the source.key, target.key, etc.... after this fails.
After last reboot, Steps performed are.
Disable HA sync and propagation on both the nodes.
Run the following command on the source appliance:
init fipsSIMsource /nsconfig/ssl/source.cert
Run the following command on the target appliance:
init fipsSIMtarget /nsconfig/ssl/source.cert /nsconfig/ssl/target.key /nsconfig/ssl/target.secre
Run the following command on the source appliance:
enable fipsSIMsource /nsconfig/ssl/target.secre
Run the following command on the target appliance:
enable fipsSIMtarget /nsconfig/ssl/target.key /nsconfig/ssl/source.secre
This is successful
Then I was moving onto:
Step 21: Create FIPS Key Primary Node
Primary> create ssl fipskey vue01CWHSM -modulus 2048 -exponent F4
Primary> show ssl fipsKey vue01CWHSM
Step 22: Export the FIPS Key
Primary> export fipskey vue01CWHSM -key vue01CWHSM.key
*Copy the vue01CWHSM.key via WinSCP to Target (Secondary)
Step 24: Import SSL Key on Secondary
Secondary> import ssl fipskey vue01CWHSM -key vue01CWHSM2.key -inform SIM -exponent F4
Secondary> show ssl fipsKey vue01CWHSM
Step #24 is what is failing. So perhaps, only Steps #21 - 24 need to happen via the GUI.
From WinSCP - I have been clearing out the source.key, target.key, etc.... after this fails.
After last reboot, Steps performed are.
Disable HA sync and propagation on both the nodes.
Run the following command on the source appliance:
init fipsSIMsource /nsconfig/ssl/source.cert
Run the following command on the target appliance:
init fipsSIMtarget /nsconfig/ssl/source.cert /nsconfig/ssl/target.key /nsconfig/ssl/target.secre
Run the following command on the source appliance:
enable fipsSIMsource /nsconfig/ssl/target.secre
Run the following command on the target appliance:
enable fipsSIMtarget /nsconfig/ssl/target.key /nsconfig/ssl/source.secre
This is successful
Then I was moving onto:
Step 21: Create FIPS Key Primary Node
Primary> create ssl fipskey vue01CWHSM -modulus 2048 -exponent F4
Primary> show ssl fipsKey vue01CWHSM
Step 22: Export the FIPS Key
Primary> export fipskey vue01CWHSM -key vue01CWHSM.key
*Copy the vue01CWHSM.key via WinSCP to Target (Secondary)
Step 24: Import SSL Key on Secondary
Secondary> import ssl fipskey vue01CWHSM -key vue01CWHSM2.key -inform SIM -exponent F4
Secondary> show ssl fipsKey vue01CWHSM
Step #24 is what is failing. So perhaps, only Steps #21 - 24 need to happen via the GUI.
Yes. Perhaps there is something missing in the CLI above where they key is not being transferred from HSM.
So, using GUI just for those steps: (Plus it should automatically sync the key)
Let's do:
Navigate to Traffic Management > SSL > FIPS.
In the details pane, on the FIPS Keys tab, click Add.
In the Create FIPS Key dialog box, specify values for the following parameters:
FIPS Key Name*—fipsKeyName
Modulus*—modulus
Exponent*—exponent
*A required parameter
Click Create, and then click Close.
On the FIPS Keys tab, verify that the settings displayed for the FIPS key that you just created are correct.
For Starters..... This will help rule out some things. Use the same values as with CLI, however. For modulus and so forth.
So, using GUI just for those steps: (Plus it should automatically sync the key)
Let's do:
Navigate to Traffic Management > SSL > FIPS.
In the details pane, on the FIPS Keys tab, click Add.
In the Create FIPS Key dialog box, specify values for the following parameters:
FIPS Key Name*—fipsKeyName
Modulus*—modulus
Exponent*—exponent
*A required parameter
Click Create, and then click Close.
On the FIPS Keys tab, verify that the settings displayed for the FIPS key that you just created are correct.
For Starters..... This will help rule out some things. Use the same values as with CLI, however. For modulus and so forth.
Sounds good, appreciate it... We'll give this a shot one more time today then taking a break from it for the night ;) Hopefully that will be the magic trick. I'll reply either way.
Okay. We will get there it just takes a little longer being I cannot be sitting right there in front of the appliances with you. And, we are close to having this figured out.
Oh my Gosh! So Excited!!!!! Just doing that last step via the GUI worked. I was able to import successfully and then added my second node under the HA so they are actually both Showing online.
So, will table for today as we finally had success - a HUGE Thankyou!! Wish I could give you a gift certificate or something :)
So, next question is back to the WildCard certificate and how that works with the FIPS - to apply, renew, etc. But am willing to wait until tomorrow and open a new thread on that so this one can help someone else.
FIPS.png
So, will table for today as we finally had success - a HUGE Thankyou!! Wish I could give you a gift certificate or something :)
So, next question is back to the WildCard certificate and how that works with the FIPS - to apply, renew, etc. But am willing to wait until tomorrow and open a new thread on that so this one can help someone else.
FIPS.png
I have to say Brian was extremely helpful. I was doing portions of this correct, but was missing some and he persisted with me until we got this working. I am extremely grateful, been hitting my head against this wall for awhile, so nice to advance so I can continue with the other portions of the configuration.
Awesome! Again, my pleasure. Here to help.
One the wildcard certificate, IMO, it would be better to post another question so that we can keep that resolution separate from this thread.
I figured we were close on the SIM/HSM/HA configuration.
Good to know we are one big step forward toward your final config.
One the wildcard certificate, IMO, it would be better to post another question so that we can keep that resolution separate from this thread.
I figured we were close on the SIM/HSM/HA configuration.
Good to know we are one big step forward toward your final config.
Premium Content
You need an Expert Office subscription to comment.Start Free Trial