Wi-Fi calling

Since the first hop is wifi instead of a cell tower, a malicious person could technically record the packets, but they would be encrypted. There isn't much information to be found about the type of encryption used so you may want to contact your specific carrier and see if it meets your security standards.

Interesting question.  You say you give your traders "corporate phone devices locked to that carrier".  I presume, then, that your traders are currently using wireless phones and not wired landlines.

Sidenote:  If your traders are using wired landlines, then unless you have made provision for secured connections, all conversations can be monitored by breaking into the cable pair for that phone... at a punch point, or even up on the telephone pole.
 
If you are using GSM wireless phones (aka, AT&T or T-Mobile) then the conversations are encrypted, and, the encryption was broken several years ago.  This article has information:  http://arstechnica.com/gadgets/2010/12/15-phone-3-minutes-all-thats-needed-to-eavesdrop-on-gsm-call/

Similarly, CDMA wireless phones also have encrypted conversations and the technology is somewhat harder to break, although the hackers still say it is pretty simple to do so.

In either case above it would be possible for someone with knowledge and a good scanner to be able to intercept trader calls on cellular.  On the whole, though, it would have to be a focused attack, not just sitting around in Starbucks with a laptop.
 
Cellular traffic goes by wireless to the nearest cell phone tower.  From the cell phone tower, fiber or copper wires carry the conversations to the nearest exchange and on towards wherever the call is being routed.  I don't know if the fiber/copper in encrypted between tower and exchange... in any event, it would be something of a challenge to be able to access the phone company's infrastructure.

I should clarify the above paragraph just a bit.  When you are outdoors, or perhaps in an office with a clear line of sight (through walls, etc), then you will be connected to a tower.  Otherwise, the chances are good that you are connected to a phone company supplied wireless access point, for example, in a basement, in an elevator, etc, where wireless tower signals are not strong.  Depending upon the configuration, these wireless access points are either hardwired into the building trunk and sent to the exchange, or they are connected to an external antenna which communicates with the cell tower.
 
When you use wifi phones, you are connected to the nearest access point in your office, and your level of wifi security is dependent upon the encryption method and length and randomness of the key you use.  I judge that with a decent key and WEP security your wifi encryption is as good as, if not better, any cellular wireless communications.
 
Next the phone conversation runs from the access point through your router and cable modem, and on down to your ISP.  This is not VOIP protocol but something similar for routing the conversation to your ISP's switches.  I believe, but have not confirmed that the conversation is transmitted via secure VPN but you might want to check that.
 
Your ISP will have from one to many POP's (point of presence) for each phone company.  Your conversation is passed from the ISP's switches to the telephone company's equipment at the POP.  Thereafter, just as with a cell tower, your conversation is carried via fiber or copper to an exchange for further routing.  Note that this is different that true VOIP calling where the conversation is carried as IP packets from start to end.
 
You can see that there are multiple potential areas for a phone conversation to be compromised (and if you are the NSA, you can have it all!).  But generally, for wireless conversations, it would be necessary to record, then decrypt, while attempting to intercept hardwired communications would require physical access to the wire/fiber pairs.  Considering that 1000's of phone conversations can be run down a single fiber optic cable, I judge that intercepting the one you want would present a serious challenge... unless you are the NSA.

WiFi calling is an application supported on the newer iPhone IOS and uses IKE phase one port 500 and IKE phase two (IPSEC) port 4500 when initialising a call, it uses secure encryption and authentication methods so packets that could possibly be captured could not be decrypted easily even if at all. I have recently implemented this in for my current business and the cost saving per site in comparison to the signal extension was considerable.

Sorry - I should have been clearer.

My question really is - is it possible for carrier to record telephone calls when using Wifi callling (yes, I know this is really a question for them, but I wanted to get some background knowledge).

So:

Using the normal cell network, our carrier records the telephone calls of traders who have a company mobile phone. The recordings are held on their network for us to listen to.

If we allowed Wi-fi calling on these devices, does the conversation still always go through their network? Or is it possible to go point to point and avoid their network, and thus we have a potential loophole?

If we allowed Wi-fi calling on these devices, does the conversation still always go through their network?

The call still goes through their network.  The wifi call is routed by your ISP from the ISP switches to a POP (point of presence) supplied by the phone company that is a physical presence in the ISP's datacenter.  As soon as this occurs, you are in the phone network.

And as you said, verify this point with the phone company... but it's the way many IP based company phone systems make contact with the rest of the world's non IP telephones... through a POP.

Wifi calling is not VOIP which is totally IP based.  Wifi calling simply provides an alternative route into the telephone company exchange.

It is designed to assist call quality where cell tower coverage is poor, and to potentially save money.  It also has the added benefit for the phone company of reducing cell tower traffic.

Got it!

So: Let me say, for example, that the carrier in question is ATT.

Wi-Fi calling:  Provides an alternative route to the ATT network? Regardless of who I am calling, even someone on the same Wi-FI network, there is no Point-to-Point calling...everything still has to touch ATT.

VOIP: These calls CAN be Point-to-Point and avoid ATT?

It is designed to assist call quality where cell tower coverage is poor, and to potentially save money.  It also has the added benefit for the phone company of reducing cell tower traffic.

How does it save money?