Trix VOIP Network Betup

I have recently attempted to set up a trixbox VOIP server and want to know the best way to setout the network. I have an SDSL line from Zen which gives us 5 external ip addresses. We have a Billion BiPAC 8500 router This is what I want (i think):

Ext IP 1 - straight to our video conference unit (completely separate to the office LAN)

Ext IP 2 - office network (normal)

Ext IP 3 - our voip server. The only confusion on this one is that I want the server to be on the same LAN as the office but it's incoming and outgoing route to go via ext ip 3. Ext Ip3 need to point straight to the voip server for this to work.) Im hoping if I can do some kind of routing that I dont need to use NAT port forwarding for this bit.

Can the above be done? Is there a better layout for the network?

Many Thanks for any help!

Steve