Does the IPSEC terminate on your SSG5?

The likely issue is that the ipsec is being negotiated with the SSG5 and is failing.

L2TP is UDP 1701, but since you are using l2TP over IPSEC, there is no need to open up ports, since the IPSEC policy should provide the path to the internal Server as part of the configuration.
i.e. ipsec policy allows the remote user to connect to the Internal Windows 2003 server where RRAS is running.

The IPSec shouldn't, to my knowledge, be terminating on the SSG5. It should presumably be passing through to the internal RRAS server. Having said that, I could be wrong; not sure if it needs to terminate on the SSG5 or not.

Here is my current policy configuration on the SSG5 from untrust to trust to the internal RRAS server:


9 Untrust  NAMEOFPOLICY Any          MIP(XXX.XXX.XXX.XXX)
                                                   GRE                  Permit enabled ---X-X
                                                   L2TP        
                                                   L2TP-UDP1701
                                                   L2TP-UDP4500
                                                   L2TP-UDP500
                                                   PING        
                                                   PPTP-1723  
                                                   PPTP-47    

Please note that I have changed the policy name to NAMEOFPOLICY and the MIP to XXX.XXX.XXX.XXX for security purposes.

I manually created the L2TP-UPD policies. Please note that, as you can see, PPTP passthrough is configured and works fine.

What do I need to add to enable L2TP/IPSec traffic to passthrough properly to the RRAS server?

Sorry, I should have said "I manually created the L2TP-UDP Custom Services".

Check the RRAS server's log to see whether IPSEC errors are occuring.
Did you check whether your IPSEC from remote to the RRAS is establishing?
IPSEC has to be configured as a policy using the ipsec MMC on both the remote client and the RRAS.

I checked the RRAS server log, but an entry from the server log looks like this (machine name changed for security purposes):

192.168.20.2,USERNAME,06/16/2011,20:01:18,RAS,NAMEOFSERVER,4,192.168.20.2,6,2,7,1,5,130,61,5,64,1,65,1,31,166.205.14.159,66,166.205.14.159,25,311 1 192.168.20.2 05/25/2011 06:46:35 328,44,5499,8,192.168.10.158,12,1500,50,595,51,1,55,1308268878,45,3,40,1,4108,192.168.20.2,4147,311,4148,MSRASV5.20,412

I actually made another EE post about this requested assistance on dissecting these logs because I have no idea (nor can I find anything online) how to read these. They're very complex. If you can point me towards a reference on this that would be extremely appreciated.

Furthermore, on the SSG5, don't IP protocols 50 and 51 needs to be opened up for IPSec to passthrough? How do I add these custom services?

It's always a good idea to use the logging of session in the policy - that shows if every required port/service is hitting that policy.