ASA 5505 Split Tunneling Issue

I am having a very interesting issue with our ASA 5505.  After hours of troubleshooting here is what I finally figured out is going on.
Basically I setup a client VPN and enabled split tunneling for access to the local network.
When I connect to the VPN and I look in the local routing table of a machine that is connecting in, it will list the gateway of the asa with a higher metric than the split tunneled gateway.
The weird thing is that it will sometimes work..   When it is working properly, the other gateway is listed first like it should be.
Hopefully all of that made sense.
Here is my config..

pnetfw1# sh run
: Saved
:
ASA Version 7.2(2)
!
hostname pnetfw1
domain-name default.domain.invalid
enable password vS2IOBQWprXYBVhr encrypted
names
!
interface Vlan1
 nameif inside
 security-level 100
 ip address 10.20.20.1 255.255.255.0
!
interface Vlan2
 nameif outside
 security-level 0
 ip address x.x.x.x 255.255.255.224
!
interface Ethernet0/0
 switchport access vlan 2
!
interface Ethernet0/1
!
interface Ethernet0/2
!
interface Ethernet0/3
!
interface Ethernet0/4
!
interface Ethernet0/5
!
interface Ethernet0/6
!
interface Ethernet0/7
!
passwd 2KFQnbNIdI.2KYOU encrypted

ftp mode passive
clock timezone CDT -6
clock summer-time CDT recurring
dns server-group DefaultDNS
 domain-name default.domain.invalid
same-security-traffic permit intra-interface
access-list NONAT extended permit ip any 10.20.20.0 255.255.255.0
access-list TEST_VPN1 extended permit ip 10.20.20.0 255.255.255.0 any
access-list SPLIT_LIST standard permit any
access-list ICMP extended permit icmp any any
pager lines 24
logging enable
logging asdm informational
mtu inside 1500
mtu outside 1500
ip local pool VPNPOOL1 10.20.20.230-10.20.20.235 mask 255.255.255.0
ip local pool VPNPOOL2 10.20.20.201-10.20.20.240 mask 255.255.255.0
icmp unreachable rate-limit 1 burst-size 1
icmp permit any outside
asdm image disk0:/asdm-522.bin
no asdm history enable
arp timeout 14400
global (outside) 1 interface
nat (inside) 0 access-list NONAT
nat (inside) 1 0.0.0.0 0.0.0.0
nat (outside) 1 access-list TEST_VPN1
route outside 0.0.0.0 0.0.0.0 x.x.x.x 1
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout uauth 0:05:00 absolute
group-policy PNETVPN1 internal
group-policy PNETVPN1 attributes
 dns-server value 204.117.214.10
 vpn-tunnel-protocol IPSec
 password-storage enable
 split-tunnel-policy tunnelspecified
 split-tunnel-network-list value SPLIT_LIST

aaa authentication ssh console LOCAL
http server enable
http 10.20.20.0 255.255.255.0 inside
no snmp-server location
no snmp-server contact
snmp-server enable traps snmp authentication linkup linkdown coldstart
crypto ipsec transform-set TS1 esp-3des esp-md5-hmac
crypto dynamic-map DYN1 20 set pfs
crypto dynamic-map DYN1 20 set transform-set TS1
crypto map OUT_MAP 65535 ipsec-isakmp dynamic DYN1
crypto map OUT_MAP interface outside
crypto isakmp enable outside
crypto isakmp policy 10
 authentication pre-share
 encryption 3des
 hash md5
 group 2
 lifetime 86400
crypto isakmp nat-traversal  20
tunnel-group PNETVPN1 type ipsec-ra
tunnel-group PNETVPN1 general-attributes
 address-pool VPNPOOL1
 default-group-policy PNETVPN1
tunnel-group PNETVPN1 ipsec-attributes
 pre-shared-key *


!
class-map inspection_default
 match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
 parameters
  message-length maximum 512
policy-map global_policy
 class inspection_default
  inspect dns preset_dns_map
  inspect ftp
  inspect h323 h225
  inspect h323 ras
  inspect rsh
  inspect rtsp
  inspect esmtp
  inspect sqlnet
  inspect skinny
  inspect sunrpc
  inspect xdmcp
  inspect sip
  inspect netbios
  inspect tftp
!
service-policy global_policy global
prompt hostname context
Cryptochecksum:5dd42e391e21d5b1663d79f8232bf786
: end
 

Local Routing Table:
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0       10.20.20.1    10.20.20.235       1 - asa
          0.0.0.0          0.0.0.0      172.21.1.10     172.21.2.18       10 - split tunneled gateway (172.21.1.10)
       10.20.20.0    255.255.255.0     10.20.20.235    10.20.20.235       10
     10.20.20.235  255.255.255.255        127.0.0.1       127.0.0.1       10
   10.255.255.255  255.255.255.255     10.20.20.235    10.20.20.235       10
        127.0.0.0        255.0.0.0        127.0.0.1       127.0.0.1       1
       172.21.0.0      255.255.0.0      172.21.2.18     172.21.2.18       10
       172.21.0.0      255.255.0.0       10.20.20.1    10.20.20.235       10
      172.21.2.18  255.255.255.255        127.0.0.1       127.0.0.1       10
      172.21.10.1  255.255.255.255      172.21.2.18     172.21.2.18       1
   172.21.255.255  255.255.255.255      172.21.2.18     172.21.2.18       10
    x.x.x.x        255.255.255.255      172.21.1.10     172.21.2.18       1
        224.0.0.0        240.0.0.0     10.20.20.235    10.20.20.235       10
        224.0.0.0        240.0.0.0      172.21.2.18     172.21.2.18       10
  255.255.255.255  255.255.255.255     10.20.20.235    10.20.20.235       1
  255.255.255.255  255.255.255.255      172.21.2.18     172.21.2.18       1
Default Gateway:        10.20.20.1 - asa (when working it will be 172.21.1.10)
===========================================================================
Persistent Routes:
  None