SSH to Cisco ISR 1111 failing - need ideas

I am trying to connect to the outside interface of a Cisco ISR 1100 with SSH. If I do a remote session with the user at the site - I can SSH to the ISR at the inside address. But if I try to ssh to the external interface of the ISR I don't even get a login prompt. There is no ACL on the outside address. There is an ACL on vty 0 4 but it includes the source address I'm coming from and I even removed that ACL to prove the point temporarily.

Any other suggestions with regards to getting ssh to work on the outside interface as well as the inside? Is there a good debug that could help?

I have ip http secure-server in the config.

line vty 0 4
 access-class 5 in
 exec-timeout 30 0
 privilege level 15
 authorization exec SECURE
 logging synchronous
 login authentication SECURE
 length 0
 transport preferred ssh
 transport input ssh
line vty 5 15
 access-class 5 in
 exec-timeout 30 0
 privilege level 15
 authorization exec SECURE
 logging synchronous
 login authentication SECURE
 length 0
 transport input ssh