VLAN "flickering" - ping request timed out infrequently

I am setting up a VLAN on our network, mostly made up of cisco 3750 switches. It will only be used on one switch. This VLAN will be tunneled through an ASA box to another network in another company. It should not talk to any other VLANs except to be able to use the tunnel (though for starters I will keep it open to test)

I've set up:
vlan access-map map_ki 10
 action forward
 match ip address ki_acl
interface Vlan54
 description ....
 ip address 192.168.54.1 255.255.255.0
ip access-list extended ki_acl
 permit ip any any

in the asa box I added to the outside accesslist permiting this VLAN out to any on port 80

My problem:
it seems to "time out". if I ping or try to access a webpage it sometimes completes the action and sometimes not.
if I ping -t an inside ip and google side by side they might give 10 successes then 20 request timed out then again success. And the two pings are not consistent either.

I've tried changing ports on the switch as well as the cables so I've pretty much ruled those out.

how do I troubleshoot this?