How do I reduce the latency to my server?

I have a Dell PowerEdge server running Linux RedHat. Within the past week, the latency to the server has varied substantially. Essentially, the server may be accessible at times and not accessible at other times. The ping replies from the server vary---sometimes I'll get two replies and at other times I'll get four replies for a consistent amount of time. Do you have any idea as to what may be the problem? I would like to get consistent ping replies with low latency and have the server accessible at all times. Please see the reply statistics below. Also, please see the output for ethtool for the NIC card below. Lastly, please see the switch port data below.

Reply from 208.74.72.163: bytes=32 time=1906ms TTL=52
Reply from 208.74.72.163: bytes=32 time=998ms TTL=52
Reply from 208.74.72.163: bytes=32 time=1007ms TTL=52
Reply from 208.74.72.163: bytes=32 time=1984ms TTL=52
Reply from 208.74.72.163: bytes=32 time=1004ms TTL=52
Reply from 208.74.72.163: bytes=32 time=13ms TTL=52
Reply from 208.74.72.163: bytes=32 time=12ms TTL=52
Reply from 208.74.72.163: bytes=32 time=11ms TTL=52
Reply from 208.74.72.163: bytes=32 time=11ms TTL=52
Reply from 208.74.72.163: bytes=32 time=986ms TTL=52
Reply from 208.74.72.163: bytes=32 time=14ms TTL=52
Reply from 208.74.72.163: bytes=32 time=11ms TTL=52
Reply from 208.74.72.163: bytes=32 time=980ms TTL=52
Reply from 208.74.72.163: bytes=32 time=981ms TTL=52
Reply from 208.74.72.163: bytes=32 time=982ms TTL=52
Reply from 208.74.72.163: bytes=32 time=11ms TTL=52
Reply from 208.74.72.163: bytes=32 time=12ms TTL=52

[root@pptp1 ~]# ethtool eth0 | more
Settings for eth0:
        Supported ports: [ MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Advertised auto-negotiation: Yes
        Speed: 100Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: g
        Wake-on: d
        Current message level: 0x000000ff (255)
        Link detected: yes


FastEthernet0/18 is up, line protocol is up (connected)
  Hardware is Fast Ethernet, address is 000e.8453.d252 (bia 000e.8453.d252)
  MTU 1500 bytes, BW 100000 Kbit, DLY 1000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full-duplex, 100Mb/s
  input flow-control is off, output flow-control is off
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input never, output 00:00:01, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 1000 bits/sec, 2 packets/sec
     814503777 packets input, 1117917448 bytes, 0 no buffer
     Received 84987 broadcasts (0 multicast)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 2806 multicast, 0 pause input
     0 input packets with dribble condition detected
     800766599 packets output, 557768400 bytes, 0 underruns
     0 output errors, 0 collisions, 2 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 PAUSE output
     0 output buffer failures, 0 output buffers swapped out