iptables limit connection per ip correct way ?

i have set some rules in ip tables as example i added

iptables -A INPUT -p tcp --syn --dport 22 -m connlimit --connlimit-above 4 --connlimit-mask 32 -j REJECT --reject-with tcp-reset

and thats how my iptables looks like

Chain INPUT (policy ACCEPT)
target     prot opt source               destination
REJECT     tcp  --  anywhere             anywhere             tcp dpt:20000 flags:FIN,SYN,RST,ACK/SYN #conn src/32 > 2 reject-with tcp-reset
REJECT     tcp  --  anywhere             anywhere             tcp dpt:ssh flags:FIN,SYN,RST,ACK/SYN #conn src/32 > 2 reject-with tcp-reset

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

but there is always ips by-pass the limit and they go up to 400 connection per ip

what is the correct way to limit the connection per ip ?

i am running Ubuntu 15.04 server