Cannot add machines to Vcentre appliance

OK I have 2 sites A & B
Site A has 6 ESX 5.1 hosts
Site A has 4 ESX 5.1 hosts
Currently all are managed by a Windows VCentre 5.1 on Site A
No issues, except the server itself is heavily loaded and vcentre is sluggish

So installed VCentre 5,5 appliance at site A
Works fine, SSO & AD integration. Grand no problem.

I then tested adding a host from site A to appliance.
Seemed to be OK, except that it's part of an HA cluster and that would fail etc.
So didnt add, but seems that it will.

But when it comes to adding hosts from site B to the appliance I get an error

"Call "Datacenter.QueryConnectionInfo" for object "Cluster" on vCenter Server "appliance" failed."

So Googling says check DNS, check adding by FQDN, check network management agents, check password

None of it makes any difference.

Using VSphere client at site B I can connect to hosts on Site A & B
Using VSphere client at site A I can connect to hosts on Site A & B

Virtual appliance usable from Site A & B
Virtual appliance can ping hosts on sites A & B

It's not password, it's not lockdown.
Reboot management network, or host makes no difference.
Connectivity doesn't appear to be the issue however when I do try
to add a host from Site B, it doesn't bother prompting be or the SHA thumbprint

It does for the servers on Site A.

One interesting thing  mentioned here
http://www.neilbryan.ca/failed-to-add-host-using-vcenter-5-1-appliance-datacenter-queryconnectioninfo/

Disabling IP6.

Not sure how to do that