ssh tap layer 2 tunnel - traffic in one direction - help to debug

Is routing setup correctly on both sides? missing traffic can be cause by not sufficiently setup routes.
(You are adding a new interface on both systems...)

i am adding bridge interface to each side that has tap and local interface attached.

its a layer 2 tunnel so no routing are involved.
as mentioned ssh -o tunnel=ethernet

Tip: Remove all the optional args + get your command working.

So best first step... if possible... get this working...

ssh root@host

Select allOpen in new window

Once this is working then, work on your Tunnel.

The syntax -w 10:10 at least for latest stable SSH... is invalid...

Version I'm running on Ubuntu Focal...

# ssh -V
OpenSSH_8.3p1 Ubuntu-1~ppa0~20.04.0, OpenSSL 1.1.1j  16 Feb 2021

Select allOpen in new window

Man page says valid -w arg forms are...

1) [-W host:port]

2) [-w local_tun[:remote_tun]]

So... -w 10:10... seems incorrect...

I avoid working with raw packet flow... so I might be mistaken...

Best double check this for your SSH version...

once running with -w 10:10 it creates tap10 interface on each side. when you run it manually it works. while running tcpdump i can see icmp.
i have also tried to attach ip on each br interface and i can ping each other so tunnel is up.

For IP routing is definitely involved... then at least ARP needs to work to allow intra broadcast domain activity,
(Without ARP nothing will work).   So you need tools like arping to see it there is traffic.

ICMP already is IP. 

What do you suggest to check on the remote server i do see arp -i will share tcpdump later

Arp shows your local table you need to check the communication.   (anything shown means you received ARP packets from other systems.
Are there records from the other side?  then remote can send to you.
Now you need to check the other side.
  arpping some-remote-ip
(Arpping is a kind of ping using arp packets in stead of ICMP).
If that works remote sites should also have your local systems in the ARP. and communication should be possible.

Usage:
  arping [options] <destination>

Options:
  -f            quit on first reply
  -q            be quiet
  -b            keep on broadcasting, do not unicast
  -D            duplicate address detection mode
  -U            unsolicited ARP mode, update your neighbours
  -A            ARP answer mode, update your neighbours
  -V            print version and exit
  -c <count>    how many packets to send
  -w <timeout>  how long to wait for a reply
  -i <interval> set interval between packets (default: 1 second)
  -I <device>   which ethernet device to use
  -s <source>   source ip address
  <destination> dns name or ip address

Select allOpen in new window

the remote server is running on esxi as vm. i think that i have an idea that it could block bridge packets (as the tap is bridged to physical interfaces)
can i allow promiscous mode just for one vm/nic or it has to be on portgroup/vswitch level?

I have no ESX env. sorry no answer there (from me).
From a linux/ system level promiscuous mode is an interface state the doesn;t filter pacets. (causing more traffic to be processed).
So you should be able to set one NIC in promiscous mode.. probably this means more data is processed .