Windows 10 20H2 Network drives shows as not connected but it works fine

check event viewer for more info
no errors in device manager?
did you reboot the PC?
did this happen after an update , or software install?

No Errors in event viewer and Device manager. This happened in new 20H2 install and OS upgrade scenario too. The issue we are facing is similar to the issue. The solutions mentioned in the link also did not work
Red X on mapped network drives. (microsoft.com) 

This is one of those issues that keeps giving and I have seen it several times since around windows 7.
Over the years there have been several things that cause this problem. (a quick google search off "red x on network drive" will show you how common it is) 
In general the following steps are what typically fixes it:
On the server side run the following to disable the autodisconnect (run it from admin command prompt)

net config server /autodisconnect:-1

Select allOpen in new window

Then on the client side
open regedit
go to
HKLM\SYSTEM\CurrentControlSet\Service\lanmanworkstation\parameters
modify the KeepConn  key to a higher value (this is the keep alive timeout)

Your situation may be different but i would start with the above and see if it helps. (you can always change it back)

I have also seen this a lot over the years... After a reboot, if the network is just a bit slow to finish loading, the OS sees that the drives didn't connect and marks them red... Windows just checks too early, and doesn't have any code to tell it to wait until the network stack is fully connected...

Could just be the icons file is corrupted.  My favorite is when the recycle bin always shows something in it when empty.
Here is the top result on Google to help on resetting icon cache: https://answers.microsoft.com/en-us/windows/forum/all/windows-10-desktop-icons-refreshing/acf43995-2f0d-4f86-8904-c39e3043c2aa

do you see them in disk management ?
look if this helps :  networking - Windows 10 Doesn't Find Network Drives or Computers - Super User 

Nobus: disk management does not show mapped network drives. 

None of the steps mentioned above is fixing the issue. The issue is that I cannot disconnect and reconnect the drives.If we try to disconnect, it has below error.
 
 

I would run the usual SFC /scannow and then DISM /online /cleanup-image /restorehealth
from Power shell.  Might be file or drive corruption.
Another thought would be Windows repair FREE from tweaking.com

You can remove them from the registry then reboot.
The location will vary depending on how it was mapped.
HKCU\Network
or
HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\MountPoints2\
or
HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Map Network Drive MRU

You can also try to remove the map using the NET USE command (this will try to remove any mapped network drives) 
NET USE * /DEL  /Y