Solved

internet file transfer

Posted on 2006-06-19
17
279 Views
Last Modified: 2008-01-09
hi,
I was just wondering how yahoo makes it possible for com1 and com2 to connect to each other? The reason I am asking this is because I have created small network application, which main purpose is to transfer a file from com1 to com2, and unless I open/forward the port on com2, com1 is unable to connect to it. Both PCs use same mini DLS routers.

Yahoo has no problem at all when I do the same task because it creates direct connection but, I have no idea how... does it use some sort of network address translation internally or there is something else to it..?
0
Comment
Question by:davidlars99
  • 8
  • 7
17 Comments
 
LVL 18

Expert Comment

by:Ravi Singh
Comment Utility
Hi, not 100% sure if yahoo messenger supports this, but a lot of the peer-to-peer software today uses the UPnP protocol. Typically routers support this as a feature and allow applications to programmatically configure ports...

More Info - http://en.wikipedia.org/wiki/Upnp
C# Implementation - http://www.pixvillage.com/blogs/devblog/archive/2005/03/31/183.aspx
0
 
LVL 13

Author Comment

by:davidlars99
Comment Utility
hi Zephyr__, I assumed so but, I don't think yahoo uses UPnP because it is disabled on my router. Can you still benefit from UPnP if it's disabled?
0
 
LVL 18

Expert Comment

by:Ravi Singh
Comment Utility
Hi again, doubt its usable if its disabled, could it be the case that them ports are already open (predefined port mappings)? Yahoo Messenger uses the following ports in case you want to check:

5000 (TCP - Chat)
5001 (TCP - Chat)
5055 (UDP - Phone)
5100 (TCP - Webcam*)

from: http://support.dlink.com/faq/view.asp?prod_id=1428
0
 
LVL 13

Author Comment

by:davidlars99
Comment Utility
yes I know it does but, I have a DSL mini-router and as far as I know there has to be port forwarding enabled for those port that yahoo uses...  

I'll check if my router has pre-configured forwarding for yahoo ports, and let you know.
0
 
LVL 13

Author Comment

by:davidlars99
Comment Utility
by the way, I have Westell 6600 router
0
 
LVL 18

Expert Comment

by:Ravi Singh
Comment Utility
I can't find any specific information relating to how yahoo messenger overcomes problems related to NAT (in terms of direct p2p connections), but I have in the past come across various ways to bypass NAT to create p2p friendly applications (some of them involve the use of an intermediate server to "introduce peers". Some of the methods used are explained on the webpages below, they should give you a better idea:

NAT and Peer-to-peer networking:
http://alumnus.caltech.edu/~dank/peer-nat.html

How to communicate peer-to-peer through NAT (Network Address Translation):
http://www.mindcontrol.org/~hplus/nat-punch.html

Building NAT-Friendly Applications:
http://www.intel.com/cd/ids/developer/asmo-na/eng/79524.htm

Hope this helps.
0
 
LVL 13

Author Comment

by:davidlars99
Comment Utility
Zephyr__, will this examples work if there's a intermediate server, which introduces peers to each other but both computers need to have port forwarding done?

I am confused...
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 18

Expert Comment

by:Ravi Singh
Comment Utility
Hi,

Using the server to introduce the peers eliminates the need for port forwarding on the gateway (router) devices. So if for example peer1 wanted to communicate with peer2 (and both are behind a NAT enabled gateway) then using the approach explained in the second link (http://www.mindcontrol.org/~hplus/nat-punch.html)... peer1 would send a packet to the server, in order for the packet to reach the server it would go through the local NAT enabled gateway, the gateway would inject a "return address" into the packet (based on some unused port) and send the packet onto the server. The server would get the packet, note the return address for the packet and send the return address to peer2, peer2 can then communicate to peer1 using the return address, when peer2 sends a packet to peer1 on the return address... the NAT device on peer1 would recognise the return address should map to peer1 on the local network and forwards the packet to peer1...

The above was a simplified explanation, if you read the the webpage mentioned, it explains it in more detailed (pay extra attention to the section "Introducing the Introducer")... yahoo may or may not support this approach, but its one known way of bypassing NAT's limitations for p2p communication

Hope this helps
0
 
LVL 13

Author Comment

by:davidlars99
Comment Utility
I read that section about 10 times...  :-)

what I don't understand are two things:
  1.  how can peer1 send a packet to peer2 without properly connecting to it, using Socket.Connect(EndPoint)?
  2.  how can peer2 accept a packet from a stranger such as peer1, when peer2 has never seen peer1's IP address?

or maybe I should cut the crap and just try it out?
0
 
LVL 18

Accepted Solution

by:
Ravi Singh earned 500 total points
Comment Utility
Hi david,

The server would typically store the current IP's of the "online" peers, so once a peer has authenticated on the server, the server would probably send the authenticated peer a list of the online peers with their proper "global" ip addresses, thus allowing peers to establish direct connections with other peers.

Peer2 can accept the packet because the NAT gateway maintains a list of return addresses which it assigns to packets on their way out of the local network (i.e. if peer2 sends a packet to the server - the packet is first intercepted by the router - at this point the packets return address is the local peer computers' ipaddress + the port its listening on - the router changes the return address to the ip address of the router + an available port number on the router - it stores this info in a local lookup table - so when the packet eventually arrives at the server its return address is the routers ip:port number - the server would then send this to peer1 (for example when peer1 authenticates on the peer network) - peer1 can then send a packet directly to that ipaddress+port number of the router (which peer2 is under), the router performs the lookup and notices that the incoming packet has the destination address of the one assigned by the router as a return address - the router then forwards this packet to peer2 on the port number which the originating software was listening on.

So the packet goes to the port which the software was originally listening on, I guess its then upto you to perform some checks to see whether it is actually a packet that your listening for (using some secret headers etc in the data).
0
 
LVL 13

Author Comment

by:davidlars99
Comment Utility
ok finally got it to work. tested on DLS connection, where two computers have same exact speed (768 kb/sec) so, 36.1 MB file took exactly 29 minutes to transfer from peer1 to peer2, which is exactly 10 minutes faster than yahoo file transfer on both same computers...  ;)

[SERVER SOCKET]

using System;
using System.IO;
using System.Net;
using System.Text;
using System.Net.Sockets;
using System.Threading;

namespace ServerSocket
{
      class EntryPoint
      {
            static Socket cs;
            static Socket ss;            
            const int MAX_CONN = 1;
            const int MAX_BUFFER_SIZE = 66560; // tested on DSL connection
            const int MIN_BUFFER_SIZE = 1024;

            static FileStream buffer;

            static int totalBytes = 0;
            static int bytesRcvd = 0;
            static byte[] temp_buffer;

            static AsyncCallback receiveCallback;
            static AsyncCallback acceptCallback;

            [STAThread]
            static void Main(string[] args)
            {
                  receiveCallback = new AsyncCallback(ReceiveCallback);
                  acceptCallback = new AsyncCallback(AcceptCallback);

                  try
                  {
                        IPEndPoint ipe = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 3535);
                        ss = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                        ss.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, 1);
                        ss.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReceiveBuffer, MAX_BUFFER_SIZE);
                        ss.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.Debug, 0);
                        ss.Bind(ipe);
                        ss.Listen(MAX_CONN);
                        
                        while (true)
                        {
                              Console.WriteLine("Listening on - "+ipe.ToString()+"\n");
                              IAsyncResult acceptDone = ss.BeginAccept(acceptCallback, null);
                              acceptDone.AsyncWaitHandle.WaitOne();                              

                              string x = Console.ReadLine().Trim().ToLower();
                              if (x=="")
                                    break;
                        }
                  }
                  catch (Exception ex)
                  {
                        Console.WriteLine(ex.Message);
                  }
                  finally
                  {
                        if (ss.Connected)
                              ss.Close();            
                        
                        GC.Collect();
                  }                  
                  Console.WriteLine("All Sockets have been disposed!");
                  Console.ReadLine();
            }
            static void AcceptCallback(IAsyncResult ar)
            {
                  try
                  {
                        cs = ss.EndAccept(ar);            
                        ar.AsyncWaitHandle.Close();
                        cs.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReceiveBuffer, MAX_BUFFER_SIZE);
                        cs.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.SendBuffer, MIN_BUFFER_SIZE);
                        cs.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.Debug, 0);
                        Console.WriteLine("Remote client - "+cs.RemoteEndPoint+"\n");            

                        buffer = new FileStream("new.txt", FileMode.Create, FileAccess.Write, FileShare.None);       
                        temp_buffer = new byte[MAX_BUFFER_SIZE];                  
                        cs.BeginReceive(temp_buffer, 0, MAX_BUFFER_SIZE, SocketFlags.None, receiveCallback, null);
                  }
                  catch (Exception ex)
                  {
                        Console.WriteLine(ex.Message);
                  }
            }
            static void ReceiveCallback(IAsyncResult ar)
            {
                  try
                  {
                        bytesRcvd = cs.EndReceive(ar);      
                        ar.AsyncWaitHandle.Close();
                        if (bytesRcvd>0)
                        {
                              unchecked { totalBytes += bytesRcvd; }
                              buffer.Write(temp_buffer, 0, bytesRcvd);                                    
                              temp_buffer = new byte[MAX_BUFFER_SIZE];
                              cs.BeginReceive(temp_buffer, 0, MAX_BUFFER_SIZE, SocketFlags.None, receiveCallback, null);
                        }
                        else
                        {                                                
                              if (cs.Connected)
                              {
                                    cs.Shutdown(SocketShutdown.Receive);
                                    cs.Close();
                              }
                              Console.WriteLine("Total data received:  "+Size(totalBytes));                                          
                              buffer.Close();      
                        }
                  }
                  catch (Exception ex)
                  {
                        Console.WriteLine(ex.Message);
                        if (cs.Connected)
                              cs.Close();                  
                  }
            }
            static string Size(int number)
            {
                  int mb = 1048576;            
                  if(number>mb)            
                        return String.Format("{0:0.0 mb}", Convert.ToDecimal(number)/mb);                  
                  else                  
                        return String.Format("{0:0.0 kb}", Convert.ToDecimal(number)/1024);                  
            }
      }
}




[CLIENT SOCKET]

using System;
using System.IO;
using System.Net;
using System.Text;
using System.Net.Sockets;
using System.Threading;
using System.Windows.Forms;

namespace ClientSocket
{
      class EntryPoint
      {
            static Socket s;
            const int MAX_BUFFER_SIZE = 66560; // tested on DSL connection
            const int MIN_BUFFER_SIZE = 1024;
            static int bytesLeft = MAX_BUFFER_SIZE;

            static FileStream fs = null;
            static byte[] fs_buffer;
            static int fs_bytesRead = 0;
            static int fs_length = 0;
            static int dataSent = 0;
            
            static ManualResetEvent connDone;
            static ManualResetEvent threadRun;

            [STAThread]
            static void Main(string[] args)
            {
                  connDone = new ManualResetEvent(false);
                  threadRun = new ManualResetEvent(false);
                  OpenFileDialog dialog = null;
                  try
                  {
                        dialog = new OpenFileDialog();                                                
                        if (dialog.ShowDialog()==DialogResult.OK)
                        {
                              IPEndPoint ipe = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 3535);
                              s = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                              s.Blocking = true;
                              s.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.SendBuffer, MAX_BUFFER_SIZE);
                              s.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReceiveBuffer, MIN_BUFFER_SIZE);      
                              s.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.Debug, 0);
                              s.BeginConnect(ipe, new AsyncCallback(ConnectCallback), ipe);
                              connDone.WaitOne();

                              fs = File.OpenRead(dialog.FileName);                                                                                    
                              Console.WriteLine("Data Transmission in process - "+DateTime.Now+"\n");
                              fs_length = (int)fs.Length;      

                              Thread t = new Thread(new ThreadStart(Read_Send));
                              t.Start();                              
                              threadRun.WaitOne();                                                            
                        }
                  }      
                  catch (Exception ex)
                  {
                        Console.WriteLine(ex.Message);
                  }
                  finally
                  {
                        if (s.Connected) s.Close();      
                        if (dialog!=null) dialog.Dispose();
                        if (fs!=null) fs.Close();

                        GC.Collect();
                  }
                  Console.WriteLine("Data Transmission completed - "+DateTime.Now+"\n");
                  Console.WriteLine("Total data sent:  "+Size(dataSent));
                  Console.ReadLine();
            }
            static void ConnectCallback(IAsyncResult ar)
            {
                  try
                  {
                        IPEndPoint ipe = (IPEndPoint)ar.AsyncState;
                        s.EndConnect(ar);
                        ar.AsyncWaitHandle.Close();      
                        Console.WriteLine("Remote host - "+ipe.ToString()+"\n");
                  }
                  catch (Exception ex)
                  {
                        Console.WriteLine(ex.Message);
                  }
                  finally
                  {
                        connDone.Set();      
                  }
            }
            static void Read_Send()
            {
                  unchecked
                  {
                        try
                        {
                              while (dataSent<fs_length)
                              {                        
                                    if ((fs_bytesRead+MAX_BUFFER_SIZE)>fs_length)
                                          bytesLeft = fs_length-fs_bytesRead;      
                        
                                    fs_buffer = new byte[bytesLeft];
                                    fs.Read(fs_buffer, 0, bytesLeft);      
                                    fs_bytesRead += bytesLeft;

                                    dataSent += s.Send(fs_buffer, 0, bytesLeft, SocketFlags.None);
                              }
                        }
                        catch (Exception ex)
                        {
                              Console.WriteLine(ex.Message);
                        }
                        finally
                        {
                              Thread.Sleep(TimeSpan.FromSeconds(1));
                              if (s.Connected)
                                    s.Shutdown(SocketShutdown.Send);
                              threadRun.Set();      
                        }
                  }
            }
            static string Size(int number)
            {
                  int mb = 1048576;            
                  if(number>mb)            
                        return String.Format("{0:0.0 mb}", Convert.ToDecimal(number)/mb);                  
                  else                  
                        return String.Format("{0:0.0 kb}", Convert.ToDecimal(number)/1024);                  
            }
      }
}

0
 
LVL 13

Author Comment

by:davidlars99
Comment Utility
Server socket is asyncronious and client socket is syncronious blocking. I couldn't get asyncronious client socket to work with files over 100 MB. I was getting System.StackOverflowException.
0
 
LVL 18

Expert Comment

by:Ravi Singh
Comment Utility
Not 100% sure on what the cause could be, but if the exception is occuring in relation to how large the file being transferred is then i'd guess that the callbacks are causing the problem... have you tried increasing the buffer sizes?



0
 
LVL 18

Expert Comment

by:Ravi Singh
Comment Utility
Hi Bob, sorry for not replying sooner. I feel the original question was answered as close as possible..
0
 
LVL 13

Author Comment

by:davidlars99
Comment Utility
yes it was! thank you.
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

Extention Methods in C# 3.0 by Ivo Stoykov C# 3.0 offers extension methods. They allow extending existing classes without changing the class's source code or relying on inheritance. These are static methods invoked as instance method. This…
This article introduced a TextBox that supports transparent background.   Introduction TextBox is the most widely used control component in GUI design. Most GUI controls do not support transparent background and more or less do not have the…
Sending a Secure fax is easy with eFax Corporate (http://www.enterprise.efax.com). First, Just open a new email message.  In the To field, type your recipient's fax number @efaxsend.com. You can even send a secure international fax — just include t…
This video discusses moving either the default database or any database to a new volume.

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now