Solved

internet file transfer

Posted on 2006-06-19
17
294 Views
Last Modified: 2008-01-09
hi,
I was just wondering how yahoo makes it possible for com1 and com2 to connect to each other? The reason I am asking this is because I have created small network application, which main purpose is to transfer a file from com1 to com2, and unless I open/forward the port on com2, com1 is unable to connect to it. Both PCs use same mini DLS routers.

Yahoo has no problem at all when I do the same task because it creates direct connection but, I have no idea how... does it use some sort of network address translation internally or there is something else to it..?
0
Comment
Question by:davidlars99
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 8
  • 7
17 Comments
 
LVL 18

Expert Comment

by:Ravi Singh
ID: 16937534
Hi, not 100% sure if yahoo messenger supports this, but a lot of the peer-to-peer software today uses the UPnP protocol. Typically routers support this as a feature and allow applications to programmatically configure ports...

More Info - http://en.wikipedia.org/wiki/Upnp
C# Implementation - http://www.pixvillage.com/blogs/devblog/archive/2005/03/31/183.aspx
0
 
LVL 13

Author Comment

by:davidlars99
ID: 16938184
hi Zephyr__, I assumed so but, I don't think yahoo uses UPnP because it is disabled on my router. Can you still benefit from UPnP if it's disabled?
0
 
LVL 18

Expert Comment

by:Ravi Singh
ID: 16938422
Hi again, doubt its usable if its disabled, could it be the case that them ports are already open (predefined port mappings)? Yahoo Messenger uses the following ports in case you want to check:

5000 (TCP - Chat)
5001 (TCP - Chat)
5055 (UDP - Phone)
5100 (TCP - Webcam*)

from: http://support.dlink.com/faq/view.asp?prod_id=1428
0
Instantly Create Instructional Tutorials

Contextual Guidance at the moment of need helps your employees adopt to new software or processes instantly. Boost knowledge retention and employee engagement step-by-step with one easy solution.

 
LVL 13

Author Comment

by:davidlars99
ID: 16938520
yes I know it does but, I have a DSL mini-router and as far as I know there has to be port forwarding enabled for those port that yahoo uses...  

I'll check if my router has pre-configured forwarding for yahoo ports, and let you know.
0
 
LVL 13

Author Comment

by:davidlars99
ID: 16938538
by the way, I have Westell 6600 router
0
 
LVL 18

Expert Comment

by:Ravi Singh
ID: 16938634
I can't find any specific information relating to how yahoo messenger overcomes problems related to NAT (in terms of direct p2p connections), but I have in the past come across various ways to bypass NAT to create p2p friendly applications (some of them involve the use of an intermediate server to "introduce peers". Some of the methods used are explained on the webpages below, they should give you a better idea:

NAT and Peer-to-peer networking:
http://alumnus.caltech.edu/~dank/peer-nat.html

How to communicate peer-to-peer through NAT (Network Address Translation):
http://www.mindcontrol.org/~hplus/nat-punch.html

Building NAT-Friendly Applications:
http://www.intel.com/cd/ids/developer/asmo-na/eng/79524.htm

Hope this helps.
0
 
LVL 13

Author Comment

by:davidlars99
ID: 16938862
Zephyr__, will this examples work if there's a intermediate server, which introduces peers to each other but both computers need to have port forwarding done?

I am confused...
0
 
LVL 18

Expert Comment

by:Ravi Singh
ID: 16941640
Hi,

Using the server to introduce the peers eliminates the need for port forwarding on the gateway (router) devices. So if for example peer1 wanted to communicate with peer2 (and both are behind a NAT enabled gateway) then using the approach explained in the second link (http://www.mindcontrol.org/~hplus/nat-punch.html)... peer1 would send a packet to the server, in order for the packet to reach the server it would go through the local NAT enabled gateway, the gateway would inject a "return address" into the packet (based on some unused port) and send the packet onto the server. The server would get the packet, note the return address for the packet and send the return address to peer2, peer2 can then communicate to peer1 using the return address, when peer2 sends a packet to peer1 on the return address... the NAT device on peer1 would recognise the return address should map to peer1 on the local network and forwards the packet to peer1...

The above was a simplified explanation, if you read the the webpage mentioned, it explains it in more detailed (pay extra attention to the section "Introducing the Introducer")... yahoo may or may not support this approach, but its one known way of bypassing NAT's limitations for p2p communication

Hope this helps
0
 
LVL 13

Author Comment

by:davidlars99
ID: 16942006
I read that section about 10 times...  :-)

what I don't understand are two things:
  1.  how can peer1 send a packet to peer2 without properly connecting to it, using Socket.Connect(EndPoint)?
  2.  how can peer2 accept a packet from a stranger such as peer1, when peer2 has never seen peer1's IP address?

or maybe I should cut the crap and just try it out?
0
 
LVL 18

Accepted Solution

by:
Ravi Singh earned 500 total points
ID: 16942787
Hi david,

The server would typically store the current IP's of the "online" peers, so once a peer has authenticated on the server, the server would probably send the authenticated peer a list of the online peers with their proper "global" ip addresses, thus allowing peers to establish direct connections with other peers.

Peer2 can accept the packet because the NAT gateway maintains a list of return addresses which it assigns to packets on their way out of the local network (i.e. if peer2 sends a packet to the server - the packet is first intercepted by the router - at this point the packets return address is the local peer computers' ipaddress + the port its listening on - the router changes the return address to the ip address of the router + an available port number on the router - it stores this info in a local lookup table - so when the packet eventually arrives at the server its return address is the routers ip:port number - the server would then send this to peer1 (for example when peer1 authenticates on the peer network) - peer1 can then send a packet directly to that ipaddress+port number of the router (which peer2 is under), the router performs the lookup and notices that the incoming packet has the destination address of the one assigned by the router as a return address - the router then forwards this packet to peer2 on the port number which the originating software was listening on.

So the packet goes to the port which the software was originally listening on, I guess its then upto you to perform some checks to see whether it is actually a packet that your listening for (using some secret headers etc in the data).
0
 
LVL 13

Author Comment

by:davidlars99
ID: 16973361
ok finally got it to work. tested on DLS connection, where two computers have same exact speed (768 kb/sec) so, 36.1 MB file took exactly 29 minutes to transfer from peer1 to peer2, which is exactly 10 minutes faster than yahoo file transfer on both same computers...  ;)

[SERVER SOCKET]

using System;
using System.IO;
using System.Net;
using System.Text;
using System.Net.Sockets;
using System.Threading;

namespace ServerSocket
{
      class EntryPoint
      {
            static Socket cs;
            static Socket ss;            
            const int MAX_CONN = 1;
            const int MAX_BUFFER_SIZE = 66560; // tested on DSL connection
            const int MIN_BUFFER_SIZE = 1024;

            static FileStream buffer;

            static int totalBytes = 0;
            static int bytesRcvd = 0;
            static byte[] temp_buffer;

            static AsyncCallback receiveCallback;
            static AsyncCallback acceptCallback;

            [STAThread]
            static void Main(string[] args)
            {
                  receiveCallback = new AsyncCallback(ReceiveCallback);
                  acceptCallback = new AsyncCallback(AcceptCallback);

                  try
                  {
                        IPEndPoint ipe = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 3535);
                        ss = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                        ss.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, 1);
                        ss.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReceiveBuffer, MAX_BUFFER_SIZE);
                        ss.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.Debug, 0);
                        ss.Bind(ipe);
                        ss.Listen(MAX_CONN);
                        
                        while (true)
                        {
                              Console.WriteLine("Listening on - "+ipe.ToString()+"\n");
                              IAsyncResult acceptDone = ss.BeginAccept(acceptCallback, null);
                              acceptDone.AsyncWaitHandle.WaitOne();                              

                              string x = Console.ReadLine().Trim().ToLower();
                              if (x=="")
                                    break;
                        }
                  }
                  catch (Exception ex)
                  {
                        Console.WriteLine(ex.Message);
                  }
                  finally
                  {
                        if (ss.Connected)
                              ss.Close();            
                        
                        GC.Collect();
                  }                  
                  Console.WriteLine("All Sockets have been disposed!");
                  Console.ReadLine();
            }
            static void AcceptCallback(IAsyncResult ar)
            {
                  try
                  {
                        cs = ss.EndAccept(ar);            
                        ar.AsyncWaitHandle.Close();
                        cs.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReceiveBuffer, MAX_BUFFER_SIZE);
                        cs.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.SendBuffer, MIN_BUFFER_SIZE);
                        cs.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.Debug, 0);
                        Console.WriteLine("Remote client - "+cs.RemoteEndPoint+"\n");            

                        buffer = new FileStream("new.txt", FileMode.Create, FileAccess.Write, FileShare.None);       
                        temp_buffer = new byte[MAX_BUFFER_SIZE];                  
                        cs.BeginReceive(temp_buffer, 0, MAX_BUFFER_SIZE, SocketFlags.None, receiveCallback, null);
                  }
                  catch (Exception ex)
                  {
                        Console.WriteLine(ex.Message);
                  }
            }
            static void ReceiveCallback(IAsyncResult ar)
            {
                  try
                  {
                        bytesRcvd = cs.EndReceive(ar);      
                        ar.AsyncWaitHandle.Close();
                        if (bytesRcvd>0)
                        {
                              unchecked { totalBytes += bytesRcvd; }
                              buffer.Write(temp_buffer, 0, bytesRcvd);                                    
                              temp_buffer = new byte[MAX_BUFFER_SIZE];
                              cs.BeginReceive(temp_buffer, 0, MAX_BUFFER_SIZE, SocketFlags.None, receiveCallback, null);
                        }
                        else
                        {                                                
                              if (cs.Connected)
                              {
                                    cs.Shutdown(SocketShutdown.Receive);
                                    cs.Close();
                              }
                              Console.WriteLine("Total data received:  "+Size(totalBytes));                                          
                              buffer.Close();      
                        }
                  }
                  catch (Exception ex)
                  {
                        Console.WriteLine(ex.Message);
                        if (cs.Connected)
                              cs.Close();                  
                  }
            }
            static string Size(int number)
            {
                  int mb = 1048576;            
                  if(number>mb)            
                        return String.Format("{0:0.0 mb}", Convert.ToDecimal(number)/mb);                  
                  else                  
                        return String.Format("{0:0.0 kb}", Convert.ToDecimal(number)/1024);                  
            }
      }
}




[CLIENT SOCKET]

using System;
using System.IO;
using System.Net;
using System.Text;
using System.Net.Sockets;
using System.Threading;
using System.Windows.Forms;

namespace ClientSocket
{
      class EntryPoint
      {
            static Socket s;
            const int MAX_BUFFER_SIZE = 66560; // tested on DSL connection
            const int MIN_BUFFER_SIZE = 1024;
            static int bytesLeft = MAX_BUFFER_SIZE;

            static FileStream fs = null;
            static byte[] fs_buffer;
            static int fs_bytesRead = 0;
            static int fs_length = 0;
            static int dataSent = 0;
            
            static ManualResetEvent connDone;
            static ManualResetEvent threadRun;

            [STAThread]
            static void Main(string[] args)
            {
                  connDone = new ManualResetEvent(false);
                  threadRun = new ManualResetEvent(false);
                  OpenFileDialog dialog = null;
                  try
                  {
                        dialog = new OpenFileDialog();                                                
                        if (dialog.ShowDialog()==DialogResult.OK)
                        {
                              IPEndPoint ipe = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 3535);
                              s = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                              s.Blocking = true;
                              s.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.SendBuffer, MAX_BUFFER_SIZE);
                              s.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReceiveBuffer, MIN_BUFFER_SIZE);      
                              s.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.Debug, 0);
                              s.BeginConnect(ipe, new AsyncCallback(ConnectCallback), ipe);
                              connDone.WaitOne();

                              fs = File.OpenRead(dialog.FileName);                                                                                    
                              Console.WriteLine("Data Transmission in process - "+DateTime.Now+"\n");
                              fs_length = (int)fs.Length;      

                              Thread t = new Thread(new ThreadStart(Read_Send));
                              t.Start();                              
                              threadRun.WaitOne();                                                            
                        }
                  }      
                  catch (Exception ex)
                  {
                        Console.WriteLine(ex.Message);
                  }
                  finally
                  {
                        if (s.Connected) s.Close();      
                        if (dialog!=null) dialog.Dispose();
                        if (fs!=null) fs.Close();

                        GC.Collect();
                  }
                  Console.WriteLine("Data Transmission completed - "+DateTime.Now+"\n");
                  Console.WriteLine("Total data sent:  "+Size(dataSent));
                  Console.ReadLine();
            }
            static void ConnectCallback(IAsyncResult ar)
            {
                  try
                  {
                        IPEndPoint ipe = (IPEndPoint)ar.AsyncState;
                        s.EndConnect(ar);
                        ar.AsyncWaitHandle.Close();      
                        Console.WriteLine("Remote host - "+ipe.ToString()+"\n");
                  }
                  catch (Exception ex)
                  {
                        Console.WriteLine(ex.Message);
                  }
                  finally
                  {
                        connDone.Set();      
                  }
            }
            static void Read_Send()
            {
                  unchecked
                  {
                        try
                        {
                              while (dataSent<fs_length)
                              {                        
                                    if ((fs_bytesRead+MAX_BUFFER_SIZE)>fs_length)
                                          bytesLeft = fs_length-fs_bytesRead;      
                        
                                    fs_buffer = new byte[bytesLeft];
                                    fs.Read(fs_buffer, 0, bytesLeft);      
                                    fs_bytesRead += bytesLeft;

                                    dataSent += s.Send(fs_buffer, 0, bytesLeft, SocketFlags.None);
                              }
                        }
                        catch (Exception ex)
                        {
                              Console.WriteLine(ex.Message);
                        }
                        finally
                        {
                              Thread.Sleep(TimeSpan.FromSeconds(1));
                              if (s.Connected)
                                    s.Shutdown(SocketShutdown.Send);
                              threadRun.Set();      
                        }
                  }
            }
            static string Size(int number)
            {
                  int mb = 1048576;            
                  if(number>mb)            
                        return String.Format("{0:0.0 mb}", Convert.ToDecimal(number)/mb);                  
                  else                  
                        return String.Format("{0:0.0 kb}", Convert.ToDecimal(number)/1024);                  
            }
      }
}

0
 
LVL 13

Author Comment

by:davidlars99
ID: 16973384
Server socket is asyncronious and client socket is syncronious blocking. I couldn't get asyncronious client socket to work with files over 100 MB. I was getting System.StackOverflowException.
0
 
LVL 18

Expert Comment

by:Ravi Singh
ID: 16973564
Not 100% sure on what the cause could be, but if the exception is occuring in relation to how large the file being transferred is then i'd guess that the callbacks are causing the problem... have you tried increasing the buffer sizes?



0
 
LVL 18

Expert Comment

by:Ravi Singh
ID: 17142718
Hi Bob, sorry for not replying sooner. I feel the original question was answered as close as possible..
0
 
LVL 13

Author Comment

by:davidlars99
ID: 17143464
yes it was! thank you.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

We all know that functional code is the leg that any good program stands on when it comes right down to it, however, if your program lacks a good user interface your product may not have the appeal needed to keep your customers happy. This issue can…
Performance in games development is paramount: every microsecond counts to be able to do everything in less than 33ms (aiming at 16ms). C# foreach statement is one of the worst performance killers, and here I explain why.
There's a multitude of different network monitoring solutions out there, and you're probably wondering what makes NetCrunch so special. It's completely agentless, but does let you create an agent, if you desire. It offers powerful scalability …
Do you want to know how to make a graph with Microsoft Access? First, create a query with the data for the chart. Then make a blank form and add a chart control. This video also shows how to change what data is displayed on the graph as well as form…

688 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question