Solved

How to connect to multiple parallel SFTP using JSCH?

Posted on 2013-10-30
6
4,927 Views
Last Modified: 2013-10-31
I'm using JSCH to connect to SFTP. What I'm trying to do is, I have some SFTP url, hostname and password stored in my database. Using query, I'll select ,say 5, url, hostname and password form my database and establish SFTP connection to transfer some file. I'm able to connect to SFTP one at time using a for or while loop. But I want to connect to SFTP parallelly. I tried the ExecutorService to create a thread pool to establish parallel connection but it is throwing an error.

ExecutorService executor = Executors.newFixedThreadPool(n);
for(int i = 0; i<n; i++){
Runnable task = new SFTPTask(outbound);
executor.submit(task);
}

executor.shutdown();
executor.awaitTermination(interval, TimeUnit.SECONDS);



class SFTPTask implements Runnable {

		SchedulerOutbound outbound;

		public SFTPTask(SchedulerOutbound outbound) {
			this.outbound = outbound;
		}

		@Override
		public void run() {

			List<SFTPUser> list = service.getList(outbound.getProvider());

			filename = outbound.getDataX12().split("IEA")[1].split("\\*")[2]
					.split("~")[0]
					+ "." + fileExt;

			try {
				BufferedOutputStream out = new BufferedOutputStream(
						new FileOutputStream(sourceFolder + filename));
				byte[] dataInByte = outbound.getDataX12().getBytes();
				out.write(dataInByte);

				out.flush();
				out.close();
			} catch (FileNotFoundException e) {

				e.printStackTrace();

			} catch (IOException e) {

				e.printStackTrace();

			}

			loginId = list.get(0).getLoginFk();
			url = list.get(0).getSftpUrl();
			login = list.get(0).getSftpLogin();
			password = list.get(0).getSftpPasswd();
			host = url.split("//")[1].split(":")[0];
			port = Integer.parseInt(url.split(":")[2]);
			System.out.println(loginId + url + login + password);
			try {

				JSch jsch = new JSch();
				session = jsch.getSession(login, host, port);
				session.setPassword(password);
				Properties config = new java.util.Properties();
				config.put("StrictHostKeyChecking", "no");
				session.connect();
				channel = session.openChannel("sftp");
				channel.connect();
				ChannelSftp channelSftp = (ChannelSftp) channel;

				channelSftp.cd(destFolder);

				File f = new File(sourceFolder + filename);

				channelSftp.put(new FileInputStream(f), f.getName());

				service.updateDataProcessed(outbound);
				 f.delete();
				channelSftp.exit();

			} catch (Exception e) {
				service.setAuthStatus(loginId, "failed");
				e.printStackTrace();

			} finally {
				if (channel != null) {
					channel.disconnect();
					session.disconnect();
					service.setAuthStatus(loginId, "success");
				}
			}
		}
	}
      

Open in new window

                           


I'm getting the following errors:

com.jcraft.jsch.JSchException: session is down

com.jcraft.jsch.JSchException: connection is closed by foreign host
0
Comment
Question by:spectrumsofttech
  • 2
  • 2
  • 2
6 Comments
 
LVL 82

Expert Comment

by:Dave Baldwin
ID: 39613412
Since SFTP runs thru SSH on port 22, I don't see how you can make multiple connections on the same port at the same time.  Unless there is a way to use multiple source ports on your machine.
0
 

Author Comment

by:spectrumsofttech
ID: 39613429
Yes, host and port may be same but the sftp login and password will be different. Shouldn't it be possible to connect to multiple sftp in parallel, if you have different login and password credentials even though host and port are the same?
0
 
LVL 35

Accepted Solution

by:
mccarl earned 500 total points
ID: 39613437
What you posted above must be some *edited* form of your code, and the bits that you have edited out and probably crucial to solving this problem!!

In particular I would need to see how and where you have declared the "session" and "channel" variables. My guess is that they are declared in a scope that means there is only one instance of these variable shared among ALL threads. And so, on line 57 for example, you will have all X number of threads trying to set the *same* session variable to different values. The same for the channel variable.
0
Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

 
LVL 82

Expert Comment

by:Dave Baldwin
ID: 39613450
I suspect that there is only one thread and only one instance of all the variables.  To make it even possible, you would have to have a thread for each instance (I believe) and multiple source ports.  I know that in TCPView when you see Firefox making multiple connections, all the source ports are different.  I may be wrong but I don't think you can have multiple instances or programs using the same port at the same time.
0
 

Author Closing Comment

by:spectrumsofttech
ID: 39613528
Your guess about session and channel was correct. That hint was enough to solve my problem. Thank you for the help.
0
 
LVL 35

Expert Comment

by:mccarl
ID: 39613867
You're welcome, glad I could help! :)
0

Featured Post

Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

INTRODUCTION Working with files is a moderately common task in Java.  For most projects hard coding the file names, using parameters in configuration files, or using command-line arguments is sufficient.   However, when your application has vi…
Java functions are among the best things for programmers to work with as Java sites can be very easy to read and prepare. Java especially simplifies many processes in the coding industry as it helps integrate many forms of technology and different d…
Video by: Michael
Viewers learn about how to reduce the potential repetitiveness of coding in main by developing methods to perform specific tasks for their program. Additionally, objects are introduced for the purpose of learning how to call methods in Java. Define …
Viewers will learn about arithmetic and Boolean expressions in Java and the logical operators used to create Boolean expressions. We will cover the symbols used for arithmetic expressions and define each logical operator and how to use them in Boole…

759 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now