Downloading Files using Camel and SFTP

In this example, we will download files from a Server using SFTP and store them in a local directory.

Create a Maven Project

Create a Maven Project

• groupId: com.skills421.examples.camel
• artifactId:  CamelBasics
• version: 0.0.1-SNAPSHOT

Update Pom.xml

[code language=”xml”]
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.skills421.examples.camel</groupId>
<artifactId>CamelBasics</artifactId>
<version>0.0.1-SNAPSHOT</version>

<dependencies>

<!– Core Camel –>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-core</artifactId>
<version>2.19.2</version>
</dependency>

<!– Needed for the FTP Component –>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-ftp</artifactId>
<version>2.19.2</version>
</dependency>

<!– Needed for com/jcraft/jsch/Logger – SFTP –>
<dependency>
<groupId>com.jcraft</groupId>
<artifactId>jsch</artifactId>
<version>0.1.54</version>
</dependency>

<!– JUnit for testing –>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
</dependency>

</dependencies>

</project>
[/code]

pom.xml

Edit FTPCopier.java

Edit ftpCopier.java to add the method sftpDownload as follows:

[code language=”java”]
package com.skills421.examples.camel.basics;

import org.apache.camel.CamelContext;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.DefaultCamelContext;

public class FTPCopier
{
public String getFtpPath(String type, String url,String remotePath,String username,String password)
{
StringBuilder ftpBuilder = new StringBuilder(type);
ftpBuilder.append("://")
.append(url)
.append("/")
.append(remotePath)
.append("?username=")
.append(username)
.append("&password=")
.append(password);

return ftpBuilder.toString();
}

public void ftpDownload(String url,String remotePath,final String localPath,String username,String password) throws Exception
{
CamelContext context = new DefaultCamelContext();

final String ftpPath = this.getFtpPath("ftp",url, remotePath, username, password);

context.addRoutes(new RouteBuilder()
{
public void configure()
{
from(ftpPath).to("file:"+localPath);
}
});

context.start();
Thread.sleep(10000);
context.stop();
}

public void sftpDownload(String url,String remotePath,final String localPath,String username,String password) throws Exception
{
CamelContext context = new DefaultCamelContext();

final String ftpPath = this.getFtpPath("sftp",url, remotePath, username, password);

context.addRoutes(new RouteBuilder()
{
public void configure()
{
from(ftpPath).to("file:"+localPath);
}
});

context.start();
Thread.sleep(10000);
context.stop();
}
}
[/code]

FTPCopier.java

Edit FTPCopierTest.java

Edit FTPCopierTest.java to @Ignore the ftp test and add the sftp test as follows:

[code language=”java”]
package com.skills421.examples.camel.basics;

import static org.junit.Assert.*;

import org.junit.Ignore;
import org.junit.Test;

public class FTPCopierTest
{

@Test
@Ignore
public void downloadFTPDir()
{
FTPCopier copier = new FTPCopier();

String url = "someftpserver.com";
String remotePath = "myfiles";
String localPath = "/Users/johndunning/Desktop/Camel/CamelOut";
String username = "username";
String password = "password";

try
{
copier.ftpDownload(url, remotePath, localPath, username, password);
}
catch (Exception e)
{
fail(e.getMessage());
}
}

@Test
public void downloadSFTPDir()
{
FTPCopier copier = new FTPCopier();

String url = "someftpserver.com";
String remotePath = "myfiles";
String localPath = "/Users/johndunning/Desktop/Camel/CamelOut";
String username = "username";
String password = "password";

try
{
copier.sftpDownload(url, remotePath, localPath, username, password);
}
catch (Exception e)
{
fail(e.getMessage());
}
}

}
[/code]

FTPCopierTest.java

Note, you will need to configure a sftp server to test your code.

Run the Test

Run the JUnit Test and you should see your files downloaded from your remote sftp server to your local directory.

Test Using a Free Public SFTP Server

Choose an sftp server from http://www.sftp.net/public-online-sftp-servers

• hostname: test.rebex.net
• login: demo
• password: password

SFTP from Server using bash commands:

[code language=”bash”]
sftp demo@test.rebex.net
Password: password

sftp> ls
list the files in the directory

sftp> get readme.txt
sftp> bye

more readme.txt
[/code]

SFTP from Server using FTPCopierTest

Now edit FtpCopierTest.java as follows

[code language=”java”]
…
String url = "test.rebex.net";
String remotePath = "";
String localPath = "/Users/johndunning/Desktop/Camel/CamelOut";
String username = "demo";
String password = "password";
…
[/code]

note: you will need to change the localPath to something meaningful for your machine.

Run the FTPCopierTest JUnit test and the file should download into your localPath