Linux File transfer

More or less secure methods of transferring files.
By: Mihai Marinof, Linux Editor


When we need to download or copy a file from a remote machine to our local one or vice-versa, most of us use a plain, simple to use, FTP client. There are actually a wealth of tools available for transferring files, and some of them perform automation functions that can easily assist your business in building site mirrors, synchronizing directory contents, and more.

These lower security tools include lftp, rsync, and wget. Many of these programs can handle more than just FTP connections, and some even have their own shell-like syntax for sophisticated use.

The high security tools include scp and sftp are specially built for using in a secure manner. These are the tools you'll want to develop a habit of using when transferring information you need to keep private. You'll need to have ssh configured between the two machines in question before you can use either of these secure tools.


1.  lftp

The lftp tool can handle six different methods of file transfer (see the man page for the full list), including FTP and HTTP as well as the option of OpenSSL-based

secure methods if they were included when the program was compiled. As example, I'll show you how to grab the latest kernel version from the kernel.org FTP. Open a terminal and type:

lftp ftp.kernel.org

Cd to v2.6 releases dir:

lftp ftp.kernel.org:/> cd /pub/linux/kernel/v2.6
(just like in a normal console, you can use TAB for auto-completion)

The v2.6 directory contains, apart from the kernel releases, changelogs and paths. We are only interested to find out what is the latest full kernel package, so in the lftp prompt, type:

lftp ftp.kernel.org:/pub/linux/kernel/v2.6> ls | grep linux | more

Scroll down using enter or space until you find the latest version. Press q to exit 'more' and, to download the package, type:

lftp ftp.kernel.org:/pub/linux/kernel/v2.6> get linux-2.6.18.tar.bz2
(2.6.18 is the latest version at the time of this guide, you'll have to get whatever is the latest version in your present time)



2.  rsync

rsync comes to us from the Samba project, at http://rsync.samba.org/. This underutilized but valuable tool is excellent for keeping Web and FTP site mirrors up to date, not to mention for keeping the contents of local directories within your network in sync. You can also use it for private "secure" purposes such as data backup, as long as you are sure to utilize rsync within an ssh connection.

Say you're using Fedora Core 5 and want to grab the latest packages without using yum. Let's also say the last update was made with yum, which left all the latest update packages at that time, in its cache directory.

Begin by finding out if there are any rsync servers running on this server. The command to use for this is:

rsync carroll.cac.psu.edu::

Since what I'm interested in is Fedora Updates, I’ll now type the following to find the contents of the fedora-linux-core section:

rsync carroll.cac.psu.edu::fedora-linux-core

This command will show you the contents of the fedora section. To download everything in the updates you don't already have, use this command:

rsync -uv carroll.cac.psu.edu::fedora-linux-core/updates/5/SRPMS/* /var/cache/yum/updates-released/packages/

The -u flag tells rsync to only grab the files that I don't already have, and the -v tells rsync to be verbose and show me the name of each file as it's grabbing them rather than just showing me the server's banner and then sitting there silently while it does its work. The path at the end tells rsync where I want the files to go.



3.  wget

GNU's wget utility is a non-interactive download tool, meaning that it has no command line features to match lftp's functionality. You have access to FTP, HTTP, HTTPS, and proxied HTTP files using this program, but you have to know ahead of time what file you're trying to download, and where it is in the system's path.

To download the Google default page:

wget http://www.google.com

If there are a list of pages, files, and so on that I want to grab for the script, I can list one URL-formatted item per line within a file. For example, if the file was ~/bin/getme, I would use:

wget -i ~/bin/data/getme

I could even tell wget to grab all of the URLs listed in a particular HTML file. If the default file I downloaded from Google was index.html and it was saved in my home directory, then I would use the following to have wget grab every URL referenced in this file:

wget -i index.html -F

Notice the need to keep the flag's option with the flag. This command will not work if you use -iF .


4.  scp

scp is a secure version of the old rcp tool, that uses ssh to prevent people from sniffing out what you're transferring. How you utilize this tool depends on whether you're using the same account name on both machines, or different account names. Otherwise, if you have your ssh set up properly, this is a pretty straightforward program.

To copy the file sample1 to the recipient host example2 using scp, from the account bob on the local machine to the account bob on the remote machine, you would type:

scp sample1 example2:

However, if you wanted to copy the same file from bob on the local machine to jane on the remote machine, you would need to use a user@host format, such as:

scp sample1 jane@example2:

To use scp over a different port, you will have to use the -oPort=74 option.



5.  sftp

sftp is an interactive tool that works over an ssh connection, mirroring the ftp program's functionality—and is in fact a nice front end to scp. You won't be using this client for anonymous downloads, but if you need to move data or other confidential information between machines, this is an excellent tool to choose. Once you've mastered the scp command, you'll find sftp simple to use, or vice versa. They share many of the same flags and mostly the same syntax.

Once I have ssh set up for a proper connection on both machines, I can open the connection using:

sftp [e-mail]

After being challenged for the password for user "tom", which is transmitted in a tunnel through the secure shell, I'm in and have the sftp prompt. Now, I can use any of the commands in the sftp man page's INTERACTIVE COMMANDS section. So think of sftp as the more advanced, interactive cousin of scp.


linux transfer to windows:
1.    Samba
2.    NFS
3.    Cygwin
4.    Rsync