Access from your machine to tazAssuming your machine is one of the lab machines, you can log in to your taz account from this machine as follows:
[MACHINE]$ ssh TAZ_LOGIN@taz TAZ_LOGIN@taz's password: [taz]$From anywhere else, you must use the fully qualified name:
$ ssh TAZ_LOGIN@taz.cs.wcupa.eduThe reason we can use the short name is due to the Search Domains entry of cs.wcupa.edu made in the network configuration.
[MACHINE]$ ssh-keygenThis will prompt for a pass-phrase (empty), and a place to store the public key (the default). Therefore, hit "Enter" three times. View the results by typing:
[MACHINE]$ ls ~/.ssh id_rsa this is the private RSA key id_rsa.pub this is the public RSA key
Add the public key to tazUse 2 shells, one on your lab machine and one on taz. On taz, if necessary, create the directory (nothing happens if it already exists):
[taz]$ mkdir -p ~/.sshOn your lab machine, dump the contents of the public key:
[MACHINE]$ cat ~/.ssh/id_rsa.pubThen select the dumped content and copy it (right-click, Copy). In the taz shell, type this:
[taz]$ cat >> ~/.ssh/authorized_keysPaste the copy content (right-click, Paste) into the shell, and finish the operation by typing:
Ctrl-DOnce you're done, you should be able to get in "for free"
[MACHINE]$ ssh TAZ_LOGIN@tazThe goal is to append the RSA key from MACHINE into the ~/.ssh/authorized_keys file on taz. If something isn't working, double-check the contents of the file on taz:
[taz]$ nano ~/.ssh/authorized_keys
The SSH config fileOn your machine, create and edit the SSH config file:
[MACHINE]$ nano ~/.ssh/configadd these lines:
Host taz Hostname taz.cs.wcupa.edu User TAZ_LOGIN
[MACHINE]$ ssh tazThis will also work from an external computer running ssh, such as MAC, Linux, or Windows/Cygwin.
Running commands through SSHInstead of logging in and running commands on taz, you can run the command directly, like this:
[MACHINE]$ ssh taz lsIf the system you are on supports the X window system , you can run your Ubuntu GUI applications remotely on your client system, typically by using the "-X" option. For example, students in Dr. Wyatt's classes could run emacs remotely:
[MACHINE]$ ssh -X taz emacsThis will usually work well within a LAN, but at Internet speeds the connection can be very slow. The Remote Desktop is a better alternative.
From taz to your lab machineRepeat the steps in the last two subsections, except this time going the other way:
- on taz, create an RSA public/private key pair
- from your machine, retrieve the public RSA key file ~/.ssh/id_rsa.pub from taz (you should be able to do it without password)
- append the RSA public key onto your machine's ~/.ssh/authorized_keys file.
Host MACHINE User LOGIN
- scp: A command-line secure copy
- rsync: A robust, efficient command-line secure copy
- sftp: The basis of most SSH transfer clients. By itself, its features are very poor compared to the FTP clients ncftp and lftp.
- gftp: a UNIX-based, multi-protocol GUI transfer client.
- FileZilla: a multi-platform, multi-protocol transfer client.
- sshfs: The FUSE-mounting
scpThe scp command acts like cp except that you include the source or target servers along with login information in the command syntax. Using taz as the target machine, the command
$ scp file taz: $ scp -r dir taz:copies file and dir, respectively, from your machine to your taz home directory. As can be expected, the "TAZ_LOGIN@taz" is necessary without the ssh config file defined as we have done so. The ":" is a critical piece of syntax in the above commands because scp will copy files on the same computer if it is missing. Going the other way, we can do
$ scp taz:file . $ scp -r taz:dir .These operations copy a file or dir, respectively, from your taz home directory to your current directory. One important difference between scp and cp is that there is no option which you can use to warn you about overwriting a preexisting file, so be careful.
recursive and exact copiesThe rsync, cp and scp commands all support the -r (recursive) option to recursively copy a directory. An enhancement over the -r option is the -a option which attempts to make an exact copy, preserving file timestamps, symbolic links, ownership, etc. The -a option usage is available in the local cp command. If we compare:
$ cp -a dir1/ dir2/
$ cp -r dir1/ dir2/
The former will preserve symlinks whereas the latter would treat a symbolic link within dir1 as the file it points to, and copy the file. The -a option is available in the rsync command, but not for scp.
- The ability to copy only "what is necessary" using an algorithm based on file differences, with the implication that the actual amount of data transferred for a network copy is minimal.
- The ability to further reduce the amount of transferred traffic by compressing/decompressing with "-z" command-line flag.
- The ability to copy based on file time-stamps, so that "newer" files aren't overwritten by "older" files when the "-u" command-line option is present.
- The ability to refine the files transferred using inclusion/exclusion rules.
$ rsync -a -u -n -v DIRECTORY1/ DIRECTORY2/When you actually key in this command, it's convenient to compress the options like this:
$ rsync -aunv DIRECTORY1/ DIRECTORY2/
The terminal "/" for rsync directory copiesA subtle issue which appears in rsync is the terminal "/" in the source directory. In particular, the following means copy the directory dir1 into the target directory dir2:
$ rsync -r dir1 dir2In contrast, the following means copy the contents of the directory dir1 into the target directory dir2:
$ rsync -r dir1/ dir2
$ unzip ~/Downloads/rsync_samples.zip $ cd rsync_samplesIf you want to view the contents of any file, simple print it, e.g.,
$ cat setup1.shRun the commands as you see them on the left and confirm the descriptions on the right.
ls -RF source
ls -RF dest
source/ a.txt b.java c.txt D/ E/ e.txt f.txt D.conf dest/ D.java D.txt
|rsync -aunv source/ dest/||(dry run) update|
|rsync -aunv dest/ source/||(dry run) update in reverse|
|rsync -aunv --delete source/ dest/||(dry run) update and delete dest files not in source|
|rsync -aunv --delete dest/ source/||(dry run) update and delete source files not in dest|
|ls dest/||see what's in dest|
|rsync -auv source/ dest/||update for real|
|ls dest/||check that the files were sent|
|rsync -auv source/ dest/||do it again: nothing new to do|
This script runs
./setup1.sh rsync -a source/ dest/and then modifies these files:
source/ a.txt, D.conf dest/ b.java, c.txt, D/f.txt
|rsync -aunv source/ dest/||(dry run) send later changes in source|
|rsync -anv source/ dest/||
no -u option
(dry run) send all changes in source
- The folders used in the rsync command always terminate with a "/" this is important. It means we are, in some form, copying the folder contents, not the folder itself.
- The "-a" option sets the target files' timestamps to that of the source, thus multiple runs with no changes in the folder contents will have no effect.
- The "-n" option is called a dry run and suppresses the actual completion of the operation. It should always be used in conjunction with the "-v" (verbose) option so that you get a report of what would happen without committing to it.
- The "--delete" deletes files in the target which are not in the source. Thus this command:
rsync -a --delete source/ dest/will make the contents of dest exactly the same as that of source.
rsync to tazThe real importance of rsync is in its ability to do the same actions across a network connection. Like the scp command, to create a network transfer, we preface either the source or the target with this syntax:
login@host:Here are some commands which test the transfer to taz:
rsync -anv rsync_samples/ taz:rsync_samples/
transfer dry run
|rsync -av rsync_samples/ taz:rsync_samples/||do the transfer|
|rsync -av rsync_samples/ taz:rsync_samples/||repeat it, confirm nothing new|
exclusions/inclusionsThe rsync command has a fairly complicated way of refining which files should participate in the copy command using these options:
--exclude=PATTERN exclude files matching PATTERN --exclude-from=FILE read exclude patterns from FILE --include=PATTERN don't exclude files matching PATTERN --include-from=FILE read include patterns from FILE --files-from=FILE read list of source files names from FILEWe will focus on only one of these:
--exclude-from=FILE read exclude patterns from FILEThe FILE which rsync refers to can contain lines with glob patterns common to other commands, i.e., they use the special characters *, ?, etc. For example an exclude file may be this:
a.txt *.javaTo make the exclusion/inclusion more explicit, patterns use the prefixes with + and -, respectively:
+ include-pattern - exclude-patternIf these prefixes are used, there is no difference between --include-from and --exclude-from specification files. One has to be careful to place the + and - characters left justified in the line with exactly one blank between the character and the file name. The pattern notation also permits the usage of an initial "/" which means that the file name must match the pattern from the top of the directory. Thus the exclude file:
- *.txtwould exclude all the .txt files, whereas
- /*.txtwould exclude only the top level .txt files. Patterns can also employ the ** token, meaning to recursively enter all subdirectories. Thus
- D/*.txtwould exclude only the .txt files in D, but not in its subdirectories, and
- D/**.txtwould exclude all .txt files in D or any of its subdirectories. The rsync_samples directory provides several pattern files which you can use to test.
ls -RF source
ls -RF dest
source/ a.txt b.java c.txt D/ E/ e.txt f.txt D.conf dest/ D.java D.txt
|rsync -aunv source/ dest/||run it for reference|
|rsync -aunv --exclude-from=X source/ dest/||X:||
+ *.conf - *
|rsync -aunv --exclude-from=Y source/ dest/||Y:||
- D - *.java
|rsync -aunv --exclude-from=Z1 source/ dest/||Z1:||
+ D - *
|rsync -aunv --exclude-from=Z2 source/ dest/||Z2:||
+ D + D/* - *
|rsync -aunv --exclude-from=Z3 source/ dest/||Z3:||
+ D + D/** - *
Windows and other systems provide a number of GUI Secure-copy alternatives, possibly the best being WinSCP for Windows.
scp via a shellFor simplicity, we'll show a shell approach which would be viable for MAC/Linux/Windows+Cygwin systems. We'll assume that 2002 is the port forwarded to 22 on MACHINE. Edit the ~/.ssh/config file, assumed to have the "tunnel" entry created and add a new entry.
... Host MACHINE_tunnel User TAZ_LOGIN Host taz.cs.wcupa.edu LocalForward 2002 MACHINE:22 ... # new entry: Host MACHINE Hostname localhost User LOGIN Port 2002
$ scp FILE_ON_CLIENT MACHINE: # or, $ scp -r DIRECTORY_ON_CIENT MACHINE:
$ sudo apt-get install sshfsThen create a mount point:
$ mkdir ~/TAZMount your taz home directory onto this mount point by:
$ sshfs taz:/home/TAZ_LOGIN ~/TAZYou can then see the contents of your taz home directory by:
$ ls ~/TAZTo unmount this file system, do:
$ fusermount -u ~/TAZYou cannot close it by right-clicking and selecting Unmount (on Linux). To do the same thing to your lab machine, activate the "taz tunnel" with ~/.ssh/config entries as above. Create a dedicated directory:
$ mkdir ~/MACHINEMount your lab machine's home directory onto this by:
$ sshfs bashir:/home/LOGIN ~/MACHINE
$ mkdir ~/TAZThen mount your taz home directory onto your local machine:
$ sshfs taz:/home/TAZ_LOGIN ~/TAZTo remove the mount use the umount command (different than on Linux):
$ umount ~/TAZ