| Both sides previous revision
Previous revision
Next revision
|
Previous revision
|
fileaccess [2017/05/09 19:43]
mgstauff [rsync - secure remote copy] |
fileaccess [2018/03/26 14:57] (current)
mgstauff [What server to transfer to/from ?] |
| - contact the sysadmins to discuss using our secure FTP server | - contact the sysadmins to discuss using our secure FTP server |
| |
| ====== Transfering To/From Your Cluster Account ====== | ====== Transferring from PACS/Sectra or Scanners ===== |
| | |
| | Data from PACS and scanners can be transferred directly to the cluster. [[pacs| See the PACS page]]. |
| | |
| | ====== Transferring To/From Your Cluster Account ====== |
| |
| Generally you'll need to move files between your local/desktop computer and data directories on the cluster. There are a number of ways to do this: | Generally you'll need to move files between your local/desktop computer and data directories on the cluster. There are a number of ways to do this: |
| 170.212.169.225 - /data/picsl, /data/grossman | 170.212.169.225 - /data/picsl, /data/grossman |
| 170.212.169.49 - /data/tesla-data, /data/tesla-home | 170.212.169.49 - /data/tesla-data, /data/tesla-home |
| | crich - /data/jux |
| |
| ---- | ---- |
| ===Linux & Mac OSX ( & Windows ) === | ===Linux & Mac OSX ( & Windows ) === |
| |
| To use ''rsync'' on Windows, [[logging_in#windows_and_the_linux_bash_shell|see here]]. | |
| | To use ''rsync'' on __Windows__, [[logging_in#windows_and_the_linux_bash_shell|see here]]. |
| |
| This is a powerful command line program for copying files between computers on the network. The recommended command for transferring to the cluster is this: | This is a powerful command line program for copying files between computers on the network. The recommended command for transferring to the cluster is this: |
| |
| rsync rsync -prltD --chmod=Dug+rwx,Dg+s,Fug+rw,o-rwx <path-to-files-on-your-computer> <yourusername>@chead:</data/your-data-directory/target-directory> | rsync -prltD --chmod=Dug+rwx,Dg+s,Fug+rw,o-rwx <path-to-files-on-your-computer> <yourusername>@chead:</data/your-data-directory/sub-directory> |
| | |
| The options specified above tell rsync to recursively copy all files and directories that you specify in the command, and to modify file and directory ownership in a way that's appropriate for most cluster directories, and to preserve creation/modification dates. | The options specified above tell rsync to recursively copy all files and directories that you specify in the command, and to modify file and directory ownership in a way that's appropriate for most cluster directories, and to preserve creation/modification dates. |
| Also, **symlinks** are copied as symlinks, meaning the directory or file to which the symlink points is not copied. | Also, **symlinks** are copied as symlinks, meaning the directory or file to which the symlink points is not copied. |
| |
| ''path-to-files-on-your-computer'' is the path to the files on your computer that you want to copy to chead. **NOTE** that you should NOT have a ''/'' slash at the end of the path, if you want the directory itself to be copied to your destination. If you do have a ''/'' slash at the end, only the contents of the directory will be copied, and not the directory itself. Here's the example of this from the ''rsync'' man page: | ''<path-to-files-on-your-computer>'' is the path to the files on your computer that you want to copy to chead. **NOTE** that you should NOT have a ''/'' slash at the end of the path, if you want the directory itself to be copied to your destination. If you do have a ''/'' slash at the end, only the contents of the directory will be copied, and not the directory itself. Here's the example of this from the ''rsync'' man page: |
| |
| | |
| rsync -av /src/foo/ /dest/foo | rsync -av /src/foo/ /dest/foo |
| |
| ''yourusername@chead'' tells rsync to login on chead using your username | ''<yourusername>@chead'' tells rsync to login on chead using your username |
| |
| '':/data/your-data-directory/destination-directory'' is the path to your data directory on chead, e.g. /data/jet/mgstauff/destination | '':</data/your-data-directory/sub-directory>'' is the path to your data directory on chead, e.g. /data/jet/mgstauff/destination |
| |
| ===Permissions and Ownership considerations=== | ===Permissions and Ownership considerations=== |
| |
| rsync -prltD --chmod=Dug+rwx,Dg+s,Fug+rw,o-rwx | rsync -prltD --chmod=Dug+rwx,Dg+s,Fug+rw,o-rwx |
| | |
| The ''-a'' option is really an aggregate of these options: ''-rlptgoD''. So above, I've passed all those manually except ''-o, -g''. This means that ownership on the cluster will be preserved for any files that already exist there. Also if you're copying into a directory that uses the group 'sticky bit' to make all new files be owned by the directory's group (as we do for group data directories), then new files will get the appropriate group on the cluster. | The ''-a'' option is really an aggregate of these options: ''-rlptgoD''. So above, I've passed all those manually except ''-o, -g''. This means that ownership on the cluster will be preserved for any files that already exist there. Also if you're copying into a directory that uses the group 'sticky bit' to make all new files be owned by the directory's group (as we do for group data directories), then new files will get the appropriate group on the cluster. |
| |
