Actions

Data Transfer

From ALICE Documentation

Revision as of 09:23, 2 June 2021 by Schulzrf (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

work in progress

Best Practices and General Information

  • User scp for smaller files and sftp for larger files
  • For newly created accounts only the following directories exists: /home/<username> on the home file system and /data/<username> on the shared scratch file system. There is a symlink from your home directory to your shared scratch directory, i.e., /home/<username>/data can also be used.
  • You can use git on ALICE to clone repositories to your space on /home or /data.

Between your computer and ALICE

For many ALICE users, the most convenient way to transfer files between their computer and ALICE is to use a GUI (graphical user interface)-based SFTP client. The SFTP client is installed and run on your computer. It works by connecting to the SFTP server running on sftp-gw.alice.universiteitleiden.nl, enabling you to transfer files back and forth.

ALICE login credentials are required for access. Most third-party SSH clients can maintain an open connection so as to minimize the number of authentication requests during a transfer session. Refer to the documentation for how to do this for your preferred client.

From the Internet to ALICE

You can transfer a file from the Internet directly to your project directory on ALICE (without first downloading to your computer). For example, if you want to transfer a repository from GitHub, use the command git clone REPOSITORY_URL, where REPOSITORY_URL is the link you copied from Github. If you want to transfer a file from a web page, you can use the command wget URL. If you need to transfer data from a private location (i.e., one that requires logging in), the site may or may not allow you to use wget for the transfer.

If you need to frequently transfer files, plan to move large amounts of data, or need assistance transferring data from a private location, feel free to contact us at helpdesk@alice.universiteitleiden.nl for advice on how to.

Creating and Editing Files on ALICE

You can always create files on your personal computer and transfer them to ALICE but sometimes it is easiest to create them directly on ALICE.

HPC supports the vi/vim, gedit, nano and emacs text editors. Nano is used in HPC training sessions because it is an easy editor to learn. Gedit is a good option if you log in with “X11 forwarding” enabled which is pre-configured X-Win32 and enabled by XQuartz’s X11.app on Mac OS. Vi/vim, which comes standard on all UNIX/Linux machines, and emacs, which is a popular coding environment, both have steeper learning curves.

To edit a file, simply type the editors name, e.g., nano or gedit, at the command line and then type in your file’s text.

Transferring files from different systems