Skip to content
Ahmad Ammar

4. Files and storage

4.1 Where things go

Clusters have several storage areas with different rules. Learn yours before you create anything big.

Kelvin2 only, according to the quick-start guide:

Area

Path

Limit

Deletion

Home

/users/$USER

GB and 100k files

None automatic

Scratch

/mnt/scratch2/users/$USER

No per-user quota

After 90 days without access

Temp

/tmp on each node

The node's disk

When the session ends

McClayRDS

Project paths

By project

QUB projects only

quota -s shows your actual allocation.

The key rule: scratch is not a backup and not an archive. Keep a copy of results that matter on your institution's storage, and ask your support team what, if anything, is backed up.

4.2 A tidy project

## cluster

## Braces expand: this makes all four folders
mkdir -p ~/project/{code,jobs,logs,envs}

## Big data lives on scratch
mkdir -p /mnt/scratch2/users/$USER/project/{data,outputs,tmp}

## A shortcut, so ~/project/scratch stands for the long path
ln -s /mnt/scratch2/users/$USER/project ~/project/scratch

4.3 Moving files with scp

scp source destination copies securely. A remote location is written kelvin:path, using your shortcut. Run it from your laptop, in a separate terminal from your cluster session.

## laptop

## Upload a file. Quotes are needed because of the space in the path
scp "D:\My Work\data.csv" kelvin:~/project/scratch/data/

## Upload a folder: -r means recursive
scp -r "D:\My Work\code" kelvin:~/project/

## Download: the source is now the remote one
scp kelvin:~/project/scratch/outputs/results.tgz "D:\My Work"

Without the shortcut, the port needs a capital -P, unlike ssh, which uses a lower-case -p:

## laptop
scp -P 55890 -i $HOME\.ssh\my-kelvin-key data.csv <username>@login.kelvin.alces.network:~/project/

A Windows trap. Do not end a quoted folder with a backslash. In "D:\My Work\", the \" is read as a literal quote mark, and you get a confusing "No such file or directory". Write "D:\My Work".

4.4 Many small files: pack first

Sending thousands of small files one by one is slow, and hard on shared storage. Pack, send, unpack:

## laptop
## c create, z compress, f file name. --exclude leaves out unwanted folders
tar --exclude=.git -czf code.tgz code
scp code.tgz kelvin:~/project/
## cluster
## x extract. && means each step runs only if the last succeeded
cd ~/project && tar xzf code.tgz && rm code.tgz

Coming back, pack only what you need, such as summaries rather than every raw output.

4.5 Large transfers

Kelvin2 only. For large amounts of data, use a data movement node rather than a login node: dm1.kelvin.alces.network or dm2.kelvin.alces.network. Tools such as rsync can resume an interrupted transfer. MobaXterm and WinSCP offer drag and drop.

4.6 Files that arrive read-only

Copied files can keep restrictive permissions, leading to Permission denied when you edit them. chmod -R u+w ~/project/code gives you write permission on everything in that folder.