Storage – CfAI

Local Scratch

Every system has a local partition available for writing to, this is /scratch.

We recommend putting your data into a sub-directory with your username (this may already be generated on some systems) this is to stop other users editing/removing your data.

We recommend using scratch as much as possible as this will always be faster than any shared network storage.

Scratch space is not shared and is only available locally, so if you require the data over many systems, please ensure you copy this to and from a shared system prior/end of your work.

Scratch is never backed up and is liable to be erased as soon as a system is faulty.

Shared Network

Home

Your home space is on a shared network server and is automatically mapped to you ( ~/ and /mt/cfai/$USERNAME ). Your home directory is restricted access to only yourself.

By default all users get 50GB, we can expand this upon request up to 100GB. It is backed up daily.

The home server is used by everyone all the time, so we request that you do not run batch jobs, jupyter notebooks or any significant IO heavy work from/to your home directory.

Please note that the system is designed for research purposes only and not to store sensitive / Personally Identifiable Information (PII), we do not encrypt this data or encrypt the backups.

Quota

To find out your storage usage/quota on Home please use the zquota command and you should see an output similar to below

login~$ zquota

Quota Report for username
Mount Point Used Total Last Checked
/mt/cfai 14G (9%) 150G Mon 12 Apr 2021 12:22:06 BST

The zquota command will now also display any group batch storage usage, this will show the entire group usage and quota and not your individual allocation.

Anti-Virus Scanning

We periodically scan your file storage allocation for viruses. We receive a notification as soon as a virus is detected so we can apply the appropriate action, by default the system will not remove anything it suspects is a virus but we may do so manually.

A log of any files suspected to be a virus can be found in your home space under ~/.virusscan.