Storage – IPPP
Every system has a local partition available for writing to, this is /scratch.
We recommend putting your data into a sub-directory with your username (this may already be generated on some systems) this is to stop other users editing/removing your data.
We recommend using scratch as much as possible as this will always be faster than any shared network storage.
Scratch space is not shared and is only available locally, so if you require the data over many systems, please ensure you copy this to and from a shared system prior/end of your work.
Scratch is never backed up and is liable to be erased as soon as a system is faulty.
Your home space is on a shared network server and is automatically mapped to you ( ~/ and /mt/home/$USERNAME ).
By default all users get 20GB, we can expand this upon request upto 50GB. It is backed up daily.
The home server is used by everyone all the time, so we request that you do not run batch jobs, jupyter notebooks or any significant IO heavy work from/to your home directory.
User Batch space (or “User Batch” ) is available at /mt/user-batch/$USERNAME, this storage is aimed at your IO heavy work that requires personal storage, we generally recommend that you copy (ideally tar’d) your data to/from rather than running directly from Batch but we are aware that this is not always possible.
By default, users get 1TB, there is currently limited expansion upon request all the way up to 10TB depending on the current overall utilisation.
User Batch is NOT backed up, the amount of data stored is far too great to economically backup, so please be aware that data loss on this system is permanent.
Group Batch space is currently experimental and requires some prior configuration before use, so please get in touch and we can provision the storage to your requirements.
Group Batch space (or “Group Batch” ) is available at /mt/group-batch/$GROUPNAME, this storage is aimed at your IO heavy work that requires shared storage, we generally recommend that you copy (ideally tar’d) your data to/from rather than running directly from Batch but we are aware that this is not always possible.
By default, groups get a shared 1TB, there is currently limited expansion upon request all the way up to 10TB depending on the current overall utilisation.
Group Batch is NOT backed up, the amount of data stored is far too great to economically backup, so please be aware that data loss on this system is permanent.
Other Useful Storage
Your home contains a directory called www ( ~/www ), this is directly mapped on our publicly accessible webserver. Anything put into here will be available via https://www.ippp.dur.ac.uk/~$USERNAME/
Pre-Compiled HEP Software
We have a small repository of pre-compiled HEP Software for you to use at /mt/hepsw/
Batch was removed on the 15th March 2021. Please see “User Batch” and “Group Batch” above.
To find out your storage usage/quota on Home and User Batch please use the zquota command and you should see an output similar to below
login~$ zquota Quota Report for username Mount Point Used Total Last Checked /mt/home 14G (9%) 150G Mon 12 Apr 2021 12:22:06 BST /mt/user-batch 127M (0%) 1.0T Mon 12 Apr 2021 12:22:03 BST
The zquota command will now also display any group batch storage usage, this will show the entire group usage and quota and not your individual allocation.
We periodically scan your file storage allocation for viruses. We recieve a notification as soon as a virus is detected so we can apply the appropriate action, by default the system will not remove anything it suspects is a virus.
A log of any files suspected to be a virus can be found in your home space under ~/.clamscan or /mt/user-batch/$USERNAME/.clamscan.