ʯÁñÊÓƵ

RDIM Step 2 - Manage Organise Data Data Storage - Active or Working Data

Data Storage - Active or Working Data

Active data and working data are the same thing, read the definition of Active Data.

Appropriate data storage is a critical aspect of good research data management.

Many factors can lead to data loss or misuse with devastating consequences for your research and research career. Safeguarding against these should be a priority.

Researchers may need different storage and collaboration solutions at different stages of the Research (Data and Information) Asset Lifecycle. The options, also listed under ‘Active Storage and Collaboration Options’ are suitable for storing active (working) data, collaborating with other researchers, and/or creating backups.

Safe data is data stored in a “safe” data storage system:

  • The system operates with a low probability of failure
  • It is managed by ʯÁñÊÓƵ staff or an approved third-party provider
  • It is designed for mid to long-term data storage.

It is important to note that while you may consider the data on the system you are currently using for analysis as your primary or main data, these systems do not necessarily qualify as “safe” storage.


ʯÁñÊÓƵ approved storage options for safe active data include:

  • ʯÁñÊÓƵ Microsoft OneDrive | up to 5 TB – staff and students
  • ʯÁñÊÓƵ Microsoft Teams  | up to 25 TB per team - available to ʯÁñÊÓƵ staff on request
  • ʯÁñÊÓƵ | greater than 50 GB, up to many TB - access via ʯÁñÊÓƵ QRISCloud Cache or direct
  • Storage as dictated by Research Partner and/or Funding Agency
  • ʯÁñÊÓƵ research file shares - limited availability, based strictly on research need i.e. highly sensitive data

Note: Use Research Data ʯÁñÊÓƵ for completed data only | less than 100 MB - contact researchdata@jcu.edu.au for sensitive data or larger datasets.

Options that are suitable for backup storage include:

  • Shared university network drive (e.g. G, H etc.)
  • Desktop equipment (e.g. external drive/s, laptop, RAID systems, etc.)
  • External cloud storage/collaboration space (e.g., Dropbox, Google Drive).

Options that should NOT be used for storing research data include:

  • ʯÁñÊÓƵ High Performance Computing (HPC)
  • ʯÁñÊÓƵ research file shares - limited availability, based strictly on research need i.e temporary storage during analysis where the data needs to be local to the application for optimal performance (e.g. ArcGIS software and laptops)

While you may use many and varied different systems for data analysis during your research (e.g., ʯÁñÊÓƵ HPC, Metashape and Galaxy), these should only serve as temporary storage during the active analysis phase. They are not replacements for a long-term data storage solution. Never store your only copy of crucial data on these systems.

ʯÁñÊÓƵ HPC may have been used for long-term storage in the past; however, this practice is no longer recommended

For archiving completed data see Data Storage - Completed Data.

The basic rules for storing data and safeguarding against data loss are:


DO keep three copies in separate places i.e., on at least two different types of media (physical device or cloud) and in another location (physical location or cloud)
Ensure at least one copy is stored on a ʯÁñÊÓƵ approved option*


DON'T keep the only copy of your research data on a physical device e.g., hard drive (PC, laptop or external HDD) or USB key. These can easily be lost, damaged or fail.

The optimal combination of storage solutions will depend on your specific workflow, the volume and sensitivity of your data, and your preferences for file access and collaboration. For instance, you may prefer to work on your PC’s hard drive and synch to ʯÁñÊÓƵ Microsoft OneDrive if factors such as internet access, performance, or application compatibility are important. On the other hand, synching from OneDrive back to your PC (ensure you have sufficient space), facilitates collaboration and provides access to version history and a cloud backup if local storage fails. In practice, a combination of these approaches is likely to be helpful.

IMPORTANT: The following hypothetical research projects and storage options are provided for guidance only and are not prescriptive. To discuss a specific project and storage requirements in more detail please contact researchdata@jcu.edu.au

General

  1. ʯÁñÊÓƵ Microsoft OneDrive*
  2. Synchronised with hard drive on personal computer or laptop;
  3. Backed up to an external hard drive or cloud service

Field-work based:
no internet access

  1. Mobile device (tablet or laptop) for offline data collection in the field;
  2. Copied to an external hard drive to create local backups; and
  3. Synchronised with ʯÁñÊÓƵ Microsoft OneDrive* on return from the field

Computational analysis:

  1. Hard drive on personal computer or laptop for day-to-day work and analysis (ideally synchronised with ʯÁñÊÓƵ Microsoft OneDrive*)
  2. ʯÁñÊÓƵ HPRC for large-scale processing and simulations; and/or ʯÁñÊÓƵ QCIF Research Data Storage (QRISCloud)* for large datasets (>~50 GB) and collaboration; and
  3. External hard drive(s) for local backup and portability

Sensitive data:

  1. Dedicated ʯÁñÊÓƵ “R share” drive* for highly sensitive data and collaboration within the research team;
  2. ʯÁñÊÓƵ Microsoft OneDrive* for remote access and external collaboration via link (non-identifiable data)
  3. Encrypted external hard drive stored onsite for offline backup

Active Storage and Collaboration Options

ʯÁñÊÓƵ Microsoft OneDriveʯÁñÊÓƵ Microsoft Teams

Analysis phase  only

Storage Capacity 5 TB 25 TB per team Up to 5 TB and
 250, 000 files		> 50 GB up to many TB and 1,000,000 files
Collaborating with ʯÁñÊÓƵ researchers✔✔✔✔
Collaborating with external researchers✔✔1✔2✔
Sensitive Data✔✔✘✔
Remote Access✔✔✔✔
Data Stored in Australia✔✔✔✔
Deleted File Recovery✔3
via Service Now request 		✘✘✘
Suitable for Backups✔✔✘✔
Best Feature Quick to setup, easy to use, access from day one.Includes other methods for collaboration along with file sharing. Excellent for use with HPC compute for data analysis.Large-scale data storage
Getting SetupʯÁñÊÓƵ Microsoft OneDriveMake a collaboration tool request.

Notes

  1. Microsoft Team team owners (ʯÁñÊÓƵ staff) can add or .
  2. Researchers can apply (via ServiceNow) for external collaborators to have access to their ʯÁñÊÓƵ HPC account.
  3. Within 30 days.