Storage Migrations in 2026

ALL users of ITS-RC services will be impacted by these changes

Research Computing has acquired a new, fast, and very capable storage system. The storage capacity brought in is additive since we are not decommissioning the old storage system -- this is incredibly important as demand for research data grows faster than we can accommodate. However, the "old storage system" (currently serving /proj and /users) is indeed old and under limited vendor maintenance; we MUST transition to the new storage system ASAP so we can reconfigure the old one to a more appropriate purpose given its physical and fiscal characteristics.

New VAST

In March 2026 a new primary storage system was deployed based on VAST Data and Cisco hardware. This will be the primary "hottest tier" of data for the foreseeable future. It is the fastest most capable storage system ever deployed by ITS-RC. Data on this platform is mounted as /hickory/proj, /hickory/users, and /vast/som

Isilon

The storage system underlying /proj and /users will be re-purposed into nearline storage once the data migrations described here are complete. This is a Dell PowerScale (previously Isilon) storage cluster that is primarily spinning disk (vs SSD). It will be unmounted from the clusters to prevent computing directly against data stored here; users will shuttle data back and forth between primary (VAST) and nearline (Isilon) storage using data mover nodes. Nearline storage is a great place to hold onto high volumes of colder unutilized data, and as a place to organize, curate, package and prepare data for cloud archive.

Old VAST

The storage system underlying /work also happens to be from VAST Data, however it is a few years older and much smaller. We must vacate all /work folders. This platform will be repurposed into a "proper /scratch file system," where files older than a certain date are automatically deleted. This is a very convenient option for intermediate files, as one does not need to worry about cleaning up after a run -- simply copy out data that needs to be retained, or write it directly to the new VAST, i.e., workflows that write intermediate files to /work, and files that require durability to /hickory.

Need for appropriate match between research data and storage tier

Storage systems vary significantly in performance, capability and price. It is critical for the Carolina research community to improve processes and efforts resulting in research data being stored on the tier that is appropriate for its current use cases. Data being actively computed upon should be on the most capable system (/hickory). Nearline storage provides a lot of capacity for data that is best kept on campus for now but is not actively being computed upon.

Note on individually scoped vs group scoped data

As researchers require more storage capacity, we wish to deploy that capacity into a shared/collaborative location whenever possible as opposed to increasing an individual user's capacity. We are more inclined to increase quotas on /hickory/proj, /hickory/groups, or /som folders than an individual's /users folder.

In regard to data sharing, we wish for users to NOT share their individually scoped data folders (e.g., legacy /users, legacy /work, or new /hickory/users). We prefer to establish new shared storage locations to supports collaborative activities.

Steps for migrating an individual user to the new storage architecture

Step 1:
- The user contacts Research Computing (RC) and agrees to migrate their data to the new system; an RC team member is assigned as primary contact for the migration
- The user does a review and clean-up of their existing /users/o/n/onyen and /work/users/o/n/onyen folders, identifying and removing unnecessary data
- The user determines there is no sharing of data from their /users and /work folders (e.g., group ownership of a directory by a PI group). If there is sharing, the user works with RC to discuss creation of a separate, shared folder under /hickory/proj (or /vast/som if related to the School of Medicine) for data sharing
Step 2: The user agrees to NOT modify or add any data in their /users or /work folders nor submit Slurm jobs to the cluster until further notice. Access permissions on their legacy /users and /work directories are removed (by changing top-level ownership to "root," and dropping read, write and execute permissions for "group" and "other" (i.e., drwx------).
Step 3: RC copies all of the user's /users/o/n/onyen data to /hickory/users/o/n/onyen/from_users For most user directories set with the default quotas, this is typically accomplished within 24 hours.
Step 4: RC copies all of the user's /work/users/o/n/onyen data to /hickory/users/o/n/onyen/from_work For most user directories set with the default quotas, this is typically completed within 24 hours.
Step 5: RC notifies the user that migration is complete; the user validates and confirms or disagrees. Repeat this process until both parties are in agreement
Step 6: The user resumes work by modifying all of their scripts, worklflows and jobs previously using /users and /work to instead use /hickory/users (or a shared space in /hickory/proj, /vast/som or other new VAST-based mount if appropriate)
Step 7: RC removes all data in /users/o/n/onyen and /work/users/o/n/onyen and removes those folders

End Of Workflow

Steps for migrating /proj folders and other collaborative storage

All /proj folders will be migrated to one of two new spaces: 1) /hickory/proj or 2) /vast/som

Using pilab as an example, /hickory/proj/pilab will replace /proj/pilab; for PI's primarily affiliated with the School of Medicine, /vast/som/pilab will replace /proj/pilab

The high-level process is:

PI and/or their project team connects with Research Computing (RC) and agrees to migrate their data to the new system; an RC team member is assigned as primary contact for the migration
Preparatory work is done by the PI and their project team (see below)
Research Computing (RC) begins copying existing data while project team continues work. The time required for data copying is dependent on size of the data, complexity of the space's directory hierarchy, file counts, etc.
RC and the project team decide when a lock-out * period begins; no changes can be made to existing data in the /proj source folder until further notice
RC conducts a final sync of data
The project team validates copy of data
Upon acceptance, the project team retargets all work at new storage location; RC removes all access to data on the old system (/proj)directories (by changing top-level ownership to "root," and dropping read, write and execute permissions for "group" and "other" (i.e., drwx------)
RC will leave the old data in place for no less than 30 days as a disaster recovery option. Data is then deleted from the old system after 30 days

There must be a lockout period for the final sync of data during which time there can be no changes to the data in the /proj folder. Preparation work by the PI and project team can greatly improve the migration experience and reduce the necessary lockout period.

Note: Large /proj folders can take a significant amount of time to sync from the old to new storage system. File counts are often more impactful than sheer volume of data. Using a Library analogy: a library with 1 billion pamphlets takes far longer to inventory than a library with 1 million large volumes, even if both consume the same amount of shelf space. The issue isn't how much space the books occupy on the shelves, but counting every book.

PI and their project team's preparatory work for migration

Step 1: Delete any/all unneeded data
Step 2: Decide if you will migrate the entire /proj folder as-is, or break it into subunits and execute the migration steps on each of the subunits within your /proj folder.
Step 3: Look for areas with "too many files"; if there is a large number of relatively small files and your workflows are not negatively impacted, use tar or tar with gzip to reduce file counts. Ask for assistance if unsure.