HPCC Service Status

Network Mounts Restarting on AMD24

Fri, 10 Jul 2026 12:00:00 +0000

Due to recent security updates, we will be remounting all file systems on AMD24 nodes immediately. Jobs reading or writing data during this window may be interrupted. We can work with you for priority boosts if your running jobs are time dependent. Sorry for any inconvenience. Please contact us if you have any questions

Tuning FairShare Priority Calculation

Wed, 24 Jun 2026 12:00:00 +0000

We are currently in the process of tuning how cluster usage impacts users’ FairShare priority factor. Historically, the FairShare priority factor provided a boost to job priority for users with low recent cluster utilization, but droped off sharply after a small amount of cluster resources were consumed. This tuning aims to increase the amount of cluster resources users’ may consume before their FairShare priority drops off.

You may notice changes to job wait times during this tuning. If you have any questions about this change, please contact us. For more information about FairShare and other factors that affect job priority, please refer to our documentation.

Major OnDemand Portal Update

Fri, 12 Jun 2026 08:00:00 +0000

ICER’s OnDemand portal has been updated to version 4.2! This update introduces several new features, including:

If you have any questions about this update or encounter any issues, please contact us.

Resolved: OnDemand Portal Update on Thursday 6/11

Wed, 10 Jun 2026 08:00:00 +0000

Resolved 6/12 11:08am: The GPFS filesystem is now stable.

Update 6/12 9:36am: The new OnDemand server is experiencing intermittent issues mounting the GPFS filesystem. We are currently working to resolve the issue.

Update 6/12 9:00am: The new OnDemand server is now available. Some users may still get directed to the old server due to caching of the server address. Users will get directed to the new server as their caches expire.

At 8:00PM on Thursday, June 11th, ICER’s OnDemand portal will undergo a major version update to version 4.2. This update introduces several new features, including:

If you have any questions about this update or encounter any issues with the OnDemand portal following the update, please contact us.

Rebooting dev-amd20-v100 and dev-amd24-h200 at 8:00PM on Tuesday, June 2nd

Tue, 02 Jun 2026 10:00:00 +0000

RESOLVED 8:37PM: Both dev-amd20-v100 and dev-amd24-h200 are back online

Both dev-amd20-v100 and dev-amd24-h200 will be rebooted at 8:00PM on Tuesday, June 2nd. These reboots are being performed to update the Nvidia and Cuda drivers. If you have any questions or encounter any issues following the reboots, please contact us.

RESOLVED: Research Space NFS and SMB Service Maintenance

Thu, 14 May 2026 12:30:00 +0000

RESOLVED: 5/26/2026 - SMB and NFS service maintenance is complete.

On Friday, May 15, 2026, we will be upgrading the services that provide NFS and SMB access to research space directories on the HPCC. No downtime is expected during these upgrades, but periodic pauses or slowdowns with NFS or SMB file transfers may be noticed during the upgrade process. We will update this blog with more information as soon as the upgrades are complete.

Help desk response times slowed

Wed, 13 May 2026 16:00:00 +0000

Due to short-term staffing issues we anticipate that help desk response times will be somewhat longer than usual. We strive for an initial response to all queries within one business day but anticipate that a more realistic response time in the near future will be 2-3 business days. We will prioritize urgent requests to the greatest extent possible. We apologize for any inconvenience this may cause, and thank you in advance for your patience.

High memory capacity nodes back online

Wed, 13 May 2026 12:00:00 +0000

The high memory capacity nodes, vim-000, vim-001 and vim-002 are back online to support specific research computing needs. They support either 2.93 TB (vim-000, 001) or 5.86 TB (vim-002) of random access memory (see our documentation on cluster resources for details). These nodes are available via a separate partition and are not in the general queue. Please see our documentation on SLURM constraints for usage.

Increase in Memory Reserved for System Processes

Fri, 08 May 2026 15:00:00 +0000

Following the recent downtime, the amount of memory available for jobs on nodes has been tuned to increase system stability. On all nodes except intel18, the amount of memory available to jobs has been reduced by 5GB. A full list of node types and the memory available to jobs can be found in our documentation. This adjustment will help prevent unexpected lockups and reboots of nodes during times of high memory utilization.

Jobs scripts that request the maximum amount of memory available on a particular node type may need to be adjusted following this change. If you have any questions about this change, please contact us.

RESOLVED: HPCC Scheduled Maintenance

Thu, 07 May 2026 15:00:00 +0000

The HPCC will be unavailable on Thursday, May 7 starting at 5AM for regularly scheduled maintenance. No jobs will run during this time.

Jobs that will not be completed before this date will not begin until after maintenance is complete. For example, if you submit a four day job three days before the maintenance outage, your job will be postponed and will not begin to run until after maintenance is completed.

If you have any questions, please contact us

Update 5/11. The maintenance has been completed.