On July 12th, it was discovered that the data machine nodes are not properly responding to diagnostic commands. However, these nodes are still available and scheduling jobs.

For example commands like

checknode nal-004

or

scontrol show node nal-004

will show output like

Node nal-004 not found

At the moment, this appears to be limited to nodes in the data machine. We are investigating and will update with any resolution or further issues.

UPDATE: (Monday, July 15, 2:30PM) This issue still persists, and ICER sysadmins are continuing to diagnose. We believe that this may also affect some buy-in nodes outside of the Data Machine. As a workaround, users can use the -a option with scontrol, like:

scontrol show node nal-004 -a