- Current System Load
- Known Issues
- Current Issues
- Recent Issues
- Maintenance
- Service Calendar and Maintenance
Current System Load
The plot below shows the status of the CPU nodes on the current Cirrus service for the past day (note: the Cirrus GPU nodes are not included in this plot).
A description of each of the status types is provided below the plot.
CPU
- alloc: Nodes running user jobs
- idle: Nodes available for user jobs
- resv: Nodes in reservation and not available for standard user jobs
- down, drain, maint, drng, comp: Nodes unavailable for user jobs
- mix: Nodes in multiple states
GPU
- alloc: Nodes running user jobs
- idle: Nodes available for user jobs
- resv: Nodes in reservation and not available for standard user jobs
- down, drain, maint, drng, comp: Nodes unavailable for user jobs
- mix: Nodes in multiple states
Known Issues
We are experiening a heavy load on the metadata server. Our systems team are investigating but we suspect this is due to user(s) performing many I/O operations. We apologise for the inconvenience this is causing users.
Service Alerts
| Status | Type | Start | End | Scope | User Impact | Reason |
|---|---|---|---|---|---|---|
| Ongoing | Service Alert | 2024-04-22 07:00 | 2024-04-26 17:00 | Full Cirrus service - No service available | Users will be unable to connect to Cirrus and will not be able to access their data. Jobs will not run. | Work at the Advanced Computing Facility (ACF) where Cirrus is located is taking place and the power will be removed from Cirrus. |
Recently Resolved Service Alerts
This table lists resolved service alerts from the past 30 days. A full list of historical resolved service alerts is available.
| Status | Type | Start | End | Scope | User Impact | Reason |
|---|---|---|---|---|---|---|
| Resolved | Service Alert | 2024-04-17 10:30 | 2024-04-18 09:00 | RPOOL, Solid state file system | There should be no user imapct due to resilency of the system | Swapping and commissioning of one of the disks within the solid state file system |
| Resolved | Service Alert | 2024-04-15 10:00 | 2024-04-15 10:30 | Outage to DNS server which will impact Cirrus and Cirrus SAFE | Users can still connect to service but may be unable to access external websites (eg GitLab) | Migration of server in preparation of the wider power work affecting site the following week |
| Resolved | Service Alert | 2024-04-11 10:00 | 2024-04-11 10:40 | Cirrus ticketing server | May be a delay in processing new user requests via SAFE | Migration of the rundeck ticketing system |
| Resolved | Service Alert | 2024-03-24 23:00 | 2024-03-25 11:00 | CPU compute nodes | Around half of the CPU compute nodes on Cirrus are unavailable. Jobs running on affected compute nodes will have failed. | A power incident caused CPU compute nodes to fail. |
Service Calendar and Maintenance
This section lists recent and upcoming maintenance sessions. A full list of past maintenance sessions is available.
No scheduled or recent maintenance sessions
Maintenance Logs for previous periods
At Risk Maintenance Sessions
There is an ‘At-Risk’ Session provisionally booked every Wednesday from 1000 - 1200. A user mailing will be sent if any work is going to take place which may impact users.
Service Calendar
We maintain a calendar for the Cirrus service that lists upcoming events (such as training courses and maintenance sessions):
We keep maintenance downtime to a minimum on the service but do occaisionally need to perform essential work on the system. Maintenance sessions are used to ensure that:
- software versions are kept up to date;
- firmware levels on HPE and third-party peripheral equipment are kept up to date; essential security patches are applied;
- failed/suspect hardware can be replaced;
- new software can be installed; periodic essential maintenance on HPE electrical and mechanical support equipment (refrigeration systems, air blowers and power distribution units) can be undertaken safely.
Additional maintenance sessions can be scheduled for major hardware or software updates; major upgrades to facility plant and infrastructure; acceptance testing following major service upgrades and statutory electrical testing.