Web Application Outages

Posted on February 19, 2011 by Brian Lesser
A number of Web applications are temporarily offline due to a critical hardware fault in a database server. The applications include the Ethics on-line system and other applications on runner.ryerson.ca. The hardware fault occurred in the late evening of Friday, February 18. We estimate the failed server will not be replaced until after 1 PM today (Saturday, February 19.)
[Update: The outage was resolved at 12 noon today(Feb. 19)]
Please accept our apologies for the problems this outage has caused.
-Computing and Communications Services
Posted in All Categories | Comments Off on Web Application Outages

Resource Management System Outage

Posted on February 17, 2011 by Brian Lesser
At approximately 5 a.m. this morning, a system disk failed in one of our Ryerson Resource Management System (RMS) servers. The disk failed during a backup which delayed the recovery process.
[Update: service was restored at 3 PM.]
We are currently working on restoring the systems that have been affected including:
  • Online Identity Activation: http://ryerson.ca/accounts/
  • Manage My Online Resources within Self Serve module in the my.ryerson portal.
  • RMS Administration
The following systems that make web services call to RMS are also impacted:
  • eHR is not able to do live updates to RMS
  • Research Information System: https://www.runner.ryerson.ca/ors/researchgrant/login/login.cfm
We apologize for the inconvenience this outage has caused.
– Computing and Communications Services
Posted in All Categories | Comments Off on Resource Management System Outage

Voice-Mail System Outage

Posted on January 25, 2011 by
At 3:45 pm this afternoon, Call Pilot, the system that provides voice mail and the Call Attendant service (call transfer) went down. As a result, there was no voice-mail during this period. The Call Attendant service was re-directed to the Voice-Enabled directory system. At 4:13 pm, Call Pilot was returned to service.

Bell has been called to investigate.

We apologize for the inconvenience.

– Computing and Communications Services

Posted in All Categories, phones | Tagged | Comments Off on Voice-Mail System Outage

RMAIL Delivery Slowdown

Posted on January 20, 2011 by

Due to a mass mailing from outside Ryerson, the RMail and Groupwise systems, as well as other departmental e-mail systems relying on spam and filtering services provided by CCS experienced a considerable slowdown in delivering mail yesterday (Jan. 19). The slowdown began at 6 PM and lasted until after midnight. One server experienced a longer delay until 5 AM.

Measures have been implemented to avoid this particular problem in the future, and we are looking into additional options to reduce the risk of similar re-occurrences.

– Computing and Communications Services

Posted in All Categories, GroupWise, RMail | Tagged , , | Comments Off on RMAIL Delivery Slowdown

Power Interruption in POD and Jorgenson buildings

Posted on January 14, 2011 by
Due to some electrical work being carried out in the POD building, there was a short power interruption around 10:40am in some areas of the POD and Jorgenson buildings that resulted in a brief loss of network connectivity. Data center equipment is protected by uninterruptible power supplies and services were not affected.
– Computing and Communications Services
 
 

 

Posted in All Categories, network | Tagged , | Comments Off on Power Interruption in POD and Jorgenson buildings

RMAIL Delivery Slowdown

Posted on January 14, 2011 by
Due to a mass mailing from outside Ryerson, the RMail and Groupwise systems, as well as other departmental e-mail systems relying on spam and filtering services provided by CCS experienced a considerable slowdown in delivering mail yesterday (Jan. 13). The slowdown began at 5 pm and lasted until roughly 11 pm. One server experienced a longer delay until 5:15 am.
We are looking into options to avoid service delays as a result of remote mass mailings.
– Computing and Communications Services
 

 

Posted in All Categories, GroupWise, RMail | Tagged , , | Comments Off on RMAIL Delivery Slowdown

Call Pilot Outage

Posted on January 5, 2011 by
At 10:30 this morning, Call Pilot, Ryerson’s Voice Mail system was out of service for 15 mins. During this time, the Auto-Attendant and all voice mail was unavailable.
Our initial examination of the system did not find any reason for the outage. Bell Service personnel have been called to investigate.
We apologize for any inconvenience the outage caused.

– Computing and Communications Services

Posted in All Categories, phones | Tagged | Comments Off on Call Pilot Outage

Emergency upgrade to perimeter firewall

Posted on December 16, 2010 by

The perimeter firewall is scheduled to be upgraded at 6:00 am, Friday, December 17. The upgrade is required to correct problems with the session counter and a possible memory leak. These problems may have caused the latency and jitter experienced during real time video and audio applications such as streaming, and video and audio conferencing.

The upgrade will take about 60 minutes. Sessions will be lost during this interruption. Internet access will be maintained as connectivity will fail-over to the back-up link.

We apologize for any inconvenience this upgrade may cause.

– Computing and Communications Services

Posted in All Categories, Internet | Tagged | Comments Off on Emergency upgrade to perimeter firewall

System outage on Dec. 28, 8:00 AM – 8:00 PM

Posted on December 16, 2010 by

On Tuesday, Dec. 28, 2010 between 8:00 AM and 8:00 PM, CCS is planning to upgrade 2 production SAN (storage) appliances to the latest version of the software, as well as applying a number of patches.

These 2 SAN (storage) appliances being upgraded work as a pair which serves storage to many servers. Each appliance will be failed over to the other while it is being upgraded and patched.

During the upgrade process we need to shutdown RMail and Blackboard. All other services should not be interrupted during the fail over. There is still a small chance that a lock up can occur with the servers, which may result in all the storage becoming unavailable to the clients. This risk is low and we have been preparing the environments thoroughly to reduce this risk.

List of services that may be affected/unavailable include:
– RMail *** Unavailable
– my.ryerson portal *** Unavailable
– eHR *** Unavailable
– RAMSS *** Unavailable
– Library eResources *** Unavailable
– all other applications accessed through the my.ryerson portal *** Unavailable
– RMS
– Alfresco
– Virtual Apps (Citrix)

We apologize for any inconvenience this outage may cause.

– Computing and Communications Services

Posted in All Categories, eHR, eResources, my.ryerson, RAMSS, RMail | Tagged , , , | Comments Off on System outage on Dec. 28, 8:00 AM – 8:00 PM

Major systems restored after a switch failure at 9:30 AM

Posted on December 8, 2010 by

As of 10:30 AM this morning, the my.ryerson portal, RAMSS, eHR, Library eResource and other applications were returned to service following the outage that began at 9:30 AM.

We apologize for the problems this outage has caused.

-Computing and Communications Services

Posted in All Categories, eHR, eResources, my.ryerson, RAMSS | Tagged , , | Comments Off on Major systems restored after a switch failure at 9:30 AM