CAS/my.ryerson Service Disruption: Tuesday, Jan. 15 2013 5:00 PM

One of the my.ryerson.ca application servers was experiencing a problem due to high utilization and was affecting some users access to the my.ryerson.ca portal.  The server has been restarted and is working normally.

There is also an intermittent problems with logging into the my.ryerson.ca portal caused by our Central Authentication System (CAS) having problems connecting to our LDAP directory service. Staff are investigating the problem and hope to have full service restored soon. This note will be updated as more information becomes available.

Update at 6:24 PM: LDAP services have been restarted and CAS is now working normally.

Update at 9:58 PM: This problem recurred at 9:29 PM and was again cleared by 9:43 PM by restarting services.

 

Please accept our apologies for the disruption in service.

-Computing and Communications Services

Posted in All Categories | Comments Off on CAS/my.ryerson Service Disruption: Tuesday, Jan. 15 2013 5:00 PM

Ryerson Mobile Service Disruption: Tuesday, Jan. 15 2013 2:50 PM

Just before 3:00PM, one of the RAMSS front end servers crashed and was rebooted.   However, one of the services running on this server, an integration broker that provides  student schedule information to Ryerson Mobile failed to come up properly.   Unfortunatately, this particular service is not currently load balanced and is still unavailable.   We are in the process of restoring the service.

Other RAMSS services are not affected by this issue.

Update at 6:40 PM: The mobile services have been restarted and are now working normally.

We apologize for any inconvenience.

-Computing and Communications Services

Posted in eResources, RAMSS | Comments Off on Ryerson Mobile Service Disruption: Tuesday, Jan. 15 2013 2:50 PM

Scheduled www.ryerson.ca and the my.ryerson portal outage: Friday, January 18, between 6 am and 7 am

To mitigate issues that caused my.ryerson and CAS services interruptions on Dec 26, 2012 and Jan 1, 2013, CCS will be moving www.ryerson.ca and uPortal to a new load balancer on Friday, January 18, between 6 AM and 7 AM. www.ryerson.ca, the departmental web sites hosted on the CQ Content Management System, the my.ryerson portal and all applications accessed through the my.ryerson portal will not be available during this time frame.

We apologize for any inconvenience.

-Computing and Communications Services

Posted in All Categories, Blackboard, eHR, Google Apps, my.ryerson, RAMSS | Tagged , , , , | Comments Off on Scheduled www.ryerson.ca and the my.ryerson portal outage: Friday, January 18, between 6 am and 7 am

HR service outage: Sunday, Jan. 6 2013 17:08PM – Monday, Jan 7 2013 8:46AM

Due to maintenance work being carried out on a server chassis, one of the HR batch server file systems went into read only mode on Sunday, Jan 7 at approximately 5:08 PM.   This caused  some automated HR jobs to fail overnight.

Also, sometime yesterday, the HR web servers run into a limit of the number of files than can be opened at the same time, causing the system to be unresponsive.  The open file limit has been increased and services restored as of 7:46 AM this morning.

We apologize for the inconvenience.

-Computing and Communications Services

Posted in eHR | Comments Off on HR service outage: Sunday, Jan. 6 2013 17:08PM – Monday, Jan 7 2013 8:46AM

my.ryerson and CAS, uPortal, RAMSS service interruption: Jan. 1 8:00 AM – Jan. 1 9:25 AM

At approximately 8:00 AM this morning (Jan 1, 2013) the CAS authentication service became unresponsive. uPortal and RAMSS were also affected. We believe the problem is likely the same one that affected CAS on December 26 – a problem with the SSL accelerators:

http://servicealerts.blog.ryerson.ca/2012/12/26/my-ryerson-and-cas-service-interruption-dec-26-811-am/

The CAS web servers have been restarted at 8:40 AM but not all services are back to normal.

While CCS did take steps following the Dec 26 problem that seemed to work during the Dec 28 and 30 enrolment periods we are investigating why the problem recurred this morning.

Please accept our apologies for the frustration and inconvenience this morning’s outage caused.

-Computing and Communications Services

Posted in All Categories | 15 Comments

my.ryerson and CAS service interruption: Dec. 26 8:11 AM – Dec. 26 9:08 AM

At 8:11 AM this morning (Dec 26, 2012) both the my.ryerson.ca portal and the CAS authentication service became unresponsive. CCS staff are investigating and we will update this alert when more information becomes available.

Update 9:08 AM: Service has been restored by restarting services for both the my.ryerson.ca portal and CAS authentication system. We are investigating the cause of the outage and can see that the service is still slow for some people.

Update 8:21 PM: As a preventative measure uPortal and CAS will be restarted at 6 AM on Dec. 28 and 30 and on Jan. 1 and 3. As a result there will be a brief outage of a few minutes after 6 AM on those mornings.

Update Dec. 27, 2012 at 12:37 PM: Preliminary investigations have shown that an SSL accelerator may have been overwhelmed by requests beginning at roughly 8:03 AM on Dec 26. A second SSL accelerator has been added in order to increase capacity. Additional CCS staff will monitor systems performance during the opening of the next RAMSS enrolment period.

Note: SSL (Secure Sockets Layer) is a protocol designed to securely encrypt information in transit on the Internet. The SSL accelerator is a network device that provides high-speed encryption/decryption services. It improves application performance by relieving servers of complex public key and bulk encryption tasks.

Please accept our apologies for this outage.

-Computing and Communications Services

Posted in All Categories | 7 Comments

CFAPS print service disruption: Friday, Dec. 21 2012 11:28 AM – 13:40 PM

A corrupt HP print driver caused some users of the Central File and Print Sharing(CFAPS) service to experience printing problems between 11:28 AM – 13:40 PM today, Friday, Dec. 21.   The print server was restarted and normal services restored at 13:40 PM.

We apologize for the inconvenience.

-Computing and Communications Services

Posted in All Categories | Comments Off on CFAPS print service disruption: Friday, Dec. 21 2012 11:28 AM – 13:40 PM

Scheduled RAMSS outage, December 24, 2012 from 5:30 am to 12 noon

Due to a hydro power upgrade project that must be completed at Ryerson, RAMSS will be unavailable on December 24, 2012 from 5:30 am to 12 noon. This is unavoidable and we apologize in advance for the inconvenience that this may cause to some students.

Note: In case that the work cannot be performed on December 24, it will be rescheduled for December 27 from 1 am to 7 am.

UPDATE (Dec. 24, noon): The electrical work is still ongoing and we expect full services restored around 2 PM.

-Computing and Communications Services

Posted in All Categories, RAMSS | Tagged | Comments Off on Scheduled RAMSS outage, December 24, 2012 from 5:30 am to 12 noon

Mailing list (listserv) server outage: Wed., Dec. 19, 2012 2:00 PM – 3:15 PM

The mail server (listserv) stopped responding this afternoon around 2:00 PM due to a loose network cable.     Service was restored around 3:15 PM. Messages submitted during this period were queued up on the RMail system and started being processed after the service was restored.

We apologize for the inconvenience.

-Computing and Communications Services

Posted in RMail | Comments Off on Mailing list (listserv) server outage: Wed., Dec. 19, 2012 2:00 PM – 3:15 PM

CAS service interruption: Dec. 16 11:7 PM – Dec 17 2:12 AM

Central Authentication Service (CAS) application server’s threads had reached the max and could not handle new requests at 11:57 PM on December 16, 2012. Services were restored at 2:12 AM on December 17, 2012.

This affected all CASified applications, including my.ryerson portal and all applications accessed through the portal, Google Applications, Confluence and Jira.

We apologize for the inconvenience.

-Computing and Communications Services

Posted in All Categories, Blackboard, eHR, Google Apps, my.ryerson, RAMSS | Tagged , , , , | Comments Off on CAS service interruption: Dec. 16 11:7 PM – Dec 17 2:12 AM