The president and CEO of BC Ferries issued a public apology after the company’s website, app and phone system crashed for several hours on the final day of the Victoria Day long weekend.
Nicolas Jimenez said BC Ferries is “deeply sorry” for the outage, which started Monday morning (May 26) and lasted well into the afternoon. The outage meant that travellers could not find out whether their sailings were running on schedule.
“I hope that people understand how deeply people here, the 5,100 employees, feel when these incidents occur,” he said. “It’s absolutely crushing for us when something like this happens, because we know how important our business is to people moving around the province to go where they need to go. We are working really hard to make sure these things don’t happen and when they do, we bring our A-game to making sure that we recovering as fast as possible.”
Jimenez said the origins of the crash lie at a data centre in Kamloops, where one of the applications was feeding extra data into the system, affecting the other applications. “It caused a failure and all those systems went down,” he said.
He added that teams immediately started to identify, isolate and address the problems in trying to restore each individual system.
“Because there were so many systems that were impacted, it took some time to get there, but once, they did, by mid-afternoon we obviously recovered and the business was up and running,” Jimenez said. “We know it impacted a lot of people and we are really sorry for any inconvenience that this caused.”
Jimenez said it is not clear what caused the additional data to be fed into the system. “Our technical team is still trying to understand that,” he said.
He said staff are currently working on ways to prevent something like this from happening again. “We have already put into place today measures to add capacity, so that this issue won’t happen again, but we need to step back and look at our (IT) architecture more broadly to ensure that it is built to purpose.”
It is not clear how many people the outage impacted, but it came during one of the busiest travelling days in the year.
Jimenez acknowledged that the timing of the outage was awful.
“Yes, (this) happening on a long weekend is not a good thing,” he said. “More people are using your service, but let’s be clear — if this happened on a Wednesday afternoon in February, for anyone trying to use our service, it would be equally problematic…I want our systems to be as reliable as possible because we are a 365-days-a-year business.”
Monday’s outage combined with other issues over the weekend. They include Saturday’s complete cancellation of late afternoon and evening service from and to Bowen Island because of staffing shortages, leaving many people stranded, even as the company had hired water taxis to help those who urgently needed to leave.
On Sunday, the line of foot passengers looking to buy tickets at the Swartz Bay Terminal near Victoria snaked well outside the terminal.
Deborah Marshall, executive director, public affairs for BC Ferries, said the system reached passenger capacity on the noon, 3 pm and 4 pm sailings at Swartz Bay yesterday.
“When this happens we hold sales for foot passengers for a short period of time until the vessel is cleared,” she said in commenting on a photo circulating on social media. “So, while there was a long queue in the photo, we had between two and four ticket agents working at any given time.”
Monday’s outage coupled with Saturday’s cancellations and Sunday’s long line up offered critics of BC Ferries plenty of evidence with BC United MLA Jordan Sturdy pointing fingers at the provincial government generally and Jimenez specifically.
“Looks like NDP interference in BC Ferries from replacing the Chair to imposing a new CEO is having predictable results,” he wrote on Twitter.
Jimenez assumed his current role in early March 2023 after having served as president and CEO of ICBC for five years.
BC Ferries Board Chair Joy MacPhail — a long-time figure in provincial politics with deep ties to the governing New Democratic Party — helped oversee Jimenez’s appointment.
Jimenez declined to comment on Sturdy’s post. “That’s a political comment and my job is not political,” he said. “My job is to run this business, so that it serves the interests of the 20-million-plus people who use it … that’s what I am focused on.”
Jimenez also indirectly addressed criticisms that recent events appear symptomatic of BC Ferries’ state. “We designed the system to work in our peak moments ,” he said. “If issues are going to happen, they are going to happen because there is something that we haven’t planned for. You can’t prevent every issue from occuring, certainly not the ones that you know about. But you design systems to ensure that they are robust and they can withstand the stress that is put on them during difficult and busy periods.”
He added that BC Ferries had 95 additional sailings over the long week, while having hired more people in the last 12 months than at any time during the company’s history.
“We have done all these different things to increase staffing resilience in this company, knowing that more people are going to be travelling today than they were last summer, even pre-pandemic … I can tell you that with the one exception that we had on our Bowen Island route, we were running 400 sailings a day over the long weekend and moved more than 400,000 people over the long weekend,” he said.
“The system, with this being the exception, ran as we designed it. So going forward into the next long weekends and really every weekend over the summer, we are going to keep bringing the same kind of discipline to every single day, which is to make sure sailings go out on time and they arrive safely,” he added.