Inside Penn when tech broke | Philly Health Insider

Summer is traditionally a slow time for news, but the latest reports just keep heating up. (And we are not even talking about national politics.)

This week, we take you inside Penn Medicine as it navigated a global tech outage that forced mass cancellations of surgeries, procedures, and routine appointments, and left doctors documenting emergency care with pen and paper. We also highlight a spate of news: Crozer Health’s for-profit owner asks for a bailout, Redeemer Health’s credit rating got downgraded by S&P, and the Wistar Institute will open a new HIV research center.

And we dive into the latest U.S. News & World Report hospital rankings, and why one local health system isn’t going to plaster its honors on billboards.

That’s just the top local health headlines. Let’s get into it.

📮If you work at Penn or Main Line Health, tell us what your day was like when CrowdStrike malfunctioned. Did you work at all? Have you been able to reschedule patients’ appointments?

If someone forwarded you this newsletter, sign up here.

— Aubrey Whelan and Abraham Gutman, Inquirer health reporters, @abrahamgutman and @aubreywhelan.

The emergency calls to Penn Medicine executives began shortly after 2 a.m. on Friday morning.

Staff were trying to log on to their computers, only to be met with a bright blue screen with a few lines of text — the “Blue Screen of Death,” as it’s known in the tech world. Something had rendered nearly all of the 50,000 computers at Penn inoperable.

“We first thought it was ransomware,” said Penn Medicine CEO Kevin Mahoney.

But within a few hours, it became clear that the problem had originated from the cybersecurity company that Penn relies on to protect tens of thousands of servers and PCs.

Penn was one of the most impacted hospital systems in the Philadelphia area. The reason, Penn CEO Mahoney said, was that the system went all in on cybersecurity — and that meant going all in on CrowdStrike, an industry leading brand.

The hospital’s first patients with prescheduled appointments were set to arrive around 5:30 a.m.

Penn staff prepared paper records for physicians to document cases and worked to determine whether the imaging machines that treat “the true emergencies” — strokes, heart attacks, and other health conditions where time is of the essence — could still function.

Penn first canceled any surgery it could cancel for the morning. Then for the rest of the day. By midday, all Penn staff had to offer some patients was to jot down their name and contacts, promising to reschedule them next week.

Providers worked hard to make sure emergencies were treated and critical procedures still happened, such as bone marrow transplants and surgically fixing a fractured limb with exposed bone.

By evening, Penn IT staff were bringing systems back online. And Penn has since extended evening and weekend hours at some clinics, vowing to get most canceled appointments rescheduled within two weeks.

Read more to see the next steps that Penn leaders are considering to prevent a similar incident from ever again happening.

This week’s number: 2.

That’s how many Philadelphia-area hospitals made it to the latest U.S. News & World Report honor roll of the nation’s 20 best hospitals.

West Philadelphia’s Hospital of the University of Pennsylvania and Penn Presbyterian Medical Center, ranked as a combined entity, also placed nationally for multiple specialties, including cancer (10th), ear, nose, and throat (7th), and neurology and neurosurgery (8th).

Thomas Jefferson University Hospital in Center City also received national honors for a few specialties behind Penn.

Wills Eye Hospital, which is affiliated with Jefferson Health, ranked 2nd out of 10 in ophthalmology.

Read more to learn why Penn won’t be bragging about its honors on a billboard — and how other Philly-area hospitals fared in the rankings.

Each week, we highlight state inspections at the various hospitals in our region. Up this week: Mercy Fitzgerald Hospital, which is part of Trinity Health Mid-Atlantic. State inspectors visited the Delco hospital three times between November 2023 and April, conducting a mental health survey and investigating an already corrected issue. On the April visit, inspectors cited the hospital for failing to have doctors who supervise trainees sign off on patient charts.

Read more on what the inspectors found.

The Mazzoni Center, Philadelphia’s largest LGBTQ health-care agency, has taken wide-ranging steps in the past year to shore up its troubled finances without compromising the quality of patient care, CEO Sultan Shakir says.

“There is a way to get health centers into the black by just cramming patients in as quickly as you can,” Shakir said. “And we’re clear that that’s not the way we want to do it.”

Read on about what steps Mazzoni took to get back in the black.

Domenick Bucci, vice president of medical management and medical policy at Independence Blue Cross, is the new president of the Philadelphia County Medical Society.

He replaces John Vasudevan, co-director of the Penn Medicine running and endurance sports program, who completed his tenure this summer.

Wistar Institute is investing $24 million to open a center dedicated to researching how to equip the immune system to fight HIV by developing new treatment options, including potentially a vaccine.

“An AIDS vaccine has been very elusive, which means we probably don’t know everything there is to know about the disease,” said Dario Altieri, Wistar’s CEO. “The time is now to really bring together everything that has been achieved so far and make that additional leap.”

The new HIV Cure and Viral Diseases Center will be located at 3675 Market St.

Whew, that’s all for this week. Tell us if we’re missing any other news, and we’ll try to include it in our next edition of Philly Health Insider. For a chance to be featured in this newsletter, email us back.

By submitting your written, visual, and/or audio contributions, you agree to The Inquirer’s Terms of Use, including the grant of rights in Section 10.