At ClosedLoop, AI Does More Than Innovate — It Saves Lives

In the beginning of 2020, Carol McCall was in the thick of working on ClosedLoop’s first submission round for the Centers for Medicare & Medicaid Services’ (CMS) AI Health Outcomes Challenge. The goal? To stand out as a small Austin-based startup among more than 300 other healthcare contestants — including big players like IBM and the Mayo Clinic — by building an AI-powered platform that could predict different endpoints and adverse events among every single beneficiary in Medicare.

And then Covid-19 hit.

“It was a multi-year, multi-stage contest that started in 2019, and right in the middle of getting ready to submit the first round, the pandemic began,” the chief health analytics officer said. “So everybody just paused, and we had to power down.”

As McCall was packing her work things to head home and shelter in place, leadership pitched an idea: “Given all the things we’ve been building for CMS, don’t you think we could create something that would be useful for Covid-19?” 

McCall was immediately on board.

“We built what we call the Covid Vulnerability Index,” she said. “It didn’t predict whether or not you would get Covid-19, but it could predict if you were a person who was likely to get really sick from it — and rank that possibility low to high through machine learning. Nobody had data on Covid-19 yet, but we knew what it looked like when people got serious pneumonia and influenza, and what put them at risk for other respiratory issues.”

After launching the index, ClosedLoop then open-sourced it for any specialist who wanted to pour in their data and provided trainings for the platform through webinars while working from home. “At the end of it all, organizations that serve more than 10 million people poured their data into the top of our model and got answers out,” McCall said. “I was really proud of that.”

For teams at ClosedLoop, this was more than an opportunity to be first in an unknown global landscape — it was about optimizing healthcare by utilizing AI and data science to create a clearer future for every patient. And for leaders like McCall, it was a chance to honor the organization’s mission and showcase that the “A” in AI doesn’t stand for “artificial” in her eyes, but “augmented.” By pairing the capabilities of physicians and clinicians with machine learning, highly relevant data can flow like water from a tap right when it’s needed to help make better decisions.

In April 2021, their efforts had paid off: ClosedLoop won the largest healthcare-focused AI challenge in history and received a $1 million prize for their efforts. Not only was the victory proof that a small startup with massive goals could rival some of the biggest names in healthtech, but it solidified that their team collaboration and tight-knit culture was a force to be reckoned with.

Built In sat down with McCall, AI Delivery Manager Sasha Savinovich and Director of Product Marketing Andro Hsu to learn more about how their teams are shaping the future possibilities of healthcare — and how a shared cultural vision allows them to accomplish their best work.

When did you realize what a difference ClosedLoop’s products can make for others?

Carol McCall, Chief Health Analytics Officer: We had an experience a few years ago with a teen who we predicted needed attention, and at the time there were multiple models being run by outside sources to analyze potential anomalies, including our own. After digging further, we learned that this was a person with a history of suicidal ideation who had been recommended and declined for CPS and other services, and the other models had just blown past it. Because of what we were able to analyze and find through our software, people were able to get involved and intervene. So that’s a story where on that day we can point to the models making a difference.

Andro Hsu, Director of Product Marketing: I actually just talked to one of our clients who learned of us through the Covid-19 index, and they’ve looked at the data and found that the rate of hospitalizations went down by 50 percent between January 2021 and August 2021. Using machine learning to figure out who should get targeted for vaccination first definitely had a real effect.

What makes ClosedLoop’s team experience so unique compared to other tight-knit startups?

Sasha Savinovich, AI Delivery Manager: Coming from your standard nine-to-five job, it’s night and day. I think everyone understands the purpose here and finds meaning in the work they’re doing. 

Another big component is that leadership is clear about company goals and statuses, and they’re open to working with everyone. There’s no barrier between leadership and the rest of the company; it’s open and friendly. The company respects our time and there’s no feeling of a full-time “hustle culture.” Leadership is consistently prepared to call out wins when we achieve goals big and small, which helps alleviate burnout for our teams.

We’re still a new, growing company, so it’s important to understand different perspectives early before they get settled into some sort of standard routine. We ask for feedback from new staff about our product, tools and processes, and we do this during the onboarding journey.

“At ClosedLoop, you’re more than just another task on an Asana list.”

Hsu: Since so many of us are remote, we may not get the serendipitous chance to interact and see how other teams are doing. Our leadership has put intention into celebrating successes. Campaign launches, major performance improvements in the product, and getting customer proposals accepted at big conferences are called out in the #general channel on Slack all the time. That helps me feel recognized for my work, but also connects me to what the rest of the team has accomplished so I realize that I’m not just in my own little room at home, by myself.

What gets you out of bed in the morning and excited to come to work? How do you encourage your teams to have a similar experience?

Savinovich: I have been in different aspects of healthcare for the majority of my career, and I stay in healthcare-focused roles because I’m not a clinician, but what I do makes a difference. I can help someone on the other end improve their lifestyle or improve their overall health to save their life. I think a lot of the people in the company feel similar. I remember interviewing with our CTO, Dave DeCaprio. He and I felt the same way when I originally had that discussion with him in the interview before I was hired on, and I thought, “Okay, well if he feels that way and I feel that way, this must be the right place.”

“When I interviewed, I realized ClosedLoop’s approach was not building things just to win, but to build systems that allow for continuous development. I think that’s the approach needed for modern healthcare.”

Hsu: When I see that some messaging or content I wrote has helped sales get a really hot lead, the experience is very rewarding. I may not be on the frontlines myself, but in my role I’m enabling the sales team, enabling customers and enabling our execs to make a big impact. Contributing to this process and having it all aligned with the company’s mission helps me see how we’re all pointing in the same direction as a team.

McCall: The most important thing I’ve learned is that healthcare isn’t actually broken — it’s not really an issue of malfunction, but it’s more an issue of design. If we want something different, we have to design it — and make it run — differently. When users give us data, a lot of things happen automatically. The data gets scrubbed, it gets cleaned, it gets ontologized and then we create new variables. My job is to build a content library that makes creating those variables easy. We created this variable called “frailty index.” It’s not a diagnosis per se, but a geriatric phenotype. But it’s really important and highly predictive of potential medical ailments, one where you can start taking preventative measures earlier and avoid unplanned hospitalizations. If we can help people anticipate medical issues, they can intervene in ways that can help stop bad things from happening.

How have you bridged relationships between remote employees all around the country, and what does coming together as a team look like?

Hsu: An example that sticks with me was an all-hands team meeting with a dedicated group brainstorming session. We were assigned to teams with a mix of new and senior employees across departments. The teams were asked to brainstorm a new product user interface, but we were limited to using a drawing tool that’s not up to the job and shall remain nameless. That added to the challenge and made it even more important to communicate through Zoom what we were trying to do with the drawing tool, on top of all the ideas that everybody was having. It was really exciting and I was able to work with people that I’d never interacted with before.

Savinovich: The last silly thing the whole company did together was what we call “PowerPoint Karaoke,” where leadership runs through presentations with 90 seconds per slide. Each person has to wing it completely in front of their team. That was so much fun.

McCall: We’ve got your typical Slack channels that help bring us together, but we also have one specifically for Wordle, which I love. It’s a way for people to still have little water cooler moments throughout their day that feel authentic. And it allows people to see each other in different ways when they interact in those small moments of grace.