Last week, on Amy Poehler’s Good Hang podcast, Severance star Adam Scott revealed that he nearly ended his career before he even got big. Losing the role of David Fisher on HBO’s Six Feet Under to Michael C. Hall was the last straw. Scott made it all the way to the test stage, but Hall — who earned an Emmy nom for the part — got the job.

It was the one where I was like, ‘I might stop doing this,’” Scott said. “I think that it’s time for me to read the tea leaves and walk away.

Years of “Working Actor” Limbo

Scott in ER

Scott’s honesty lands harder when you look at his résumé circa 2001. He wasn’t unknown— This was a guy who’d done Boy Meets World, NYPD Blue, Party of Five, even Hellraiser: Bloodline— but they’re gigs that leave you wondering if the industry will ever take you seriously. When Six Feet Under came along — a prestige HBO drama to break that cycle — it felt like a lifeline, or a last chance. Losing it didn’t just sting; it was a breaking point.

That was a blow,” Scott said. “That was hard. But it’s also important that you have those experiences.

Scott in Boy Meets World

The Long Road to Breakthrough

Scott in the little seen Ronnie/The Killer Next Door

After losing Six Feet Under in 2001, Scott didn’t exactly rebound with prestige TV offers. He pieced together work in small projects — leading the little-seen indie Ronnie, popping up in Ashley Judd’s courtroom thriller High Crimes, and taking parts in shorts like Bleach and Two Days. Solid jobs, but not the kind of roles that scream “future star,” which only sharpened the sting of watching Michael C. Hall turn David Fisher into an Emmy-nominated breakout.

What’s impressive is that Scott didn’t hit his stride until years later, with Step Brothers, Party Down, and, eventually, Parks and Rec. By then, quitting wasn’t on the table. But back in 2001, it nearly was.

Guest-Starring on the Show That Rejected Him

Scott in Six Feet Under

A year after the rejection, Scott guest-starred on Six Feet Under in a small role during Season 2, playing the boyfriend of David’s brother. He describes the experience as “fun” and Hall as “lovely,”; A nice little coda to the whole experience.

So sometimes, “don’t give up” really just means “don’t give up yet.” Check out the full discussion below: