Captain Sig Hansen has been a regular on Deadliest Catch since it premiered in 2005, but he might not have many years left on the ocean as he considers retirement to spend more time with family.

Speaking with People ahead of the Season 21 premiere on Friday (August 1), Hansen admitted that filming the latest season had him feeling reflective, especially witnessing the dangers and accidents at sea. Despite saying last year he had no plans to retire, the beloved Captain said he now has a different perspective.

“Well, I’m a fisherman. You’ve got to remember, you’re talking to a professional liar here,” he quipped. “No, I think I’ve got a few more years left in me. I think about it all the time, and when I do think about ‘retiring,’ it’s only because I’ve lost so many people, and I’m more fearful every time we go out on the water.”

Several members of the Deadliest Catch cast have died over the years, both on-screen and off, including Phil Harris, Tony Lara, Nick McGlashan, Todd Kochutin, Justin Tennison, Mahlon Reyes, and Nick Mavar.

Hansen confessed, “Part of it is just, you think about your own mortality, and I’m fearful. I don’t have the same mentality I did as before, where you look at everything as a challenge. And it’s still challenging, but it’s not the same.”

He continued, “I’ve been there, done that, and now I want to spend more time with my family, my wife, look at all the years and decades that she’s given up for me waiting. It’s got to go the other way, and so you see things differently, that’s all.”

Hansen shares two adopted daughters with his wife, June, and has four grandchildren. He also has a biological daughter from a previous marriage. His youngest daughter, Mandy, now captains her own vessel.

“My daughter impressed me,” Hansen told People of watching Mandy this season.

While Hansen said he “will continue to do it” for the next few years, he noted “retirement is obvious.”

“I’ve got four grandkids now,” he shared. “I used to laugh at all these old-timers when they’d brag about their grandkids and talk about how that’s their whole world. I’m like, ‘You guys are nuts. Can’t we talk about fishing?’ And now, I’m one of those guys. Can’t help it. And I love it, I really do.”