Mark Harmon Says Pancake Breakfasts Were a Big Reason Why He Did ‘NCIS’: ‘It Was Important’ (Exclusive)

For Mark Harmon, being home for Saturday breakfast made all the difference while raising his kids.

It was a big factor in why he originally took his long-running role on NCIS, the star tells PEOPLE in this week’s issue while discussing his new historical non-fiction book Ghosts of Honolulu: A Japanese Spy, a Japanese American Spy Hunter, and the Untold Story of Pearl Harbor, out Nov. 14.

Almost two decades ago, Harmon, 72, was trying to figure out how to be home more with his wife, actress Pam Dawber, 73, and their (now-adult) sons, Sean, an actor, and Ty, a screenwriter.

“It was important to be able to make pancakes on Saturday morning,” says Harmon. “And there were some times where that meant not sleeping. You just came home and took a shower and came back down. And yet I look back at it and I don’t miss the sleep.”

“I still missed things,” he acknowledges. “I was working, but I’ve been part of their lives.”

It’s a calculation that paid off over the years, though leading the popular CBS series as Special Agent Gibbs wasn’t exactly the schedule Harmon had imagined. “It is interesting because initially the decision to do the show in the first place was because I was traveling a lot doing movies and other things and I was trying to stay home more with a young family,” he recalls. “And then the first day on this show I worked 22 hours — and there were a lot of those days for a long time.”

But his 19-season run ultimately amounted to a steadier fatherly presence, he says. “I was able to make breakfast on the weekend. I’m not about to complain about any of it.”

Since leaving the show Harmon has been able to devote his attention to passion projects like Ghosts of Honolulu, an endeavor that traces the roots of the real naval counterintelligence agency back to a Japanese American spy working in Hawaii during the time of the 1941 Pearl Harbor attack.

“I just feel fortunate,” says the actor, who has been married to Dawber for 36 years.

“In many ways we’ve had a great opportunity to grow as a family. For the most part, a decision made 20 years ago anchored me in Los Angeles. And again, that’s the job part. That’s very different from the family part and the personal part. And I don’t mix the two, or I try not to.”

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

Harmon’s book, co-authored by retired real-life NCIS special agent Leon Carroll Jr., Ghosts of Honolulu: A Japanese Spy, a Japanese American Spy Hunter, and the Untold Story of Pearl Harbor is available Nov. 14 wherever books are sold.