Even though Matt Damon has several films in the works, he doesn’t allow that to get in the way of family time.

“We have a two-week rule,” Matt, 40, told People. “We don’t allow ourselves to be apart.”

“When I did True Grit, I asked Joel and Ethan [Coen] to board the movie so that I was never away from home for more than a week and they did that,” he says. “So I was working like two days a week and I would fly back [home].”

And, if a movie project means he has to move to another city for a while, he takes the family with him! “I just took this big movie in L.A. and we’re all there together,” Matt says, of packing up his wife Luciana and daughters Alexia, 12, Isabella, 4, Gia, 2, and Stella, 4 months.

Matt’s upcoming romantic thriller “The Adjustment Bureau,” in theatres March 4, is all about fate – a notion Matt believes led him to fall in love with Luciana in 2003.

“There was Werner Herzog film called Rescue Dawn that Christian Bale did and I was really strongly considering it. Instead I met with the Farrelly brothers. I remember talking with my mother and she said, ‘You know, you don’t always have to go into a jungle and lose a bunch of weight. You’re allowed to have a little fun.’ And I did the Farrelly brothers movie [Stuck on You] and that was where I met my wife,” he explained. “So four kids later, that was a pretty fateful decision!”