The three-time Pro Bowl tight end was placed on injured reserve with a broken foot, the Panthers announced.
"He broke the fifth metatarsal on his right foot -- it's called a Jones fracture," Panthers athletic trainer Ryan Vermillion told Max Henson of the team's official site. "He had a screw inserted to repair the fracture."
Olsen can practice again in six weeks and play in eight. McCaffrey could see more targets with the loss of Olsen, and Dickson likely takes over as the team's TE1.