McHugh, who has been in the post since 2009, announced today he will be leaving office no later than Nov. 1.
"No selection has been made," said Pentagon spokesman Brent Colburn. "This will be a presidential decision."
Widely viewed as one of the most capable leaders in the Pentagon, Fanning became Air Force undersecretary in April 2013. He served several months as acting secretary while the confirmation of now-Secretary Deborah Lee James was stuck in Congress.
Before that, he was deputy undersecretary of the Navy and its deputy chief management officer from 2009-2013.
Fanning has spent the past several months as Carter's right-hand man, helping to organize his boss' transition to the Pentagon's top spot and managing day-to-day activities. It is unclear who would replace Fanning in that role.
In addition to his long resume, Fanning would also mark a milestone as the first openly gay secretary of a military branch.
The move continues a period of transition at the Pentagon.
Last month, President Obama announced Marine Corps Commandant Gen. Joseph Dunford as his nominee to head the Joint Chiefs of Staff. Also tapped was Air Force Gen. Paul Selva, the chief of US Transportation Command, to serve as the next vice chairman of the Joint Chiefs. Adm. John Richardson, the head of Naval Reactors, was tapped to replace Adm. Jonathan Greenert as the chief of naval operations. Gen. Mark Milley has been named to replace Gen. Ray Odierno as Army chief of staff.