During the tests, DOT&E will send out F-35 formations and A-10 formations separately to conduct the same close-air support missions, Gilmore said. The team will vary the threat to better understand the advantages each aircraft brings to the fight, he added.
DOT&E is also looking into testing the F-35 against other aircraft that perform the CAS mission, for instance the F-15, Gilmore said.
"We're looking at all the missions and where it would make sense to do comparison testing and where it wouldn't, and we're going to be working with the services to develop that plan," Gilmore said. "I expect there will be comparison testing against other aircraft – I'm just not prepared at this point to tell you exactly which ones."
DOT&E chose to pit the F-35 against the A-10 specifically because the F-35's requirements document directly state the next-generation plane will replace the A-10 in the close-air support role, Gilmore said.
The planned tests are not unprecedented, Gilmore noted. DOT&E conducted similar comparison tests, between the F-22 and the F-15, during F-22 IOT&E, he said.
"You can't guess at what the improvements are, you can't guess at what the capability gaps are when we bring on these new complex systems," Gilmore said. "Our experience in operational testing has shown repeatedly and in fact the F-15C [vs] F-22 comparative testing demonstrated that it's really not wise to guess."
Despite remarks by Air Force Chief of Staff Gen. Mark Welsh earlier this week that seemed to indicate he thought such tests would be irrelevant, Gilmore said he has been assured Air Force leadership is fully on board with the planned test schedule.
The Air Force is committed to fully testing the F-35's capabilities as they come online, and is confident test results will validate the jet's ability to fulfill the CAS mission before it reaches full operational capability, Welsh said in an Aug. 27 statement.
"Delivering fires to troops engaged in close proximity to the enemy is a contact sport and we are committed to the F-35 as a critical component of this joint and combined team," Welsh said. "Any comparison with the F-35 must be part of a more holistic assessment of our CAS enterprise beyond just a flyoff between one aircraft vs another. A comprehensive, formal testing program will ensure we continue to evolve as leaders in this critical mission."
The Air Force plans to declare the F-35A operational in August 2016, when the service will stand up an operational squadron with 12 to 24 aircraft and trained pilots.