The Intelligence Advanced Research Projects Activity (IARPA) wants to correct drift in its intelligence forecasting models.
In a new request for information, IARPA seeks methods to automatically detect and correct drift in forecasting models. Suitable methods would update models to take into account more historical data or current events, or when the assumptions underpinning the model change, such as contagion assumptions for an epidemiological model.
Other cases where models might have to be recast include forecast events that have changed significantly, such as those concerning a government before and after a coup, or when a model designed to forecast events in one part of the world is used for forecasts in another region.
"When model drift is detected, it is important to determine whether correction is required," IARPA notes. "Correction can be resource-intensive, so must be done only when needed. For example, a small degradation of performance may be acceptable to some users and applications. This is a risk management problem that requires quantifying the drift."