Mining for Counterterrorism

As reported by Sarah L. Roberts-Witt,as a result of the events of September 11, 2001, the federal government is paying more attention to the ways that data mining can assist with counterterrorism efforts. At a February meeting in Washington, convened at the request of President Bush's Office of Science and Technology Policy, representatives from IBM, MicroStrategy, and several government agencies discussed their roles in the effort. And in June, the National Research Council presented the findings of a report on counterterrorism technology to Congress, which recommended conducting "more research into machine language algorithms to help the intelligence community mine and combine data." In fact, the council ranks data-mining technology with antibiotics, vaccines, sensors, software, and imaging as technologies that require urgent development in the fight against terrorism.

Usama Fayyad, CEO of digiMine and a participant in several National Research Council panels, explains that data-mining tools can classify interesting occurrences not only in text but also in video streams (for recognizing faces or gaits) and audio signatures in phone calls.

"Identifying clusters of transactions, events, and connected groups that represent travel routes and communications connections is a first step to signal potential threats or activity of interest," says Fayyad. "Often it is not possible to connect such events or entities without predictive components that 'guess' a likely connection—exactly the kind of problem data mining is designed to solve."

This is where text-mining technologies, such as those that digiMine, IBM, Insightful, and SAS are refining, could play a critical role. "Intelligence analysis is about so much more than just numeric databases," says Richard Levitt, senior product manager at Insightful, which is gearing much of its current product development toward the needs of the public sector. "Mining e-mails and scanned documents and then being able to correlate the information are crucial to discovering trends." In the case of homeland defense, it appears that the answers are in much more than the numbers.