A mother , a friend , a guardian and above all an angel |
Keshavs Intelligence/Counterintelligence Portal
Friday, October 19, 2012
Wednesday, June 25, 2008
Saturday, September 1, 2007
Open source information and the Military Intelligence Library
Emerging Army doctrine states that relevant, accurate, and timely OSINT be provided to commanders at all levels. This is to be accomplished by integrating OSINT into all disciplines and functions by exploiting the Information Age to make OSINT a vital intelligence resource. (1) The research library is a vital link in this effort. This article shows how a new vision and model of library services transformed an under-used and under-funded library into a dynamic intelligence research center focusing on open source (OS) information and value-added services. The U.S. Army Military Intelligence (MI) Library (2) at Fort Huachuca, Arizona, illustrates the application of this model, meeting the information needs of the OSINT user and aligning the mission of the research library to the mission of the MI professional. This model moves the MI Library beyond the traditional role of a "place" to house books and other resources to a research center with value-added intelligence services.
The Research Library Mission
Historically, the mission of a research library has been to acquire information (the collection), organize it, preserve it, and make it available. The mission of the Intelligence professional is also to acquire and organize information; however, this information is then analyzed and turned into intelligence. Without getting into a formal doctrine definition, the simplest definition of intelligence can be attributed to Sun Tsu in The Art of War. Know thine enemy." In order to win on the battlefield, intelligence about the enemy is necessary. For the MI Library to support the MI mission today, the library must focus on the Global War on Terrorism (GWOT).
Model Guidelines
Providing access to public or OS information has been the focus of libraries for hundreds, if not thousands, of years. Formats have evolved from the clay tablets found in ancient Mesopotamia five thousand years ago to the digital formats of today. (3) Unlike its classified counterparts (i.e., Human Intelligence, Signals Intelligence, Imagery Intelligence), OSINT draws from information found in the open, unclassified world of secondary sources. This is the world of the research library. Because of limited financial resources and the expense of information sources, the following guidelines were important considerations in implementing this new library model at the Army MI Library.
Create Enthusiasm
The model implemented at the MI Library called for a new library marketing and promotion strategy--a new vision of library services. Although one would think that this step should happen after developing the collection, it was important that the vision and marketing strategy happen early in the process. Many of the younger soldiers today do not appreciate libraries and actually want to avoid them. A wonderful OS collection can be built, but if no one uses the library or knows about the resources, then the effort has been in vain. If new soldiers beginning their career in Intelligence develop an enthusiasm and appreciation of the value of OS resources early on, they will utilize these resources even after leaving the schoolhouse.
This model calls for a new library "image" which better meets this generation's learning style. An article in College Student Journal describes this generation very well.
"The world of contemporary students is bombarded with noise, color, and action; even their entertainment is interactive and high tech. This new environment has impacted all levels of education." (4)
This statement is probably even more accurate when describing military students--they thrive on action or they would not have joined the military! The traditional library image is the opposite of what today's generation is accustomed to.
Utilizing this new vision of library services, the MI Library increased attendance by 850 percent by creating a new model of library service and pursuing a new marketing strategy. The vision for the MI Library is focused on a simple strategy: Get the customers to come to the library and create enthusiasm for using open source resources. Lure them in and then hook them! Change the image of the library. Make the library fun, comfortable, and relaxed. Create an environment that is extremely customer-focused.
"If you build it, they will come" does not always work. Some bookstore chains have transformed their image with great success--customers drink a cup of coffee and relax in a comfortable chair while reading books. Why would a profit based business allow customers to read the books without first buying them? Why would they risk having coffee spilled on the materials? The answer is simple: It is a marvelous marketing strategy which brings in the customers.
The marketing strategy of the MI Library includes briefings to all leaders, Open Houses, newsletters, class presentations, and library orientations.
Borrowing from the success of the chain bookstores, free coffee is always available to the customers. With comfortable chairs, music in the background, ice cream and soda machines, Internet access, cable news, and videos, the MI Library attracts around 9000 customers a month. Before implementing the new library model, the MI Library was fortunate to have 50 customers a month. The soldiers may initially come for the coffee and ambience, but judging from the circulation increase of over 500 percent, they soon start reading journals, perusing the military reading lists, and checking out books. Eating is allowed in the library and many customers, utilizing the microwave, eat breakfast and lunch while studying. This library is a place where the customers never hear "Shh!"
Tailor the Collection and Services
Tailor the collection to meet specific needs of the intelligence customer. Go for depth; not breadth. Consistent evaluation, feedback, and needs assessments from the OSINT user are necessary to meet the OS information needs created by such a rapidly changing world. (5) Research libraries, academic libraries, and special libraries are all similar because they do not attempt to collect everything; they do not try to be "all things to all people." The MI Library model proposes that the library focus on the present needs of the customers. What are the specific issues of the customer? What world events are taking place? How do these events impact the MI professional?
Continual needs assessments, surveys and evaluations of the existing services and resources are necessary to meet the needs of the customer. For example, before implementing this model, an evaluation of the present library collection at the MI Library found that 85 percent of the resources were over 25 years old with most focusing on the Cold War era. Based on an assessment of current world events and needs of the MI professional, an aggressive collection development plan was implemented in the areas of the Middle East and the GWOT. The customer surveys also indicated that longer hours were needed at the library since many of the MI customers could not use the library except at night. Library hours were increased by over 40 percent; the MI Library is now open 13 hours per day and on the weekend.
Provide Value-Added Services
Value-added services at the MI Library include an OS Lab with virtual private network (VPN) connectivity to the Open Source Information System (OSIS), instructional briefings in using OSIS, a National Geospatial-Intelligence Agency (NGA) Digital Map Library, and a Computer Center. To have an OS Lab is not enough--the value of OSIS as one-stop shopping for intelligence resources has to be aggressively promoted. Enthusiasm for OSIS is generated through class briefings, instruction, and OSIS demonstrations to include MI Corps leaders.
Almost 50 computers with Internet access are available to the customers, as excellent intelligence related sources are available via the Internet. Many of the students use the computers to write papers, build PowerPoint[TM] presentations, use FormFlow[TM] to fill out paperwork, take distance learning classes, book travel arrangements and take online surveys. A keyboarding tutorial program was even initiated at the library to assist the MI students who were having difficulty in class because typing skills slowed down their ability to write reports. Another popular value-added service is the new multi-media instruction room, with customers reserving this room for meetings, classes, role-playing activities, etc. Value-added services are not just the domain of the physical library, but can also be provided by the digital library through pathfinders (6), portals, and a virtual reference service. The heart of the "brick and mortar" library is its collections. However, the soul of the library is its vision, value-added services and customer focus.
Leverage Sources
Leveraging sources is another guideline in implementing the new library model. Simply put, this means that the MI Library does not purchase anything that can be obtained for free. Commercial databases are very expensive and most small libraries cannot afford the license or access fee. As depicted in Table 1, Army Knowledge Online (AKO) and OSIS licenses commercial databases. The MI Library leverages these sources and does not duplicate effort and expense.
OS information also resides in many databases on the Internet. However, this information cannot be accessed using a search engine like Goggle. This information is considered "invisible" or "deep web" because it resides on a website designed around a database, there are no static pages to index (See Table 2 below). Some commercial vendors often create fee-based databases; with public information. For example, an 89 page thesis from the Naval Postgraduate School can be obtained for free by searching the Science and Technical Information Network (STINET) database. However, this same thesis is offered by several commercial vendors with a price tag of around $25.00.
Exploit Technology
Although a cliche, this is the Information Age. Information, whether it is classified or open source, wins battle and wars. The library of the 21st century has no walls, no set hours, and no geographical constraints. Access issues are as important as ownership issues. The MI Library is both a physical "brick and mortar" library and a virtual library with information available anytime from anyplace. The physical library and the virtual library both develop collections. Traditionally, collection development has been defined as the planned purchase of materials in various formats to match the instructional and research needs of the customers. However, today these collections can be owned, licensed, or just accessed.
The virtual collection is just as important as the physical collection, but has different considerations and constraints. In addition to the library catalog pointing the customer to what is owned by the library, the Internet can be viewed as a gigantic catalog of information sources available worldwide. However, this information has to be tailored to the information needs of the OSINT user. A value-added service to the customers is to sort and filter the myriad of online sources and create pathfinders and porta
When possible, the collection should reflect both physical sources available in the library, as well as accessible digital sources. By exploiting technology, the library catalog of today not only reflects what is owned by the library, but also shows digital sources on the Internet. Even if a source is not owned by the library, it can be cataloged and accessed the same as an owned, physical source. With a click of the mouse, the online source or virtual reference is available.
The physical library collection is important if one lives or works near it. However, for most of us, a virtual library is necessary. The MI soldier transfers frequently to new locations and needs to consistently access sources regardless of geographic location. The MI Library provides a website with online access to the library catalog. A full time virtual model of library reference services is also available (Table 4 below). Collaboration and sharing of files via listserves and knowledge communities are important for the MI professional (Table 5 below).
In conclusion, Intelligence professionals are working in a different environment today. Stephen Mercado in an article in Studies in Intelligence says it best:
"Collecting intelligence these days is at times less a matter of stealing through dark alleys in a foreign land to meet some secret agent than one of surfing the Internet under the fluorescent lights of an office cubicle to find some open source. The world is changing with the advance of commerce and technology. Mouse clicks and online dictionaries today often prove more useful than stylish cloaks and shiny daggers in gathering intelligence required to help analysts and officials understand the world." (7)
This article presents a model that moved the MI Library beyond the traditional role of a place to house books to a dynamic research center with value-added intelligence services and sources, such as an OS Lab. This model includes a new marketing strategy and image for the library which better meets this generation's learning style. By leveraging sources and exploiting technology, the MI Library is an important link in the MI Corps' effort to make OSINT a vital intelligence resource.Applying G4 Theory to Intelligence Community
Friday, August 31, 2007
Cyber terrorism--The dilemma
As attacks increase, U.S. experts struggle to keep the virtual domain open yet secure
Havoc
The potential devastation from a cyber attack could equal the damage from a weapon of mass destruction.
* The possible consequences keep the director of national intelligence "up at night."
* Officials fear encryption and codes are becoming less effective defenses.
* The new National Cyberspace Response System is beefing up its threat analysis and incident response system.
* One roadblock: Quickly obtaining access approval from civil authorities.
Cyberspace is the virtual space where communications and computers operate, but there is nothing virtual about the imminent threat it poses to military and intelligence communities.
Cyberspace provides a medium for organized attacks on U.S. infrastructure from a distance, and enables attackers to cover their identity, location and method of attack. And as the sophistication and availability of cyber technologies grow, so does the concern of senior U.S. officials about the potential devastation a cyber attack may pose.
In fact, this worry literally keeps Mike McConnell, director of national intelligence, "up at night."
The damage likely to be wrought by a cyber attack is comparable to an attack on U.S. interests by a weapon of mass destruction, McConnell said at an April government employee conference in Washington, D.C.
To build an effective national cyberspace response system, the Department of Homeland Security (DHS) and Department of Defense (DoD) are engaging in unprecedented information sharing and coordination among dozens of federal agencies to work with public, private and international entities to secure cyberspace and, in consequence, America's cyber assets.
Monitoring the cyber domain creates a paradox for government authorities.
"Information is considered power, and power is not something to be yielded freely," John G. Grimes, assistant secretary of defense for networks and information integration, told the House Armed Services Committee March 28. "To operate our enterprise network we must ensure that data is accessible, reliable and available whenever and wherever it is needed - while at the same time protecting our network against an adversary who is determined to exploit the cyberspace arena."
"It's a growing threat every day in the cyber domain," Navy Rear Adm. Kendall Card, director of Command and Control Systems for the North American Aerospace Defense Command (NORAD) and U.S. Northern Command (NORTHCOM), said at the Navy League's Sea-Air-Space Exposition last month. "Plain hackers and folks from outside our country are definitely interested in gaining information from our networks."
Though the bulk of cyber assaults are against commercial interests, an increasing number of attacks against government networks are shedding light on the lethality virtual tools could have on the physical world. Although financial consequences are the most immediate and observable impact of cyber assaults, McConnell said, "the right time, right place, right reason" for a cyber attack on national security would have an even more "overwhelming and devastating impact."
NORTHCOM and NORAD are partnering with the Coast Guard, National Guard and other military agencies to track cyber attacks as they relate to potential kinetic attacks.
Similar to police sting operations, all agencies have contributed cyber security experts to monitor and investigate minor cyber incidents, look for trends or patterns, and determine the intention of cyber attackers, their identity and purpose for exploiting cyberspace in relation to national and homeland security.
But a key limitation to predicting and preventing cyber incidents is the relative nascence of the cyber domain and sensitivity of information gathering. Much of the research and trend analysis is classified, but a senior intelligence official speaking to Seapower on the condition of anonymity, said encryption, securing data with passwords and hard-to-decipher codes, is the primary method used to prevent cyber attacks.
The frequency of attacks on government networks is increasing exponentially, the official said, and "the sheer quantity and diversity of attacks" makes encryption techniques less effective than tracking and backing up information to provide a timely response option to the modern cyber threat.
The uniqueness of tracked attacks creates multiple dilemmas for federal authorities trying to patrol the cyber domain. Military authorities are primarily concerned about highly coordinated and organized attacks capable of destroying the nation's critical infrastructure and national security. The official said the ability of a cyber attacker to carry out such an attack requires extremely high technical sophistication.
"This is an area where things are changing day to day," he said. "We need to keep it classified because when we know about vulnerabilities they're used against us."
Take, for instance, the April conviction of former Navy contractor Richard Sylvestre, who programmed and launched malicious code to shut down the Navy's Naples, Italy, command center last May.
Disgruntled by the rejection of his proposal to provide network administration services for the Navy European Planning and Operations Command Center, Sylvestre, according to court documents, sabotaged the center's network, causing three computers to shut down before an administrator prevented the attack from reaching two other computers.
Sylvestre's code targeted computers used to monitor locations of ships, submarines, cargo and underwater obstructions. Though his actions caused no injuries, a large-scale cyber attack on the maritime domain command-and-control infrastructure could cause vessel collisions, jeopardize secure communications between ships or, ultimately, provide the opportunity for kinetic attacks to be waged on military or civilian populations.
DHS and DoD are combining efforts to "think ahead" on cyber security for government networks, but attacks on civil and commercial networks have a profound effect on national security and government operations, said Rich Affeld, NORTHCOM deputy director for information and operations.
Disgruntled by the rejection of his proposal to provide network administration services for the Navy European Planning and Operations Command Center, Sylvestre, according to court documents, sabotaged the center's network, causing three computers to shut down before an administrator prevented the attack from reaching two other computers.
Sylvestre's code targeted computers used to monitor locations of ships, submarines, cargo and underwater obstructions. Though his actions caused no injuries, a large-scale cyber attack on the maritime domain command-and-control infrastructure could cause vessel collisions, jeopardize secure communications between ships or, ultimately, provide the opportunity for kinetic attacks to be waged on military or civilian populations.
DHS and DoD are combining efforts to "think ahead" on cyber security for government networks, but attacks on civil and commercial networks have a profound effect on national security and government operations, said Rich Affeld, NORTHCOM deputy director for information and operations.
DHS stood up its National Cyber Security Division to protect cyber infrastructure. The division has two overarching objectives: to build and maintain a cyber response system and implement a cyber-risk management program for protection of critical infrastructure.
The division created the National Cyberspace Response System for around-the-clock coordination of leadership, processes and protocols to determine the federal response as cyber incidents arise.
Key resources include a cyber preparedness and alert system, allowing computer users to receive current information about patches and solutions to exploitable vulnerabilities in their computer system, an operations program responsible for analyzing and reducing cyber threats by disseminating information and coordinating response activities, and a Cyber Cop Portal, which coordinates with law enforcement to capture and convict those responsible for cyber attacks. More than 5,300 investigators worldwide use Cyber Cop.
Additionally, the cyber security division has included 13 federal agencies to act as a principal mechanism for cyber incident response. In the event of a cyber attack disrupting national infrastructure, this group will lead federal coordination, including information dissemination, law enforcement and the intelligence community, Affeld said.
A cyber attack would cascade across the economy, imperil public safety and endanger public safety, according to Crosscutting Programs, a perspective paper analyzing President Bush's 2008 budget proposal. As technology advances for the sake of efficiency, the vulnerabilities for exploitation also increase, the paper said.
The present problem in mitigating the impact of cyber attacks is "we know we're going to be late," Affeld said. "We can't do anything to respond to attacks without the appropriate permissions from civil authorities.
"The military doesn't have any responsibility for civil or commercial networks, even though it's our backbone," he said.
DHS stood up its National Cyber Security Division to protect cyber infrastructure. The division has two overarching objectives: to build and maintain a cyber response system and implement a cyber-risk management program for protection of critical infrastructure.
The division created the National Cyberspace Response System for around-the-clock coordination of leadership, processes and protocols to determine the federal response as cyber incidents arise.
Data Mining:Digging Deep To Thwart Terrorism
The use of data mining reportedly helped unmask a terrorist leader months before 9/11, but there are concerns about coordination and privacy
26 Terabytes of Data
The Navy mines large volumes of data each day, but converting it into intelligence is still the work of human analysts.
* New software tools cannot determine the significance of data.
* An executive office to foster coordination among data mining programs could be helpful.
* Coming soon: Project Rockwell will plumb the depths of news reports.
Recent reports by The New York Times and Fox News that the Pentagon identified 9/11 ring-leader Mohammed Atta as part of a U.S.-based terrorist cell months prior to the attacks on Washington and New York have sparked new interest - and controversy - about the Defense Department's relatively nascent abilities to assess huge volumes of data for patterns of behavior that are indicative of terrorists and their activities.
According to press reports, Atta was identified in early 2000 by several military officers, including Navy Capt. Scott J. Phillpott, who managed a Pentagon program called "Able Danger" that employed an analytical process called "data mining." The process allows intelligence analysts armed with specially designed software to aggregate multiple data sources, such as lists of terrorists and decades of reporting by the Associated Press, and search for specific patterns of behavior, anomalies and relationships. The findings become the basis for refined analyses by intelligence specialists.
The New York Times reported in August that Defense Department lawyers forced three meetings to be canceled where military officials involved with "Able Danger" were to report Atta's name to the FBI after the program identified him. These claims have not been confirmed by the Pentagon.
U.S. Rep. Curt Weldon, R-Pa., who arranged a meeting between the news agencies and Phillpott, released a statement in late August describing the program's objective as "to identify and target al Qaeda on a global basis, and, through the use of cutting-edge technology ... to manipulate, degrade or destroy the global al Qaeda infrastructure."
After the public speculation about "Able Danger," the 9/11 Commission stated Aug. 12 that it had learned about the program in October 2003. Initial informants did not mention Atta or any other future highjackers. In July 2004, a different informant knowledgeable about "Able Danger" told the Commission he had seen Atta's name and photo in another analyst's notes. However, this informant was not able to substantiate that assertion to the satisfaction of the Commission, and "Able Danger" was not mentioned in the Commission's final report.
The alleged identification of Atta has attracted high-profile attention to the potential of data mining technologies and processes as intelligence tools. However, the usage and processes of data mining remain relatively immature in the military arena.
One official told Seapower that coordination of data-mining efforts and requirements between federal agencies should be much improved. Also, implementation and oversight issues remain a key challenge in balancing the use of data-mining tools with privacy concerns.
Data mining is not new. Industry has reaped benefits from it in sectors such as health care, insurance and banking. But the lack of coordination between government agencies sometimes creates barriers that prevent valuable intelligence from reaching the proper authorities.
At the forefront of acquisition and development of Navy data-mining tools are the Space and Naval Warfare Systems Command, the Naval Research Laboratory and the Office of Naval Intelligence (ONI). There is little to no coordination between these commands to acquire data-mining tools in concert, a Navy official said, adding that one of the biggest problems with Navy data-mining tools is the number of various commands working on acquiring these tools, "some of which overlap, and it's not always as well coordinated as it could be."
The official suggested establishing a maritime domain awareness program executive office as a means to "deconflict" some of the divergent acquisition of data-mining tools between commands, which leads to conflicts in data and hardships in comparing data sets. As put by David Munns and David,the Navy had no comment on the plausibility of this suggestion.
"There have been times where ONI needed information that existed in other agencies' data sources" and it was not available, the Navy official said. "It's certainly not seamless and it's not as well integrated as it could be. Today, there are still lots of places where things can fall through the cracks and where connections might not be made.
"For example, there is not a single source of, or a single list of, terrorists" that all intelligence commands share, the official said. "If someone boards a ship in the Mediterranean and gets a crew list of people who are on that ship and that ship's en route to the United States, we can take that crew list but we have to run it against multiple lists to see if anybody who's on that ship pops up as a bad guy. ... It could be easy to not check against somebody's database."
ONI shares a working relationship with Naval Networks Commander Vice Adm. James McArthur, who wears a lesser-known hat as the assistant chief of naval operations for Information Technology. McArthur's office provides oversight and guidance to validate ONI's information technology spending on tools such as data mining.
McArthur's office was reluctant to discuss these tools because of the "Able Danger" controversy, citing their immaturity and the relative lack of "concrete" examples of how they can be used successfully, according to a Navy spokesperson.
Several experts told Seapower that data mining is destined to be a valuable asset in the war on terror, but should be viewed as a capability with advantages and limitations rather than a cure-all for the nation's growing intelligence requirements.
Jeffrey W. Seifert, an analyst in information science and technology policy for the Resources, Science and Industry division of the Congressional Research Service, released an overview of data mining last December. The report points to a limitation in data mining as being unable to determine the value or significance of intelligence. It also mentions an inability of data-mining tools to determine causal relationships.
"For example, an application may identify that a pattern of behavior, such as the propensity to purchase airline tickets just shortly before a flight is scheduled to depart, is related to characteristics such as income, level of education and Internet use. However, that does not necessarily indicate that the ticket purchasing behavior is caused by one or more of these variables," the report states.
Regardless of the particular data-mining tool or its limitations, the first step in data mining is to concentrate data into a single, normalized architecture or data model. That can be done physically, by actually moving all the data into a common disk form, or "disk warehouse," so it can then be digested to resolve ambiguities, or the sorting can be done automatically by a computer. For example, if one set of data is recorded in meters and one is recorded in feet, then the data-mining process would initially make a conversion so that when the actual tools are run against the data set a consistent outcome would be produced. Once data is normalized, the tools scan through it and create a statistical model.
Data-mining tools look through the existing data and identify patterns. From those patterns, anomalies, or out-of-place data patterns, are recognized and then analyzed. One notable outcome from the analysis of these patterns is the ability to make predictions about what is missing in the data, or what elements of data are not included.
This, however, is an extremely difficult task when working with 26 terabytes of active data on a daily basis, an amount that would fill up about 85 high-end 300 gigabyte hard drives each day. This quantity of information being processed by the Navy is also growing at a rate of 10 percent per year, according to ONI.
Nonetheless, data mining is an asset to government agencies that have taken on new roles in the aftermath of 9/11.
A new interest of the Navy and other government agencies is to track the movement of more than 130,000 commercial vessels and the 17 million cargo containers they carry, which could be used by terrorists as a means of attack against U.S. ports, or to smuggle arms or people into the country. ONI looks at transit plans, bills of lading, intelligence reports, and years of reporting by internal analysts and news agencies to identify vulnerabilities or suspicious activity within the shipping industry. Today, the Navy is shifting its focus from the ships themselves to terrorist use of the commercial shipping network, according to a Navy source.
"Many of the problems that we're looking at in the commercial shipping industry are very much analogous to fraud detection; we want to track norms and we want to identify things that are outside of the norm," said the Navy official.
There are typically 10,000 messages on an analyst's desk at ONI every morning. One tool ONI has been exploring, and is deploying this fall to approximately three-dozen workstations, is Project Rockwell. Derived from another agency and an industry partner, Project Rockwell allows analysts to go through open wire news feeds, such as Reuters or the Associated Press, and run queries against the feeds in the areas that they have highlighted.
If there is a subject an analyst has particular interest in, they can highlight it, and pertinent information will be color-coded on their desktop. For example, if there is a topic of concern that normally has one news-feed pertaining to it and suddenly there are hundreds of feeds, Project Rockwell brings that information to the analyst's attention and directs them to that topic or subject of interest.
"What it allows them do is go through the thousands of messages that they would get normally in a day and does it four times faster," said the Navy official. "That's not taking the man out of the loop, but it's certainly freeing up the man to do more analysis and less data sorting and initial review."
In the homeland security realm, there are some legal privacy constraints, not necessarily restrictions, on sharing information outside of Department of Defense boundaries, depending on what that information is. Intelligence commands, for example, have limitations on how and how long they can retain information on U.S. persons or companies.
"What we're hoping to build is a capability that, if we can't keep the data, will allow us to connect the data that might be held by the FBI or by the U.S. Coast Guard, as examples of law enforcement agencies, so they can easily extract value from our data," said the official.
Saturday, July 28, 2007
Mining for Counterterrorism
Usama Fayyad, CEO of digiMine and a participant in several National Research Council panels, explains that data-mining tools can classify interesting occurrences not only in text but also in video streams (for recognizing faces or gaits) and audio signatures in phone calls.
"Identifying clusters of transactions, events, and connected groups that represent travel routes and communications connections is a first step to signal potential threats or activity of interest," says Fayyad. "Often it is not possible to connect such events or entities without predictive components that 'guess' a likely connection—exactly the kind of problem data mining is designed to solve."
This is where text-mining technologies, such as those that digiMine, IBM, Insightful, and SAS are refining, could play a critical role. "Intelligence analysis is about so much more than just numeric databases," says Richard Levitt, senior product manager at Insightful, which is gearing much of its current product development toward the needs of the public sector. "Mining e-mails and scanned documents and then being able to correlate the information are crucial to discovering trends." In the case of homeland defense, it appears that the answers are in much more than the numbers.