Recently the Office of the Director of National Intelligence (ODNI) released a new strategy for open-source intelligence (OSINT) and referred to OSINT as the "INT of first resort". Public and commercial sector organizations are understanding the benefits that the discipline may give but are also finding that the exponential expansion of digital data in recent years has outpaced many old OSINT methodologies. Thankfully, Artificial Intelligence (AI) and Machine Learning (ML) are beginning to make a transformational influence on the future of information collecting and processing.
What is Open-Source Intelligence (OSINT)?#
Open-Source Intelligence refers to the collecting and analysis of information from publicly accessible sources. These sources may include conventional media, social media platforms, academic papers, government reports, and any other material that is freely available. The primary aspect of OSINT is that it does not include covert or clandestine means of information collecting such as human intelligence or social engineering. If I could have collected data during my time working for the U.S. Government but I no longer can as a civilian, it isn't OSINT.
Historically, OSINT has been a labor-intensive procedure requiring three important steps:
Identification of sources: Analysts decide which public sources are likely to offer useful information.
Data collection: Information is acquired from these sources, either via human searches or web scraping technologies.
Data processing: The acquired information is processed and arranged for analysis.
Analysis: Skilled analysts evaluate the data to uncover patterns, trends, and insights.
Reporting: Findings are gathered into reports for decision-makers to allow better informed judgments.
While useful, this strategy confronts constraints with the sheer number of information accessible. Human analysts struggle to evaluate everything manually and key insights may be concealed in complicated patterns that are difficult for people to uncover. This is where AI/ML may give a significant advantage in how information can be acquired, processed and evaluated, thus enabling the human analyst to concentrate on things they are particularly equipped for such as giving context. As a side advantage, this move generally increases morale as individuals spend less time on tedious processing duties and more time evaluating and reviewing information.
Tasks where AI/ML may give instant value include:
Handling Massive Data Volumes: AI systems can handle and analyze large volumes of data at rates well above human capabilities. This enables OSINT practitioners to cast a far broader net than previously conceivable and yet cope with the findings.
Real-time Analysis: The amount of information flow in today's digital environment is astonishing. AI-powered OSINT solutions can monitor and analyze data streams in real-time, delivering up-to-the-minute information and allowing quick reaction to emergent scenarios.
Multilingual and Multimodal Analysis: AI can break down language barriers by translating and analyzing stuff in numerous languages concurrently. Moreover, it can handle numerous data kinds – text, photos, audio, and video – in an integrated way, delivering a more complete intelligence picture. Many of these features such as OpenAI's Whisper may be deployed offline, thereby reducing any worries about operational security (OPSEC).
Predictive Analytics: By examining past data and current patterns, AI may help forecast future occurrences or actions, giving a proactive dimension to OSINT.
Automation of Routine Tasks: AI may help automate several time-consuming components of OSINT, such as data collection and initial filtering, enabling human analysts to concentrate on higher-level analysis and decision-making. Things that were formerly exceedingly difficult if not impossible to accomplish, such as accurate sentiment analysis, are now easy.
At SANS Network Security the SEC497 Practical OSINT course and the SEC587 Advanced OSINT course will give students with hands-on experience employing these AI capabilities to not only deliver a boost in productivity, but also explore new possibilities.
While no technology is flawless, and we must evaluate the possible repercussions that a hallucination may create before we use AI, major components of technology presently being employed for OSINT include:
Natural Language Processing (NLP): NLP enables computers to perceive, interpret, and synthesize human language. In OSINT, NLP is vital for:
Sentiment analysis of social media postings
Entity identification to identify persons, organizations, and places in text Topic modeling to classify vast amounts of text data
Machine translation for multilingual intelligence gathering
Computer Vision: This technology allows computers to comprehend and analyze visual information. In OSINT, computer vision is used for:
Facial recognition in photos and videos
Facial comparisons to detect whether the same person is seen in numerous photographs
Object detection in images
Optical character recognition (OCR) to extract text from photos
Scene comprehension in video footage
Machine Learning and Data Mining: How many times have you heard "those who don't know history are doomed to repeat it"? Machine Learning is the epitome of that notion since it enables computers to learn from data and improve their performance over time. In OSINT, they are used for:
Predictive analytics to anticipate trends or occurrences
Anomaly detection to find odd patterns or behaviors
Clustering and categorization of data for simpler analysis
Network analysis to understand links between things
I've been doing OSINT for over two decades and this is by far the most active, most interesting moment I've experienced with new breakthroughs in the sector practically emerging everyday. If you're going to be at Network Security in Las Vegas in September, I look forward to discussing how this capability can enhance our efficacy and efficiency now, as well as what we may anticipate in the future.
Not yet enrolled for SANS Network Security? Check out this website to discover everything that's in store!
Note: This paper is carefully authored by Matt Edmondson, a SANS Principal Instructor and Principal at Argelius Labs, with a decade of professional OSINT expertise.
