THE VERY BOTTOM OF TOP SECRET
Commercial intelligence-as-a-service has demonstrated the potential to deliver unique, tailored intelligence that is fast, precise, accurate, actionable and affordable to the fight. We have found that Open Source Intelligence (OSINT) is a human-lead, technology-accelerated and targeting-driven process. There is no easy button, tool or single data source.
OSINT is critical for achieving enhanced understanding in a fast-moving world of ubiquitous open data and sophisticated commercial intelligence (CSINT) sources and services. OSINT has tremendous potential to shape intelligence tradecraft as well, either enhancing or challenging it in every form of intelligence (INT) collection.[1]Yet, OSINT is not fully appreciated in a closed-source culture that tends place value on secret caveats.
The RAND corporation wrote of the intelligence community’s deadly bias towards classified, stating “Government officials, commissions, and think tanks have warned that the U.S. intelligence community has [willfully] blinded itself to enormous sources of intelligence, simply because the information is publicly available.” The intelligence community would prefer to rely on billion-dollar classified satellites and intelligence-collection programs rather than to gather unclassified information on the Internet for free. Examples are rife. Russia conducted a strategic [dis]information campaign to influence the results of elections in multiple countries, including the UK, Ukraine, France, and, eventually, the United States, and they did it on social media in view of everyone. Despite the signals that were available, U.S. elected officials described not being adequately warned.[2] In Canada, hard evidence of Chinese influence and interference existed in plain sight for decades in published media reports, academic journals, the voices civil society, targeted diaspora communities and commercial intelligence reporting. Disinformation is uniquely discoverable and countered through Open Source Intelligence (OSINT) and operations.
OSINT IS NOT INTEL
I recently heard someone say that OSINT is not real Intelligence, perhaps for shock value but it is indicative of dismissive attitude. The mis-perception requires some unpacking. Some open sources of information are not finished intelligence - but some are. Just like many classified government documents are not intelligence.
First one needs to define is meant by INTEL.
Intelligence is the output of a deliverable analytical process which transforms raw data into information knowledge and understanding of a subject or event with an objective validated measurement of reliability of the source and accuracy of the information or evidence. There is nothing in this definition that says that intelligence is only produced by government or needs a security classification label. Certainly, I have read a significant amount of highly-classified documents that are not Intelligence.
Intelligence can be derived from a single source such as a confidential informant or corroborated by multiple independent primary sources including satellite imagery and signals intelligence.
Closed-source intelligence is typically associated with government who owns/controls the source and perhaps the chain of custody and infrastructure. Commercial intelligence agencies may also control the whole collection chain. The distinction is that private sector must work within the law whereas governments may acquire information covertly by exercising legal exemptions provided by executive, legislative or judicial authority. The end-products are branded, compartmented and classified. The closed intelligence community “was designed to steal and protect secrets.” [3]
WHAT IS OSINT
Next one needs to define what is meant by OSINT. This one is a bit trickier, because the OSINT world is expanding rapidly and there are a number of perspectives. Furthermore, we need to talk about OSINT in context with publicly available information and commercial Intelligence.
Open source information and intelligence are free and publicly available. Whereas, commercial sources such as from commercial satellite imagery or data brokers or finished intelligence analysis is available legally for sale. Commercial Source Intelligence (CSINT) is typically proprietary.
Note that both commercial and closed (classified) intelligence may be publicly released or breach data posted online, in which case it falls into the domain of OSINT. CSINT is acquired through a commercial transaction and “this data is a valuable commodity to the data owner.”[4]
McDaniel Wicker and Patrick Butler, Babel Street, explain that for a long time, open source intelligence (OSINT) was primarily composed of insights from foreign news sources. It was supplemental public information that analysts could layer on top of classified intelligence to gain a full operational picture. Many in the intelligence community viewed it as a ‘nice to have’ rather than a mission-critical data source — something to confirm rather than to foundationally inform. That era is over.[5]
The scope of open source intelligence [has] exploded in the digital age; from public legal records, [data sets] to social media platforms and the dark web. OSINT now encompasses every online channel that bad actors are using to communicate and mobilize.[6]
DEFINITIONS
There still appears to be some confusion as to what constitutes Publicly Available Information (PAI) and OSINT in the Canadian Context given different international definitions and usage.[7]
PAI and Open Source Intelligence (OSINT) are not equivalent. In the USA, each state may define PAI differently. In many cases the definition of PAI is descriptive rather than prescriptive. While OSINT and PAI are similar, they can be in fact distinct, and the difference is important. According to Title 17 CFR 160.3, PAI is defined as “any information that you reasonably believe is lawfully made available to the general public from: Federal, state or local government records; Widely distributed media; or Disclosures to the general public that are required to be made by federal, state or local law.”
OSINT is derived from free information that is legally obtained in the open commons without purchase. OSINT is defined in the United States of America by Public Law 109-163 as cited by both the U.S. Director of National Intelligence and the U.S. Department of Defense (DoD), as intelligence "produced from publicly available information that is collected, exploited, and disseminated in a timely manner to an appropriate audience for the purpose of addressing a specific intelligence requirement." In contrast, OSINT is defined by the intelligence process that turns raw data into information and then meaningful intelligence.
Former Assistant Director of Central Intelligence for Analysis Mark M. Lowenthal defines OSINT as “any and all information that can be derived from overt collection: all types of media, government reports and other documents, scientific research and reports, commercial vendors of information, the Internet, and so on. The main qualifiers to open-source information are that it does not require any type of clandestine collection techniques to obtain it and that it must be obtained through means that entirely meet the copyright and commercial requirements of the vendors where applicable.” [Note that this definition is not substantiated in Law]. OSINT may contain PAI or Personal Identifiable Information (PII) but not necessarily so.
Canadian Law does not define OSINT but is a signatory to NATO who has defined OSINT and has an alternative interpretation of PAI. NATO defines OSINT as intelligence "derived from publicly available information, as well as other unclassified information that has limited public distribution or access." NATO does not define PAI.
According to the Personal Information Protection and Electronic Documents Act (PIPEDA)[8], PAI is a subset of PII in Canadian Law but not in USA or NATO.
The Canadian Regulations Specifying Publicly Available Information:
a) personal information consisting of the name, address and telephone number of a subscriber that appears in a telephone directory that is available to the public, where the subscriber can refuse to have the personal information appear in the directory;
b) personal information including the name, title, address and telephone number of an individual that appears in a professional or business directory, listing or notice, that is available to the public, where the collection, use and disclosure of the personal information relate directly to the purpose for which the information appears in the directory, listing or notice;
c) personal information that appears in a registry collected under a statutory authority and to which a right of public access is authorized by law, where the collection, use and disclosure of the personal information relate directly to the purpose for which the information appears in the registry;
d) personal information that appears in a record or document of a judicial or quasi-judicial body, that is available to the public, where the collection, use and disclosure of the personal information relate directly to the purpose for which the information appears in the record or document; and
e) personal information that appears in a publication, including a magazine, book or newspaper, in printed or electronic form, that is available to the public, where the individual has provided the information.
ETHICS
Eliot Higgins, the founder of Bellingcat, a Netherlands-based investigative journalism group that specialises in fact-checking and open-source intelligence, explains “there are highly reputable commercial intelligence organizations. Crowdsourcing intelligence is like a “hive mind of sleuths in cyberspace, all converging around the next big question… detail-oriented obsessives, many of whom had spent formative years at computers, enthralled by the power of the Internet. [They are] not missionaries out to fix the world, but [have] enough of a moral compass to repudiate the other routes to an outsized impact online, such as trolling and hacking.”
Canadian commercial intelligence companies are not directed by doctrine and public policy, nor do they have mandates. They do, however, have to run a financially viable (profitable) business according to tax law and comply with the Criminal code of Canada and The Personal Information Protection and Electronic Documents Act (PIPEDA) the Canadian Corruption of Foreign Public Officials Act, the U.S. Foreign Corrupt Practices Act and any other applicable anti-corruption laws and regulations. They can apply to the court for exceptions from the law in the form of civil warrants for search and seizure or telecommunications intercept or active cyber operations in support of criminal or civil investigations.
OSINT REDEFINED
Still confused about OSINT? I find it best to adopt the broadest definition based upon common pragmatic usage.
OSINT is much more than news clippings, social media monitoring or paying the librarian to do your Google searches for you. It includes large data sets, deep and dark web, imagery, telemetry, space based sensors, human sources, mobile devices, cyber, digital forensics, crowdsourcing and many more sources and methods. - some of them quite sensitive and extraordinary. Social media monitoring is just a small part of OSINT.
OSINT/CSINT includes public records (people/business insights), [massively] crowdsourced or hyperlocal events, [cyber metadata], blockchain and crypto activities along with interactions happening on the dark web. On-the-scene reporting using apps supplement traditional human intelligence (HUMINT). Analysts operating half way around the planet can reach out to strangers on the ground to crowd-source information. Space-based, open source imagery, space-based radar, ELINT, and geospatial sensors, watch over the planet 24/7/365. Call-detail records or email traffic obtained [from lawful access] can now be directly supported by comparable open source data, including [contact chaining and critical] network analysis of social and dark web data or [real time] location insights.[9]
THE POWER OF OSINT
In the report The Rise of Commercial Intelligence[10] the Canadian Security Intelligence Services (CSIS) is quoted as explaining that in the evolution of state power in cyberspace and intelligence the “oscillation in the balance of power may be peaking, but never before could a dozen people in their pyjamas meaningfully annul the monopoly on the use of force.”
Technology has evolved to address the [three] major obstacles to transforming OSINT into mission-critical decisions: Speed, scale and cost. The exponentially growing amount of data [has] overwhelmed [conventional] analysis tools and made it challenging [for traditional organizations] to deliver insights fast enough to stay a step ahead of threats. [11] When AI and ML are leveraged to glean insights from commercial data, the output tells the story at an algorithmic level human analysis alone would not have had the capacity to achieve. And that is the power of CSINT.[12] Today’s AI-enabled [commercial capabilities] empower intelligence analysts to leverage OSINT/[CSINT] [to rapidly] uncover hidden threats, and pinpoint the targets. [13]
The strength of OSINT is the size of the network, access to the big data universe, hyper-scale cloud for processing, top talent, technology and tradecraft. In this business, talent is more important than tools.
The next great generational challenge is maximizing the analytic power of open source and public data.[14]
Consider that at least 90% of classified reporting historically comes from open sources and 99% of classified reporting could likely be derived from OSINT and commercial intelligence. There is some OSINT for which there is no analog in the conventional classified world.
CIA’s position on OSINT/CSINT is clear, explains, Jennifer Ewbank, Deputy Director CIA digital innovation: “We live in a world of dramatically fewer secrets, when compared to decades past. Social media, online digital news platforms, ubiquitous sensing, commercial data collection, and the very Internet itself mean that we no longer lack for insights even in far-flung corners of the globe. Many questions that once had to be answered by more secretive intelligence collection are now answered with a few clicks on a mobile device.”
“While over-classification is nothing new, the clearance-culture is being abused to maintain employee livelihood”[15] by making and protecting secrets. Open and Commercial Intel can be just as sensitive or valuable as classified. I have seen essentially the same reports, based on the same primary sources, marked Top Secret and independently published by the private sector.
Here is a great contemporary example of two reports written at the same time on the same subject of Chinese interference:
The government’s report from the Independent Special Rapporteur on Foreign Interference[16] relied exclusively on internal classified documents and briefings. The content was principally dedicated to explaining internal government processes, mandates and the Rapporteur’s terms of reference. There is very little of substance about Chinese interference but at substantive cost to taxpayers. Johnson hired two communications and media relations companies[17] to support his investigation.
In contrast, the Murky Waters report[18] was produced by civil society - Alliance Canada Hong Kong (ACHK), using free Open Source Intelligence (OSINT) and is based upon published research, imperial evidence, external experts, informants and the unique insights of the affected diaspora-dissident communities. The analysis was produced at no cost to Canadians.
WARTIME OPEN SOURCE INTELLIGENCE
We have seen an explosion OSINT/CSINT of since Ukraine war, matching nation state capabilities.
In Ukraine, OSINT/CSINT has significant play in everything. Ukrainian military defense operations could have been informed by using CSINT to track the Russian soldiers’ mobile devices that for the last few weeks, were located in Russia and Belarus on the border of Ukraine. Analyzed CSINT could continue to provide insight to Ukraine and its allies to strategically plan deployment of troops and materiel. On the humanitarian side, CSINT can assist relief organizations in understanding areas of crisis and where to position medical, humanitarian and food supplies for maximum impact. More broadly, for Ukraine’s neighbors (Latvia, Estonia, Lithuania, Poland, Moldova, Romania and Hungary), CSINT could bolster analysis and assessments of Russian plans and intentions by filling gaps in the information puzzle.[19] Commercial surveillance and reconnaissance satellites provide daily high-resolution imagery and ELINT of the battlefield.
The Center for Strategic and International Studies explains that “commercial companies have combined advances in microelectronics, small satellites, and low cost to orbit to create proliferated low Earth orbit (LEO) sensing constellations: hundreds or even thousands of satellites in multiple orbital planes allowing for rapid [daily full world] revisit rates, high-resolution images and full motion video.”[20]
This “high-side” model where OSINT is sucked up for “fusion” is a narrow, outdated mindset and limits broad customer, partner, and Allied sharing. As evidenced by the increased demand for quality OSINT from the conflict in Ukraine, broad Allied and partner sharing is required to counter malign influence, spoil operations of our adversaries, and shame criminals masquerading as world leaders. Military intelligence components have been slow to respond and opportunities have been missed to counter narratives during normal operations. Moreover, the IC embeds its OSINT teams within substantially larger classified operations and often reduces OSINT as a source of last resort.[21]
The deputy director of the CIA stated that “commercial/Open Source Intelligence is the INT of first resort, informing every aspect of the intelligence community’s mission.”
ATTRIBUTES OF OSINT
Intelligence-as-a-service is unique, responsive, accurate, timely, actionable and cost-effective. It can be as trustworthy as any classified source, if we apply the same rigor in analysis.
Industry is already using OSINT and commercial sources to do foreign intelligence, counter terrorism, violent-extremism, espionage, proliferation, disinformation. Supporting military operations and targeting. Conducting human rights and war crimes investigations, cyber defence, criminal and legal cases, corporate due diligence and in fundamental research.
Disinformation is uniquely discoverable and countered by OSINT. The private sector has been leading efforts in countering dis-information, foreign influence and interference for decades.
Public OSINT outputs for “shame and blame” campaigns or other information operations are a discipline within itself. This type of marketing skill is unique and constantly evolving and distinct from internal or “behind the firewall.” Inside a closed compartmented environment.[22]
A recent CIA report observed that “The exponentially growing amount of data has overwhelmed conventional analysis tools and made it challenging for traditional organizations to deliver insights fast enough to stay a step ahead of threats. The OSINT attributes of speed, scale and cost are critical to the mission.
OSINT IN CLOSED ENVIRONMENTS
General Michael Hayden, told a crowd at CANSEC[23] that Alphabet (Google), Amazon, Apple, and Meta (Facebook), Microsoft, telecommunications providers and commercial sector data brokers have access to way more data than he ever had while serving as director of both the Central Intelligence Agency (CIA) and the National Security Agency (NSA).
The limited capacity of government remote sensing often failed to keep pace with needs for intelligence, surveillance, and reconnaissance (ISR).[24] The capabilities provided by commercial firms can be used to complement government space systems across a wide-range of national security missions and [can] fill in gaps in capabilities where the government has lagged. Imagery from commercial satellites [is] quickly vacuumed into the Intelligence Community’s (IC) classification vortex of human-base processing. The challenge for the military and intelligence community is [now] understanding how to leverage commercial capabilities for military advantage while protecting national security and maintaining the health and competitiveness of companies in the global remote sensing market.[25] Government regulation the Open Source community would be counter-productive to national security, privacy, peace and prosperity of the country.
There are significant limitations of conducting open intelligence from within closed classified environments, notably:
· Cost-effectiveness;
· Competition against a mature industry;
· Fragmented and compartmented efforts;
· Counter-culture;
· Cognitive-bias;
· Agility;
· Firewalls;
· Operational security;
· Equities;
· Restricted data sets, sources, methods, jurisdictions, mandates and policies; and
· Unclassified investments generally lose to classified dollars from a resource perspective.[26]
Open-Source Intelligence (OSINT) operating concepts in the Intelligence Community (IC) are outdated. While the IC has long cited open source in classified products, treating unclassified information as just another “INT” feeding classified systems is an inadequate model with the future datasphere approaching hundreds of zettabytes and where the most valuable data and analytic technology is coming from outside classified facilities. Re-hosting the Internet, private clouds, and commercial imagery stores on air-gapped systems is unsustainable as data volume increases in zettabyte ranges.[27]
The IC needs to stop trying to covet big data, build there own Google search engines or hyper-scale cloud environments inside classified environments. Leave open information where it is, and apply commercial tools and talent to the problem.
OSINT and the sub-discipline social media intelligence (SOCMINT) are often bound within classified frameworks where flexibility is limited. OSINT must be set free from classified frameworks to grow and flow. The battlespace has shifted more into the open and viewing OSINT as just another part of a classified product is an outdated view that bounds open data creativity and dissemination within the organizational box of systems designed to protect secrets.[28]
While traditional agencies have remained mostly closed environments, commercial intelligence organizations have the benefit of being infinitely scalable using secure cloud computing, crowdsourcing and big data fusion powered by artificial intelligence. Solutions can be used to observe patterns in data at a fast rate and reach more sources than traditional human-driven searches. [29]Commercial intelligence networks offer extraordinary capabilities to enhance nation-state programs. Allies and adversaries successfully outsource intelligence operations to a large degree.
The enemies of democracy and democratic values – are using commercial intelligence as an enabler [to] bolster their strategic advantage.[30]The democratization of data will fuel surveillance capitalism. Ubiquitous technical surveillance (UTS), like China’s SkyNet and SharpEye programs, raises the prospect of a World in which it becomes increasingly difficult to escape the proliferating technologies for wholesale data collection and analysis. OSINT/CSINT and Artificial Intelligence will be seen as both an enabler to UTS and a counter to it.
China has 100,000 open source analysts extracting value from scientific and technical developments globally[31]… a staggering line of effort. This is in addition to the 300,000 members of China’s 50 cent army of trolls producing a 488 million propaganda and disinformation social media posts per year.
Canada’s national security working group[32] recently stated that “secret intelligence is undergoing an existential crisis,” as big-data empowers commercial and open source intelligence (OSINT/CSINT) services for the provision of timely, actionable and cost-effective alternatives accessible to both the public and private sectors.
THE CANADIAN OSINT ECOSYSTEM
Canadian open and commercial intelligence ecosystem and networks are equally impressive, but are doing most work for allies overseas. The Canadian ecosystem consists of: Intelligence-as-a-service providers, investigators, analysts, sources and data brokers, platforms, managed infrastructure-as-a-services, tools and technology vendors. Most of us know each other and have established partnerships.
There is immense value to investing in trusted sovereign industrial capabilities that are entirely consistent with Canadian norms, values and laws. One can get the news from a reputable broadcast corporation, intelligence organization and think tank or you can chase tweets and read the tabloids where there stories are stale, made-up or obtained illegally.
OSINT by the private sector can provide the 99% effective solution to free-up government agencies to focus on the hard final 1% using clandestine means.
CONCLUSION
A US Intelligence agency report says “we should care because, right now, we are in the equivalent of a modern arms race to derive meaning and value from open intelligence, and those who win that race will achieve strategic advantage.”
CSINT could provide previously unknown, needed, pieces of the puzzle to solve complex investigations in the supply chain, disinformation, and counter-espionage realms, thereby protecting proprietary information, critical national infrastructure, and the very fabric of our precious democracy.[33]
Chris Rasmussen argues convincingly for the establishment of an independent open source center in his brief Avoiding the Secrecy Trap in Open Source Intelligence, [34] 21 March 2023. While I strongly agree with pretty much everything, I would suggest that creating government-centric organization for OSINT will not work in the Canadian context.
Such a centre would immediately fall under the same government mandates, policies, legal restrictions, culture and bureaucracy, with limited effectiveness while competing against a mature private sector market with tax dollars. We have lived this experienced in other domains.
I suggest that a centre be either a crown corporation similar to the Canadian Broadcast Corporation (CBC) with its own act or a government funded independent non-government organization like the Canadian Centre for Child Protection (C3P) or the Canadian Cyber Threat Exchange (CCTX). Furthermore, at least half the budget ought to be outsourced to Canadian industry for data, products and intelligence services to support a sovereign capability.
National Security and Prosperity is a team sport and what is clear is that industry and government need to collaborate intentionally. But there needs a contract vehicle for government to work with trusted Canadian providers.
[1] US intelligence is only as good as what goes into it the cyber brief by Cynthia, Saddy Eunjoo “ej” Alam and Kelli Holden march 16th, 2022 https://www.thecipherbrief.com/us-intelligence-is-only-as-good-as-what-goes-into-it [2] https://www.rand.org/blog/2021/04/the-intelligence-communitys-deadly-bias-toward-classified.html [3] https://www.thecipherbrief.com/column_article/avoiding-the-secrecy-trap-in-open-source-intelligence [4] US intelligence is only as good as what goes into it the cyber brief by Cynthia, Saddy Eunjoo “ej” Alam and Kelli Holden march 16th, 2022 https://www.thecipherbrief.com/us-intelligence-is-only-as-good-as-what-goes-into-it [5] The role of AI in open source intelligence By McDaniel Wicker and Patrick Butler, Babel Street, 25 Jan 2022 [6] The role of AI in open source intelligence By McDaniel Wicker and Patrick Butler, Babel Street, 25 Jan 2022 [8] Personal Information Protection and Electronic Documents Act (PIPEDA) is the federal privacy law for private-sector organizations. It sets out the ground rules for how businesses must handle personal information in the course of their commercial activity https://laws-lois.justice.gc.ca/eng/regulations/SOR-2001-7/index.html [9] Today’s AI-enabled [commercial capabilities] empower intelligence analysts to leverage OSINT [to rapidly] uncover hidden threats, and pinpoint the targets. [9] [10] https://www.sapperlabs.com/post/the-rise-of-commercial-intelligence [11] The role of AI in open source intelligence By McDaniel Wicker and Patrick Butler, Babel Street, 25 Jan 2022 [12] US intelligence is only as good as what goes into it the cyber brief by Cynthia, Saddy Eunjoo “ej” Alam and Kelli Holden march 16th, 2022 https://www.thecipherbrief.com/us-intelligence-is-only-as-good-as-what-goes-into-it [13] The role of AI in open source intelligence By McDaniel Wicker and Patrick Butler, Babel Street, 25 Jan 2022 [14] https://www.thecipherbrief.com/column_article/avoiding-the-secrecy-trap-in-open-source-intelligence [15] https://www.thecipherbrief.com/column_article/avoiding-the-secrecy-trap-in-open-source-intelligence [16] https://www.canada.ca/en/democratic-institutions/services/reports/first-report-david-johnston-independent-special-rapporteur-foreign-interference.html [17] https://www.cbc.ca/news/politics/special-rapporteur-david-johnston-crisis-communications-firm-1.6864312 [18] https://alliancecanadahk.com/wp-content/uploads/2023/05/ACHK_Murky_Waters_Bejings_Influence_in_Canadian_Democratic_and.pdf [19] US intelligence is only as good as what goes into it the cyber brief by Cynthia, Saddy Eunjoo “ej” Alam and Kelli Holden march 16th, 2022 https://www.thecipherbrief.com/us-intelligence-is-only-as-good-as-what-goes-into-it [20] https://www.csis.org/analysis/commercial-space-remote-sensing-and-its-role-national-security [21] https://www.thecipherbrief.com/column_article/avoiding-the-secrecy-trap-in-open-source-intelligence [22] https://www.thecipherbrief.com/column_article/avoiding-the-secrecy-trap-in-open-source-intelligence [23] CANSEC is Canada's global defence and security trade show hosted annually in Ottawa since 1998. It is hosted by the Canadian Association of Defence and Security Industries (CADSI), [24] https://www.csis.org/analysis/commercial-space-remote-sensing-and-its-role-national-security [25] https://www.csis.org/analysis/commercial-space-remote-sensing-and-its-role-national-security [26] https://www.thecipherbrief.com/column_article/avoiding-the-secrecy-trap-in-open-source-intelligence [27] https://www.thecipherbrief.com/column_article/avoiding-the-secrecy-trap-in-open-source-intelligence [28] https://www.thecipherbrief.com/column_article/avoiding-the-secrecy-trap-in-open-source-intelligence [29] Today’s AI-enabled [commercial capabilities] empower intelligence analysts to leverage OSINT [to rapidly] uncover hidden threats, and pinpoint the targets. [29] [30] US intelligence is only as good as what goes into it the cyber brief by Cynthia, Saddy Eunjoo “ej” Alam and Kelli Holden march 16th, 2022 https://www.thecipherbrief.com/us-intelligence-is-only-as-good-as-what-goes-into-it [31] https://www.thecipherbrief.com/column_article/avoiding-the-secrecy-trap-in-open-source-intelligence [32] https://www.cigionline.org/activities/reimagining-canadian-national-security-strategy/ [33] US intelligence is only as good as what goes into it the cyber brief by Cynthia, Saddy Eunjoo “ej” Alam and Kelli Holden march 16th, 2022 https://www.thecipherbrief.com/us-intelligence-is-only-as-good-as-what-goes-into-it [34] https://www.thecipherbrief.com/column_article/avoiding-the-secrecy-trap-in-open-source-intelligence