Creating Intelligence as a Community

| January 2022

The Intelligence Community plays a vital role in defense of the Nation. They are entrusted with accurately collecting, analyzing, and delivering foreign intelligence and counterintelligence information for America’s leaders who, in turn, make sound decisions to protect the country.[1] The IC conducts decision support by connecting, inferring meaning from, and making analytic judgments about disparate data sets.[2] However, this is easier said than done when the sheer magnitude of data points available today can overwhelm a human’s ability to review and analyze it in a timely manner. Advanced technologies such as Artificial Intelligence, supercomputing, and 5G can help, but for various reasons addressed in this report, it often takes too long to employ emerging technology rapidly throughout the IC. [3]

To compound the issue, the IC is made up of separate agencies with countless stove-piped systems and serial processes. Stovepipes challenge the IC’s collective ability to understand and find the most relevant data to make analytic judgments, while countries like China are rapidly advancing their Artificial Intelligence techniques by employing whole-of-nation strategies.[4]

“For decades, the IC’s traditional space platforms were designed to address the Cold War. For the most part, the Soviet Union was the single adversary, and over the years, the IC came to understand the landscape very well. However, by today’s standards, keeping tabs on the Soviets was simple. In those days, the IC focused on watching and listening to big, loud things that moved slowly, ICBMs, tank divisions, and fixed radars. It was a job they did very well; it was also a job that drove the requirements for the IC’s capabilities and systems for decades (The NRO Overview).” [5]

Now we are in a race to modernize. The IC must move quickly to extend its Cold War success into the next phase of competition or risk losing the innovation war to China.[6]

1a. Research Methodology

The information for this report was captured through interviews with government and contractor employees from the IC and DoD, reading open-source articles published by Intelligence think tanks and R&D organizations, and participating in panels, webinars, and seminars with current and former IC members.  Throughout the research process, a clear message emerged:  The IC must work with the Private Sector to modernize. For decades the IC agencies have produced vital intelligence to support decision-makers.  While tremendously successful, we must not pause. The intelligence produced using processes and technology suited for integration at human speed cannot keep up with the overwhelming amounts of data and cyber-based assaults present today.

This research paper examines areas where the IC can focus to better produce intelligence as a community. Chapter one asserts that with a purposeful effort to align key intelligence processes and with rapid access to emerging technology, the IC will be able to produce intelligence and fulfill its mission at the speed of innovation. Breaking down organizational and data silos will catapult data sharing and allow for the full adoption of AI techniques across the community.  Chapter two advocates for an orchestration function at the DNI to unite the community and guide the agencies to effectively reimagine the intelligence production processes.  Chapter three asserts that swift access to emerging technologies is critical to effectively reinventing the intelligence process. It addresses three key areas where challenges occur: the processes used to acquire technology, the skill-sets of the people who put the tech to use, and the Assessment and Authorization (A&A) process used to secure it. Chapter four wraps up the paper and provides the research summary.

Chapter 2.  Priming the IC to Perform at Enterprise Scale

“Defending against AI-capable adversaries without employing AI is an invitation to disaster. AI will compress decision time frames from minutes to seconds, expand the scale of attacks, and demand responses that will tax the limits of human cognition. Human operators will not be able to defend against AI-enabled cyber or disinformation attacks, drone swarms, or missile attacks without the assistance of AI-enabled machines. The best human operator cannot defend against multiple machines making thousands of maneuvers per second potentially moving at hypersonic speeds and orchestrated by AI across domains. Humans cannot be everywhere at once, but software can (National Security Commission on Artificial Intelligence)."[7]

2a. Analysis

In 2017, the IC released the Office of the Direct of National Intelligence (ODNI) Information Environment (IE) Data Strategy.[8] The strategy, authored jointly by the IC Chief Data Officer and the IC Information Sharing and Safeguarding Executive, calls for IC agencies to change how they manage and share data with the goal of “removing its dependencies on IC element applications, systems and databases, thus allowing it to be cataloged, self-described, and discoverable by automated means.”[9] It describes how the IC’s vast collection of intelligence information is critical to producing timely, accurate, thorough, and integrated intelligence insight; but success can only be declared if data sharing happens at all levels.

To accomplish data centricity, the IC agencies must create both a culture and an architecture for sharing data and the first step to doing that is to break down the silos that exist within and between each agency.

The Intelligence Community comprises 18 organizations that, in some cases, are siloed to the point of inefficiency.[10][11] The IC was not intentionally siloed, but, over time, as new agencies were created to address specific intelligence threats, they were later retrofitted into the IC. Since its inception in 1981, the IC has added six new organizations to its Rolodex.[12] A recent example of this is the US Space Force (USSF). The USSF was established in 2019, and during its creation, concluded that intelligence activities would be a crucial part of its mission.[13] In a press release regarding the addition, the Director of National Intelligence, John Ratcliffe, stated: “Through sharing space-related information and intelligence, the IC and DoD increase integration and coordination of our intelligence activities to achieve best effect and value in executing our missions.”[14] Later, an ODNI spokesperson stated that making the Space Force a member of the intelligence community “breaks down barriers to information sharing and ensures that Space Force leadership has access to all the intelligence it needs to be successful.”[15] If only it were that easy. Like other organizations who have joined the IC, the USSF did not start with a blank slate. Most USSF activities have been in existence for decades under the US Air Force moniker who already had systems, processes, and governance in place. The systems and procedures did not go away with the stand-up of the USSF and membership within the IC; they were simply given a new label. Resultantly, the IC will not break down information sharing barriers until it becomes intentional about what it means to be a community.  How does the IC get to that ubiquitous data access and intelligence sharing that the DNI mentioned? [16]  First, break down the silos.

2b: What are Organizational Silos and Why are They Bad for the IC?

Organizational Silos exist when agencies have different chains of command, processes, and funding streams. Organizational silos become problematic when trying to work across agencies to accomplish a joint mission. Since 9/11, the IC has worked tirelessly to create opportunities for collaboration. Still, with 18 separate silos, no matter how hard individuals and groups work, their efforts will produce only pockets of success.  Below are ways organizational silos promote an unhealthy culture and slow the decision process between organizations and agencies. [17]

Organizational Silos

Lead to a Culture of Blame. In many cases, humans blame other humans for failures when the issues are with the system we have created.[18]

Create Longer Lead Times. The IC’s hierarchical organizations are not conducive to delivering value horizontally; they are optimized to keep decisions flowing up. Because innovation and data sharing require quick-turn decisions across the enterprise, the IC often wastes time requiring leadership to make decisions when the people doing the work are qualified to make them.[19]

Breed Unproductive Competition.  Data is the new oil, but knowledge is still power.  Each agency collects processes and reports on the data they own. Being the first to report a groundbreaking story is a feather in each agency’s proverbial cap for the analyst and the platforms that collected the data. The incentive to be first creates individual agency cultures counter-productive to realizing large-scale data analysis promises. [20]



Text Box: Figure 2 Digital Workflow DefinitionText Box: Definition: A digital workflow is the automation of processes and the internal steps needed to get work done…

Break down organizational silos by creating an orchestration function at the DNI level to reimagine critical IC processes. “Effectively operating as one team: synchronizing collection, analysis, and counterintelligence so they are fused...”[21]depends on the IC jointly creating the next revolution of intelligence processes as digital workflows. By creating digital workflows, the IC can eliminate the stove-piped nature of the intelligence process, dynamically change tasking as the requirements change, measure value, accurately trace back to national priorities, and reduce today’s time-wasting bottlenecks.  Below are the steps to begin the process.

Step 1.  Garner support and buy-in by creating a coalition of tech leaders from each agency who meet with key stakeholders to get clear about customers’ requirements. Using the feedback from the stakeholders, this coalition can identify specific processes that will most benefit from modernization (e.g., reimagining TCPED using AI). [22]

Step 2.  Focus on value and outcomes for those processes.  Organize the functions into portfolios focused on delivering value and outcomes vs. delivering systems. Fund the value stream – not the organization. [23]

Step 3. Apply systems thinking. The intelligence process is made up of a myriad of complex, interrelated environments, systems, and capabilities.  Applying systems thinking will allow for a comprehensive look at inputs and outputs and help the IC build their understanding of the areas ripe for change and achieve greater effectiveness.[24]

Step 4.  Create everything in software. Creating as much as possible in software reduces timelines to develop, secure, and test capability, therefore producing value faster during the development process. Additionally, smaller, more modular software services are more easily updated when the mission demands and streamline security processes.

Step 5.  Provide transparency. Identify Stakeholders and customers to participate in value stream portfolio reviews to consistently provide feedback, iterate on improvements, change course when necessary and take on new requirements.

2c. What are Data Silos, and Why are They an Issue for the IC?

Data silos are storage systems, applications, and databases whose infrastructure or governance, prohibits an outside group or system from accessing the data.[25] The IC collectively has thousands of systems and databases, each with the potential to be its own data silo. Below are some of the reasons the IC must break down data silos.  

Data Silos

Prohibit organizations from gleaning deep, actionable insights from data and create a barrier to a holistic view of the threat. To realize the full benefits of AI, Machine Learning, and data analysis, the IC needs a 360-degree view of the data.

Are costly because when data is shared, it is often done by making copies, creating enormous duplicates of the same data to house in the cloud or a data center. [26]

Are roadblocks for AI. AI, Machine Learning (ML), and other types of automation need authoritative data to guarantee accurate predictions. Creating and maintaining numerous copies of data is risky because each copy is out of date the minute it is produced and could alter the AI or ML prediction accuracy.

The Intelligence Community Enterprise collects more data faster and generates insights requiring an approach that breaks down data silos to ensure accurate data is readily available. The IC needs to focus beyond collection and retention and emphasize architecture, management, and curation.” (Breaking Silos and Curating Data for Impactful AI) [27]


Enable Data Sharing with an IC-wide Data Mesh. As part of the IC’s push to modernize, they procured the government’s first commercial cloud.[28][29] The cloud provided IC agencies with the common infrastructure platform necessary to break down data stovepipes. Initially, both the private and public sectors assumed that to share data it was necessary to consolidate it into centrally managed data lakes and data warehouses.[30] However, security rules, data governance regulations, and differing data schemas prevented the use of consolidated IC data lakes to house multi-agency data. Fortunately, there is a new data sharing architecture that can bridge virtual stovepipes  - it is called a data mesh.

What is a Data Mesh?

A data mesh is an architectural overlay that provides access to an infinite number of distributed data nodes while at the same time ensuring that data is secure. Invented by Zhamak Dehghani, a data mesh builds a layer of connectivity that takes away the complexity of managing and supporting data access. It fastens together data spread across silos, organizations, and locations, both on and off-premises.[31] It creates an environment for Machine Learning and data fusion by pulling data from lakes and warehouses without consolidating them, changing the schema, or underlying hardware. The following graphs depict JP Morgan’s successful implementation of a data mesh architecture for illustration.


A data mesh provides the necessary mechanisms for sharing but allows each agency to manage their data according to applicable laws and governance rules.[32]

Text Box: Figure 5 Data Mesh IllustrationDiagram

Description automatically generated

Each IC agency has different laws and governance regarding data management. Additionally, the systems, applications, formats, databases, warehouses, and data lakes, housing the data vary in location and type. Using a data mesh architecture allows the IC to share data without requiring each agency to consolidate their data into one location as initially envisioned. In the JP Morgan picture, the business processes, operational systems, and data lakes all reside within each agency’s purview. A data lake catalog will advertise data type and availability to the mesh catalog, and the consuming analyst or system will have access based on their credentials.

A data mesh provides an authoritative source for data and a way to trace data through the system.[33]

The data mesh allows for sharing data from each lake rather than copying it to the consumer applications. This way, the data owner has confidence that the most current authoritative data is being used in reports and Machine Learning predictions.  And because the data never leaves the data owners’ lake, there is no additional cost for storing copies. This data mesh’s fine-grained access control mechanisms can restrict visibility down to specific columns, records, and even individual values, so consumers and systems only get what they need to perform their function. Data owners have a way to see who is requesting the data and where it is going in real-time. A data mesh architecture would put into action the plan to break down data silos.

Key Section Takeaways: 

Create Agile teams and employ DevSecOps and MLOps across the enterprise.  Contracts, finance, and security, testers, coders, and data engineers are all equal members of the Agile team. Creating Agile teams eliminates artificial roadblocks, builds trust, and decreases lengthy handoffs. Agile teams ought to have regular access to stakeholders.[34][35]

Create intentional collaboration across the community. In small pockets of the IC, teams have successfully applied Agile practices. However, to gain value for the size and complexity of government system development, the IC requires more synchronization of effort than team-level Agile practices can provide. Applying Scaled Agile Framework to the cross-community efforts is a way for leadership to conduct organizational Strategic Planning and Agile execution at all levels.[36]

Democratize data. A data mesh creates a way for data to be made available at the lowest point of consumption before INT-specific processing. As a result, INT agnostic data analysts can glean new insight from this data that INT-specific analysis may not provide. 

Incentivize the data owner to share. The IC should create an awards program that incentivizes data owners to make their data available to as broad an audience as legally allowed.  Cloud services and a data mesh architecture enable data to be traced to the consumer and beyond, allowing the IC to determine what data is being used, how it is used, and the most requested data.  This additional insight will provide data owners a way to begin measuring value.

Require a data plan as part of the acquisition process. Incentivize system owners to detail how they will make the data discoverable in a Data Plan.

  1. Map the data plan back to the overarching IC Value Stream to see where/how it adds value.[37]
  2. Confirm that, as part of the acquisition process, the system has built-in ways to measure the use of data.

Conclusion.  Breaking down organizational and data silos are critical to the IC’s success moving forward.  Reimagining the IC’s critical processes and solidifying the new processes into automated machine to machine digital workflows will allow the IC to more rapidly incorporate changes and stay ahead of the dynamic environment.

Chapter 3. What Hinders the Rapid Adoption of Emerging Technology?


Technology is changing at an eye-watering pace. From autopilot-driven cars to robotics, technology has revolutionized our everyday lives. For decades, the government has been the benchmark for technological advancements. Countless breakthroughs come to mind, from the advent of the Internet to the creation of Space Reconnaissance. Nevertheless, over the years, the Private Sector has catapulted the Government in the creation of cutting-edge technology. [38] This is due to many factors: the exodus of talent to industry, budget cuts in R&D, and increased spending on operations and maintenance (O&M); each have left less room for innovation. [39]

How did we get here? In 1964 the US spent 1.6% of the GDP on R&D, but by 1994, it had dropped to .83%. The DoD’s funding for R&D has been declining since the early 2000s.[40] During the same timeframe, the Private Sector’s investment in R&D has doubled.[41] At first, it did not seem that the Private Sector investment had anything to do with the Intelligence Community. What the IC did was unique - created behind concrete walls and cipher-locked doors. Nevertheless, slowly, over time, the private sector began to produce technology that looked like versions of what the IC had, but markedly more flexible, compatible, and innovative.[42]  Today, the private sector is the US’s largest investor in R&D.[43] As a result, the IC should focus on partnering with industry to find ways to adopt commercially available technology more rapidly in areas where it makes sense rather than attempting to recreate it.

Artificial Intelligence Tools Require Modern and Available Infrastructure. The Government should adopt and common platforms from Private Industry wherever possible.

“… they kept hiring more developers and kept doing modern processes, agile development processes…but didn’t invest in fundamental infrastructure you needed to make software developers effective. They kept pouring money into it and getting less out,” said Kersten. “They lost the entire mobile market. If, at a leadership level, the approach doesn’t change, we’re going to see the Nokia story repeat itself for the large car brands we know today.” (Mik Kersten on the downfall of Nokia)[44]


In the Private Sector, software developers, data engineers, and researchers enjoy working with cloud platforms that are, with a swipe of a credit card, up and running in minutes. Dozens of AI tools are a mouse click away, massive datasets are imported seamlessly from the web, and microservices are shared across the developer community through hubs and channels made for collaboration. However, when these same people go to work in the IC, they are often provided with outdated or custom IT solutions that require significant modifications to provide functionality. Approvals to download software take months, and due to contractual rules, most microservices are not shared across the development community. Not to mention, it often takes weeks or months for the IT shop to provide access to development tools.[45]

To win the innovation war, the Intelligence Community cadre requires ready access to modern tools at the speed of innovation that their counterparts in the Private Sector have been using for years. The sub-chapters that follow address the roadblocks and provide recommended solutions for the IC to adopt commercially available technology at the speed of the market.

Chapter 3a. Acquiring Tech takes too long


Many Government program managers are hesitant to be innovative with the acquisitions process.  The Federal Acquisition Regulation (FAR) is a set of documents that govern procurement in much of the IC.[46] For decades, the FAR has been a source of frustration for Government Program Managers (GPM).  Many find it lengthy and hard to interpret; written in language that will stand up to litigation but challenging for general program managers to navigate and implement. Because of this, many GPMs use templated processes and approaches that have been successful in past acquisitions, but are not necessarily innovative, often locking the government and industry into lengthy non-productive contracts. [47]

Acquisition teams focus on requirements and not outcomes. As a result, the Request for Proposal (RFP) to industry contains strict requirements and solution-oriented language that box both the contractor and government into a solution that will be outdated by the time it is put on contract.[48]

The type and length of contracts cause unnecessary churn. GPMs typically acquire new technology in one of two ways. In the first scenario, contracts are executed in lengthy, prescriptive language that promote a decades-long relationships with technology that, in many cases will outlived its usefulness before the contract end. In the second scenario, thriving contracts are re-competed at artificial milestones due to ill-applied acquisition rules. In most cases, the government has invested significant time in training, documenting, securing, and advocating for a capability only to find, just as the effort is becoming fruitful, the agency is required to recompete the contract and start over, losing critical expertise and functionality they desperately need. In both scenarios, the contractors gain technical skills in cutting-edge technology, while the Government manages at the periphery.  This relationship keeps the Government reliant on contractors and in some cases, unable to accurately recognize how to apply technology appropriately.


Write contracts that focus on outcomes rather than adherence to requirements. This way, when the technology changes or feedback from stakeholders and users pushes the development in a different direction, the development team can easily change course. 

Stand-up government software factories containing a hybrid of government and contractors.  Consider a hybrid approach to contracting with a mix of government, contracted Level of Effort (LoE) cleared “gig workers” and service contracts. Incorporate Agile development methodologies into each contract. Require full Government data rights to everything developed.

Require Government employees to obtain technology certifications like AWS cloud practitioner, Scaled Agile Framework, DevSecOps, Agile, and PMP for government personnel.  It is not enough to require this for just IT employees.  Today’s IC runs on technology; therefore, everyone from HR to Management should understand the opportunities and risks of employing emerging technology.

Bolster DAU’s offerings to include courses on Agile, SAFe, technology, data, and operationalizing innovation.

Reward Government Program Managers for adopting commercially available technology and incorporating tools and products that are already available across the IC enterprise; disincentivize the creation of new tools that are otherwise available.

Continue to build upon the success of the IC Cloud acquisition. The IC was successful in contracting with the industry-leading cloud provider versus building something custom. This put cutting-edge technology into data analysts’ and engineers’ hands faster than following the traditional procurement of custom technology processes. 

Conclusion. Many Program Managers in the IC believe The Federal Acquisition Regulation is too restrictive, and its rules are holding the IC back from true innovation.  However, some of the IC’s most extensive and innovative capabilities have been procured using basic FAR rules. Hiring more technically savvy Program Managers and incentivizing them to look beyond the templated traditional acquisition processes and to hybrid and agile software factories will bring the IC closer to commercial success without rewriting one word of governance.

Chapter 3b: The Government Grows Talent; Industry Hires It

Attractive incentives, innovative work environments, and risk-tolerant culture have lured talent away from government service and into the private sector. Now, the government struggles to recruit and retain the skill sets required to innovate at industry speed. Many critical skill sets such as data analysts and computer scientists have moved to industry or are resident only in small pockets of the IC’s Research and Development (R&D) organizations.[49] This is due to several factors: issues with recruiting, organizational culture, and retention.

Recruiting. The hiring and clearance processes are lengthy and stringent. In 2018, OPM reported it took an average of 98.3 days to hire new employees.  Once hired, an employee can expect to wait from 8 to 15 months for a Top Secret clearance.[50] Many talented applicants who interview for government jobs end up taking positions elsewhere in industry because they cannot wait a year or more for the process to take place. The second is the pay. The private sector heavily recruits applicants who have technical skills, offering excellent pay and benefits. Not to mention, for the most part, the private sector work environment is relaxed, welcoming, and inclusive, without rigid organizational structures. People spend a significant amount of time at work, so they want it to be a good experience. The private sector has struck a balance, meaningful work, good pay, and an emphasis on interpersonal relationships.  

Culture. Once a candidate successfully navigates the hiring process and is an IC employee, they are incentivized to conduct daily operations. Their performance requirements are set at the beginning of the year and a rigid box-checking exercise at year-end to enforce adherence; compliance is rewarded. The chain-of-command culture of the government has created a system where employees are required to gain approval for even the smallest of changes.  Employing self-starters and innovative thinkers and then requiring them to follow rigid approval processes, creates a culture of stagnant operations - not cutting-edge innovations. Lastly, the tech industry is known for offering their employees office perks like free lunch, meditation rooms, nap pods, gym memberships, and policies that allow one to bring their dog to work.  These incentives, coupled with an emphasis on interpersonal relationships, incentivize employees to work longer hours and remain loyal to the company for years longer than expected.[51]

Retention. The private sector requires technical skills on day one. They are less interested in growing talent because stakeholders demand a near-instant return on investment, and employees need to produce value quickly. On the other hand, the IC is much more likely to hire junior people with fewer skills and develop them over time. Hiring lower-skilled tech employees is a challenge to government organizations trying to innovate at the speed of industry; in most cases, the employees just are not at the level of their peers in industry.  Then, when the junior government employee garners enough experience and training, the private sector will more than likely hire them away for better pay and benefits. 


Flatten organizations in key areas. Empower and trust employees to communicate, collaborate and make decisions across organizational boundaries. If people with the right skills are in the correct positions and empowered to make decisions, the organization will benefit from faster decision-making at all necessary levels. Employees will feel empowered, fulfilled, and the organization will gain speed and agility.

Bid For talent. Building upon Tromblay’s study “Silos and Sand” the IC has an opportunity to “unburden its workforce from the hierarchical red-tape of the org chart and change the way it finds the right people for the job. Turning sand into mortar requires a departure from the typical top-down structure of large bureaucracies. Agencies should not assign employees to a single manager. Rather, managers should function as customers, who must “bid” for talent. Although the government is not structured to provide monetary incentives, managers can offer meaningful work and cultures conducive to employees’ professional growth. This structure would in relatively short order identify which managers and employees were effective in enhancing an agency’s organizational integrity. Effective managers would need to attract employees capable of bringing projects to successful conclusions or effectively implementing ongoing functions. Employees, in turn, would have to demonstrate value in order to earn desirable assignments. As employees cohere around managers, based on expertise and competency, organizations could more easily determine which personnel contributed to organizations’ strengths, and which personnel could be swept aside like so many grains of loose sand (Tromblay, 2019).” [52]

Save the org chart for administrative purposes. Create cross-organizational teams based on creating value, and fund the value stream, not the systems. Leave the org chart in place for executive functions like performance reports.[53]

Create a culture of psychological safety that will benefit both the organization and its members.[54] For example, freedom from humiliation for mistakes will allow for better ideas to come forward faster and ensure errors are reported before they worsen.

Require IC applicants to have technology certifications, boot camps, and high school/university coursework in technology fields but rethink four-year degrees as a requirement for entry. Some of the brightest minds in technology today did not go to college, and some never graduated high school. During hiring events, look for portfolios of works accomplished versus formal degrees.

Show people that their expertise is valuable, regardless of their grade. To do this, an agency must make its personnel feel as though they are using – and appreciated for - the expertise that they bring to their roles. “Agencies can combat retention challenges by creating conditions that facilitate the emergence of knowledge from across their ranks.”[55] Organizations like the Air Force’s Kessel Run have successfully employed this model. Kessel Run is a government software factory that employees a hybrid of contractors and government personnel.  No one wears a uniform, no ranks or titles are used, and each person has an equal say in decisions within their domain of expertise.[56]

Create a better bridge between the government and the private sector. Employees’ skills and knowledge can go stale quickly, especially since innovation is increasingly the private sector’s domain.  A handful of agencies have attempted to address this reality by allowing their employees to serve stints in the private sector – a practice that can help the public sector better understand the government while also ensuring a corps of government personnel keep their skills relevant.[57][58]

Conclusion. Industry’s promise of attractive incentives has enticed technologists away from government service. As a result, the government now has fewer experts to bring emerging technology into the IC. For that reason, the government may be less likely to expertly navigate tough decisions regarding emerging technology acquisition and operations. Their reliance on single solution contracts for their expertise leads to stove-piped and custom solutions that inhibit data sharing throughout the IC. To bring technologists back to the IC, the government must provide them with ready access to cutting-edge technology, and data while creating an environment where taking an informed risk is encouraged and unpunished.

Chapter 3c:  Upgrading Cybersecurity

The IC is duty-bound to protect America’s secrets at all costs or risk causing damage to the country and its allies. The IC should be applauded for its work keeping America’s most important information safe from exploitation. However, there must be a balance between security and innovation.

Security is critical – But should not be the antithesis to innovation. The IC procures hundreds of systems each year. After a years-long acquisition and system engineering process, the government installs and secures mission-critical equipment in air-gapped data centers. The data centers have millions of square feet of compute, storage, and network equipment. Each piece has been hardened, secured, and granted an Authority To Operate (ATO) - a year-long endeavor that begins only after the procurement and engineering is complete. By the time the systems are purchased, installed, and secured, they are several years old. Not only is the technology out of date but so is the support and warranty.[59]  The National Institute of Science and Technology Risk Management Framework process is conducted by assessing the risk, applying the hundreds of requirements to the engineering backlog, and creating security documents to assessors at various milestones along the engineering process.[60] Once the paperwork is approved and Authority to Operate is granted, any tweak to the system requires updating the volumes of documentation. This places a significant burden on both the engineers and the security assessors. Then, each time the NIST requirements (called “controls”) are updated, systems engineered under the old requirements must determine how to incorporate the new mandates or risk being non-compliant.

Securing commercially available technology is not easy. Most features of commercial technology are meant to be used on the public Internet with connectivity to other services and freely connected backend capabilities (camera, storage, wireless, etc.). But due to security requirements, many innovative parts of commercial technology are disabled to be secure. Therefore, the IC buys items and disables all but a small percentage of the functionality.

Security assessors may not have the skills to assess emerging technology. As a result, innovative ideas for securing mission systems are often disapproved, further denying the use of innovative commercial capabilities to accelerate the ICs mission.


Hire security people who can code and put them on the Agile development team. Entrust them with securing the system. Couple engineering requirements and security requirements into the same deliveries to ensure a fully functioning, secure system is delivered.

Text Box: Figure 7 Incremental engineering and security drops deliver value to the user faster than the traditional Waterfall process that only delivers secuirty and value at the end of the project.

Break software into smaller microservices with fewer lines of code and deliver value incrementally. This will allow both functional and security testers to ascertain and fix issues quickly and iteratively. It is much easier to determine where the errors are with small pieces of code than to run tests on millions of lines, only to find the fix that was put in place impacted other parts of the code.


A screenshot of a computer

Description automatically generated with low confidence

Automate the security process. Modern infrastructure platforms such as the cloud provide security professionals with the ability to spin-up infrastructure environments using NIST 800-53 requirements and security overlays in minutes, reducing months of manual configuring and human observations for compliance.  When anomalies in the pre-arranged environments are detected, security professionals can have the system automatically correct the issue, report it to a higher-level dashboard, or take specific action to shut down, or change configurations to protect data from intrusion. 

Conclusion. The IC has a duty to protect the Nation's classified domain, a duty they take seriously, and execute flawlessly.  Innovation in the cybersecurity realm has been challenging for many reasons. Due to outdated governance, training, and education, cybersecurity professionals are often unable to use automated means and must rely on manual processes to detect anomalies. Many security professionals sit outside of the process, and the handoffs between engineering and security create an unnecessary lag in getting mission-critical capability to the analyst.  Using automated means and adopting an iterative security engineering process putting engineers and operators together to deliver secure, innovative capabilities can quickly secure our national assets at the speed necessary to meet the mission.

Chapter 4. Research Summary:

In 2004, President Bush signed four executive orders, reforming the IC, and creating the Director of National Intelligence to “integrate foreign, military, and domestic intelligence in defense of the homeland and in support of United States national security interests at home and abroad.”[61] During the signing of the document, he declared, “Under this new law, our vast intelligence enterprise will become more unified, coordinated and effective. It will enable us to better do our duty, which is to protect the American people.” [62]

While the Executive Orders significantly reformed the Intelligence Community, both the threat and opportunity landscapes have changed considerably since the signing. The President’s desire for an intelligence integration function at the DNI level has not yet been fully realized. IC agencies are doing everything humanly possible to create intelligence as a community while the machines that can revolutionize the intelligence process are standing by in the private sector, unable to be utilized for our most challenging intelligence problems due to bottlenecks in the acquisition, talent management, and security processes. 

Fortunately, the IC can overcome the hurdles presented in this research paper by taking several actions.  The first is to get purposeful about modernizing key intelligence processes. The DNI should create an empowered cross-agency consortium focused on reimagining the intelligence process from a values-based perspective, using a strategic framework to become more agile. This framework will enable multiple cross-agency teams to create the next generation of intelligence processes seamlessly.

The second recommendation is to implement a data mesh architecture across the IC’s vast data silos, ensuring the IC can effectively employ Artificial Intelligence techniques while allowing each agency to govern the data according to the provided guidance.

Third, the IC must have a cutting-edge technical foundation by which to create fused intelligence. Therefore, acquisitions must become faster, employees must become more tech-savvy, and cyber defense procedures must become more ingrained with the engineering processes to speed approvals. To accomplish this, Government Program Managers should look outside the templated, requirements-based acquisitions and award outcome-based contracts to account for the rapidly changing requirements present in nearly all procurements.

Fourth, technology is no longer something that belongs in the basement of the IT shop; it is as ubiquitous as the desks in which we sit, so all personnel in the IC must become conversant in it. The IC can accomplish this by providing education to the current workforce while recruiting and retaining additional tech-savvy personnel in ways that have been successful in the private sector. Maintaining innovative and physiologically safe environments will entice experts and build trust, therefore allowing managers to recruit and retain tech talent.

Finally, hiring security engineers to develop innovative cyber defense capabilities alongside the functional engineers and automating much of the A&A processes will make certain the IC gets emerging technology to operations faster.

At this moment, the IC is looking at one of the most significant opportunities in its history; to examine and improve itself before disaster necessitates. The opportunity to bring in the best process, people, and technology to create intelligence as a community and stay ahead of the adversary is well within our reach.   Will the DNI lead us?

Table of Figures

Page 7: Figure 1 – Intelligence Community Silos: Scaled Agile Framework. n.d. Scaled Agile Framework. (Modified)

Page 8: Figure 2 - Data Silos Illustration: Unknown. n.d.….….

Page 10: Figure 3: Digital Workflows Definition: Digital Workflow. n.d. “What is a Digital Workflow?”

Page 12: Figure 4: Amazon Web Services Data Mesh Example: AWS for Industries. 05 January 2021. “How Cloud-based Data Mesh Technology Can Enable Financial Regulatory Data Collection.”

Page 13: Figure 5: Data Mesh Illustration: Anu Jain, Graham Person, Paul conroy and Nivas Shankar. 2021. "How JPMorgan Chase built a data mesh architecture to drive significant value to enhance their enterprise data platform." AWS Blogs. May 05.….

Page 14: Figure 6: Data Mesh in Action: Anu Jain, Graham Person, Paul conroy and Nivas Shankar. 2021. "How JPMorgan Chase built a data mesh architecture to drive significant value to enhance their enterprise data platform." AWS Blogs. May 05.

Page 30: Figure 7: Incremental engineering and security drops: Scaled Agile Framework. n.d. Scaled Agile Framework.

[1] DNI. n.d. "Intelligence Community IT Enterprise."

[2] The Director of National Intelligence. n.d. The AIM Initiative. A Strategy for Augmenting Intelligence Using Machines. Accessed December 12, 2020.….

[3] The Director of National Intelligence. n.d. The AIM Initiative. A Strategy for Augmenting Intelligence Using Machines. Accessed December 12, 2020.….

[4] Mattis, Jim. Summary of the 2018 national defense strategy of the United States of America. Department of Defense Washington United States, 2018.

[5] The National Reconnaissance Office. n.d. "The NRO Overview."

[6] Christopher Darby; Sarah Sewall, “The Innovation Wars: America’s Eroding Technological Advantage,” Foreign Affairs 100, no. 2 (March/April 2021): 142-153

[7] Schmidt, Dr. Eric E. 2021. "Emerging Technologies and Defense: Getting the Fundamentals Right." Testimony Before the Senate Committee on Armed Services. 13. 

[8] Office of the Director of National Intelligence. 2017. "Intelligence Community Information Environment Data Strategy”.….

[9]  Office of the Director of National Intelligence. 2017. "Intelligence Community Information Environment Data Strategy”.….

[10] Tromblay, Darren E. "Silos and sand: Lessons from the US Intelligence Community on leveraging workforce."

[11] Schmidt, Eric and Work, Robert. 19 March 2021. “Final Report, The National Committee on Artificial Intelligence.”…

[12] The Director of National Intelligence . n.d. Members of the IC.

[13] United States Space Force. 2019. About the USSF. December 20.

[14] ODNI. 2021. DNI Ratcliffe Welcomes U.S. Space Force As 18th Intelligence Community Member. January 8.….

[15]Hosenball, Mark. 2021. Trump Administration adds Pentagon ‘Space Force’ to U.S. spy agency group. Reuters. January 8.….

[16] Himes, Jim. 22 December 2020. Rightly Scaled, Carefully Open, Infinitely Agile: Reconfiguring to win the Innovation Race in the Intelligence Community.  Strategic Technologies and Advanced Research (STAR) Subcommittee, House Permanent Select Committee on Intelligence.

[17] Himes, Jim. 22 December 2020. Rightly Scaled, Carefully Open, Infinitely Agile: Reconfiguring to win the Innovation Race in the Intelligence Community.  Strategic Technologies and Advanced Research (STAR) Subcommittee, House Permanent Select Committee on Intelligence.

[18] Adaptive Organization. "From Silos to Value Streams." January 18, 2021.

[19] Scaled Agile Framework. n.d. Scaled Agile Framework.

[20] n.d. The AIM Initiative. A Strategy for Augmenting Intelligence Using Machines. Accessed December 12, 2020.

[21] The Director of National Intelligence . n.d. Members of the IC.

[22] Schmidt, Eric and Work, Robert. 19 March 2021. “Final Report, The National Committee on Artificial Intelligence.”…

[23] Himes, Jim and Stewart, Chris. October 2020. “Rightly Scaled, Carefully Open, Infinitely Agile: Reconfiguring to Win the Innovation Race in the Intelligence Community. A House Permanent Select Committee on Intelligence Strategy.”

[24] Westover, Dr. Johnathan H. 2020. "The Role of Systems Thinking In Organizational Change and Development." Forbes. June 15.….

[25] n.d. "Talend." What Are Data Silos? Accessed 06 01, 2021.

[26] n.d. "Talend." What Are Data Silos? Accessed 06 01, 2021.

[27] Singh, Jaspreet. 2019. "Breaking Silos and Curating Data for Impactful AI." Informa. October 07.….

[28] Office of the Director of National Intelligence. nd. "Intelligence Community Information Technology Enterprise Fact”.

[29] Government Accountability Office. June 2013. “IBM-U.S. Federal.”…

[30] Lock, Michael. 2017. “ANGLING FOR INSIGHT IN TODAY’S DATA LAKE.” AWS Blogs. October.….

[31] Dehghani, Zhamak. 2019. "How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh." Martin Fowler. May 20.

[32] Anu Jain, Graham Person, Paul conroy and Nivas Shankar. 2021. "How JPMorgan Chase built a data mesh architecture to drive significant value to enhance their enterprise data platform." AWS Blogs. May 05.….

[33] Anu Jain, Graham Person, Paul Conroy and Nivas Shankar. 2021. "How JPMorgan Chase built a data mesh architecture to drive significant value to enhance their enterprise data platform." AWS Blogs. May 05.….

[34] Scaled Agile Framework. n.d. Scaled Agile Framework.

[35] Himes, Jim. 22 December 2020. Rightly Scaled, Carefully Open, Infinitely Agile: Reconfiguring to win the Innovation Race in the Intelligence Community.  Strategic Technologies and Advanced Research (STAR) Subcommittee, House Permanent Select Committee on Intelligence.

[36] Scaled Agile Framework. n.d. Scaled Agile Framework.

[37] Milligan, Katrina, Matt, Olsen, and Alexandra Schmitt. 2020. “What the Intelligence Community doesn’t know is hurting it.” Center for American Progress.….

[38] Anu Jain, Graham Person, Paul Conroy and Nivas Shankar. 2021. "How JPMorgan Chase built a data mesh architecture to drive significant value to enhance their enterprise data platform." AWS Blogs. May 05.….

[39] Schmidt, Eric and Work, Robert. 19 March 2021. “Final Report, The National Committee on Artificial Intelligence.”…

[40] Servu, Jared. 2020. "Pentagon R&D spending still lags behind an otherwise healthy Defense budget." Federal News Network. 10 23.….

[41] Darby, Christopher, and Sarah Sewall. "The Innovation Wars: America's Eroding Technological Advantage." Foreign Aff. 100 (2021): 142.

[42] Anu Jain, Graham Person, Paul Conroy and Nivas Shankar. 2021. "How JPMorgan Chase built a data mesh architecture to drive significant value to enhance their enterprise data platform." AWS Blogs. May 05.….

[43] UNESCO Institute for Statistics. 2021. How Much Does Your Country Invest In R&R? Accessed 05 01, 2021.….

[44] Garsten, Ed. 2021. "Experts Say Auto Industry's Software Strategy Needs to Get Agile to Compete.” Forbes. March 08.

[45] Katz, Brian. October 2020. “The Analytic Edge: Leveraging Emerging Technologies to Transform Intelligence Analysis.” Center for Strategic and International Studies.…

[46] Congressional Research Service. n.d. "The Federal Acquisition Regulation (FAR): Answers to Frequently Asked Questions."

[47] Anu Jain, Graham Person, Paul Conroy and Nivas Shankar. 2021. "How JPMorgan Chase built a data mesh architecture to drive significant value to enhance their enterprise data platform." AWS Blogs. May 05.….

[48] Defense Acquisition University. June 2021. “Platforms of the Future.” TeDex. Webinar.

[49] CIO Council. May 2020. “Future of the Federal IT Workforce Update.”…

[50] Wagner, Erich. 26 February 2020. “OPM Announces Adjustments to Annual Time-to-Hire Metrics.” Government Executive.….

[51] Sheridan, John E. "Organizational Culture and Employee Retention." Academy of Management Journal 35, no. 5 (12, 1992): 1036.….

[52] Tromblay, Darren E. 2019. “Silos and sand: Lessons from the U.S. Intelligence Community on leveraging workforce.” Rosalind Franklin University of Medicine and Science; Journal of Interprofessional Workforce Research and Development. Volume 2: Issue 2.

[53] Scaled Agile Framework. n.d. Scaled Agile Framework.

[54] Delizonna, Laura. 2017. “High-Performing Teams Need Psychological Safety. Here's How to Create It.” Harvard Business Review. August 24.….

[55]  Tromblay, Darren E. 2019. “Silos and sand: Lessons from the U.S. Intelligence Community on leveraging workforce.” Rosalind Franklin University of Medicine and Science; Journal of Interprofessional Workforce Research and Development. Volume 2: Issue 2.

[56] US Air Force. n.d. Kessel Run. Accessed May 12, 2020.

[57] Air Force Institute of Technology. n.d. Education With Industry Program. Accessed June 16, 2021.

[58] Himes, Jim. 22 December 2020. Rightly Scaled, Carefully Open, Infinitely Agile: Reconfiguring to win the Innovation Race in the Intelligence Community.  Strategic Technologies and Advanced Research (STAR) Subcommittee, House Permanent Select Committee on Intelligence.

[59] Himes, Jim. 22 December 2020. Rightly Scaled, Carefully Open, Infinitely Agile: Reconfiguring to win the Innovation Race in the Intelligence Community.  Strategic Technologies and Advanced Research (STAR) Subcommittee, House Permanent Select Committee on Intelligence.

[60] National Institute for Science and Technology. n.d. “NIST Risk Management Framework.” Accessed January 17, 2021.

[61] Bush, President George W. 2004. President Signs Intelligence Reform and Terrorism Prevention Act….

[62] Bush, President George W. 2004. President Signs Intelligence Reform and Terrorism Prevention Act….


For more information on this publication: Belfer Communications Office
For Academic Citation: Davenport, Susan. “Creating Intelligence as a Community.” Edited by Natalia Angel. Paper, January 2022.

The Author