By Clinton Austin
As the Department of Defense (DoD) continues to explore different use cases for integrating Artificial Intelligence (AI) into its everyday processes, it must learn from past lessons where AI deployments failed. One such lesson is the failure of IBM Watson deployments. Since its debut on “Jeopardy!” IBM Watson has been applied to various fields, including healthcare, finance, customer service, and more, demonstrating its versatility and potential to solve complex problems across different industries.
Data Cleaning
For IBM Watson to be efficient, it requires clean data to produce positive outputs. Clean data refers to data that has been processed and prepared to ensure it is free from errors, inconsistencies, and irrelevant information. Clean data is crucial for building accurate and reliable models. However, clean data is rare in the amount the DoD produces because it often lacks uniform metatags, fails to follow a particular format, and is stored in various locations like file storage or Microsoft Teams environments.
One reason IBM Watson deployments failed was that the data needed to be cleaned, and the DoD needed trained data stewards who could add context to the data results. Developing and training good data stewards takes time, and at the height of the Global War on Terrorism (GWOT), no one wanted to give up their manpower to clean up the data. Data cleaning is also time-consuming and often gets delegated to junior enlisted personnel who do not have a contextual understanding of the data.
Data Ownership
The next challenge for a successful IBM Watson deployment was getting access to the data. Within a combatant command, the senior ranking commanding officer is typically the data owner (often delegated to the senior commanding intelligence officer for responsibility, and the senior communications officer owns the systems in which the data resides). However, it can be challenging. The individual program offices and directories that spend their funding to develop the systems have contractual ownership over the data.
For example, suppose the Logistics Directorate develops a program to keep track of material within their Area of Responsibility (AOR), and the directorate utilizes a Federal Service Integrator (FSI) to develop the program. In that case, the data is now locked behind several layers of bureaucracy. One, if the project is ongoing or if the development of the project has stopped, it will require contract modification to allow for an Application Program Interface (API) to access the underlying data or a new contract to develop new features within a legacy code base, which allows for the stated API. Secondly, it is often a burdensome process to review the legacy code base because the original development team, at the time of development, did not comment on their code, did not consider feature support, used proprietary tools that are no longer supported, or completely vendor-locked in.
Same Problem, New Technology
With the promise of the DoD CIO’s office “Cloud First” approach and its promise of Cloud Service Providers (CSPs) offering unlimited access to all the AI goodness, the DoD is falling into the same trap as it did with IBM Watson. For those unfamiliar with the “Cloud First” approach, the DoD CIO’s “Cloud First” strategic initiative is to modernize the Department of Defense’s IT infrastructure by prioritizing adopting cloud computing technologies. This approach is designed to enhance the agility, efficiency, and security of DoD operations. As the combatant commands and the services rushed toward this effort, they often needed to realize that there were a lot of hidden costs with the “Cloud First” approach while still not addressing the issues of data cleaning and ownership. What magnifies the issue further is the lack of specialized talent to implement the desired cloud-agnostic approach because it requires talent to have in-depth knowledge of each CSP to be effective.
“Cloud First” Challenge
The “Cloud First” Approach comes with many hidden costs. One is the egress cost, or the charges incurred when data is transferred from a cloud provider’s network to another network, such as the Internet or another cloud provider. Please note that there was little to no cost to ingest (data coming in) the information, but the high egress (data going out) cost prevented the combatant commands and services from being cloud-agnostic.
So, the trap is now set. With the promise of AI, the combatant commands and the services moved their file storage to a Cloud Service Provider (CSP) and, slowly, the other parts of their IT infrastructure. Moving the file storage was done with little to no data cleanup, which led to undiscovered data spillage on the existing file storage, which now resides on the CSP or data correlation issue. A data correlation issue is where one or more data points can be correlated to gain more insights. This correlation often classifies the data at a higher classification. To bypass the issue of data correlation, the combatant commands and services now overclassify the data, which causes its own set of issues, primarily higher labor costs on personnel to have the necessary security clearance to access and analyze the data.
Talent Challenge
The DoD fell into the trap of the amount of talent that would be needed to maintain these cloud infrastructures. The DoD went in with the initial thought processes that they were going to be able to repurpose the existing talent to maintain these future systems, but without realizing that there was still an ongoing war, which stepped into being able to do some deployment cycles off time that is required to study for these certifications. And there were still outstanding systems in place that were required to be able to maintain while they were trying to go to this future cloud environment.
If stated individuals do get trained, have the necessary certifications, and have the security clearance, they will struggle with the “Why” statement. Why would an individual grade of E-4 through E-6 pay or junior officers stay in the service when their counterpart in the civilian sector initially makes between $75,000 to $140,000 without the additional responsibilities of serving? Many make the decision to leave the military, which adds to the continued gap of not having enough cleared personnel with the necessary clearance for the suggested AI/ML strategy or Cloud First Approach without having to subcontract the work. Along with the historically low recruiting numbers, the gap continues to grow.
Recommendations
The Department of Defense (DoD) is at a critical crossroads in pursuing AI and ML integration. To navigate these challenges effectively, I propose the following comprehensive steps:
1. Modernize Contractual Frameworks and Legacy Systems
To integrate AI and ML effectively, the DoD must modernize its contractual frameworks and legacy systems. This involves conducting a comprehensive review of existing contractual language to identify mission-critical programs. A systematic process should be initiated to update legacy code bases, prioritizing those essential for AI/ML integration. New contractual templates must also be developed to facilitate easier data access and sharing across different DoD entities, ensuring a more streamlined and efficient approach to data management and technological advancement.
2. Implement a Data-Centric Approach
A data-centric approach is crucial for the DoD’s AI and ML initiatives. This strategy begins with establishing a robust data governance framework prioritizing data quality, security, and interoperability across all systems. Comprehensive data audits should be conducted to thoroughly understand existing data assets, their quality, and their potential value for AI/ML applications. Furthermore, clear criteria must be developed to guide decisions on which data sets should be migrated to the cloud. These criteria should consider data sensitivity, operational importance, and the potential for AI/ML utilization, ensuring that cloud migration efforts are strategic and aligned with the DoD’s broader technological goals.
3. Standardize Cloud Infrastructure
Standardizing cloud infrastructure is essential for the DoD to maximize efficiency and interoperability in its AI and ML initiatives. Although contracts have been awarded to multiple Cloud Service Providers (CSPs), focusing on a primary platform to streamline operations is crucial. Amazon Web Services (AWS) is the leading candidate due to its high adoption rates and robust security features. However, this decision should not be static; regular reassessments should be conducted to ensure the chosen platform meets the DoD’s evolving needs and keeps pace with technological advancements. A comprehensive migration strategy must be developed to facilitate this standardization, encompassing detailed timelines, resource allocation plans, and risk mitigation strategies. This approach will ensure a smooth transition to a standardized cloud infrastructure while minimizing disruptions and security risks.
4. Invest in Human Capital and Knowledge Management
Investing in human capital and knowledge management is critical for the DoD’s successful implementation of AI and ML technologies. This involves establishing dedicated knowledge management teams and data stewardship roles across the organization to ensure proper oversight and utilization of data assets. Comprehensive training programs should be developed to upskill existing personnel in cloud technologies, data management, and AI/ML applications, creating a workforce capable of leveraging these advanced technologies effectively. To address the challenge of talent retention, the DoD must create attractive career paths and implement robust retention strategies that can compete with private sector opportunities. This approach will help build and maintain a skilled workforce capable of driving the DoD’s technological initiatives forward, ensuring that the organization can fully capitalize on the potential of AI and ML in its operations.
5. Enhance Resilience and Redundancy
Enhancing resilience and redundancy is crucial for the Department of Defense (DoD) to ensure uninterrupted data access and communication in the face of potential threats to cloud access points (CAPs). The DoD must recognize that CAPs may become targets, potentially disrupting critical operations for combat commands. To mitigate this risk, robust redundancy measures should be implemented. This can include deploying AWS Snowball or similar edge computing solutions at combat commands, enabling continued data access and decision-making capabilities even under Title 10 authority. Additionally, the DoD should develop and regularly test contingency plans to prepare for scenarios where cloud access is compromised. By implementing these measures, the DoD can enhance its ability to maintain operational effectiveness and ensure the availability of critical data and communication channels in the face of potential disruptions.
6. Realistic Cost-Benefit Analysis
A realistic cost-benefit analysis is essential for the DoD’s AI and ML initiatives, moving beyond the simplistic view of cloud migration as merely a cost-saving measure. This approach requires thorough, long-term cost-benefit analyses that account for often-overlooked expenses such as data egress fees, ongoing training requirements, and system updates. These comprehensive assessments should consider both immediate and future financial implications, as well as operational benefits. Additionally, it’s crucial to regularly evaluate and report on the tangible benefits and return on investment (ROI) of AI/ML initiatives. This continuous assessment process not only justifies continued investment but also helps identify areas for improvement and optimization. By adopting this more nuanced and holistic approach to financial analysis, the DoD can make more informed decisions about resource allocation and ensure that its AI and ML investments deliver real, measurable value to its operations and overall mission.
7. Align AI/ML Initiatives with Warfighter Needs
Aligning AI/ML initiatives with warfighter needs is crucial for the DoD to maximize the impact of these technologies on operational effectiveness. This alignment requires establishing clear and direct lines of communication between AI/ML development teams and frontline military personnel, ensuring that technological solutions are tailored to real-world operational requirements. The DoD should prioritize AI/ML projects that directly enhance mission capabilities and operational effectiveness, focusing resources on initiatives that provide tangible benefits to warfighters. To measure the success of these efforts, it’s essential to develop comprehensive metrics that assess the impact of AI/ML integration on mission success and warfighter effectiveness. These metrics should go beyond technical performance to include practical outcomes in the field. By maintaining this close alignment between technological development and operational needs, the DoD can ensure that its AI/ML initiatives deliver meaningful improvements to military capabilities and contribute significantly to overall mission success.
Conclusion
The Department of Defense’s journey towards effective AI and ML integration is complex and multifaceted, requiring a comprehensive and strategic approach. The lessons learned from past failures, such as the IBM Watson deployments, highlight the critical importance of addressing fundamental issues before rushing into new technological initiatives.
The DoD faces significant challenges in data cleaning, data ownership, cloud infrastructure standardization, and talent retention. These challenges are compounded by the rapid pace of technological advancement and the competitive landscape for skilled personnel. However, by implementing the recommended steps, the DoD can build a solid foundation for successful AI and ML integration.
The key to this success is a shift towards a data-centric approach that emphasizes the importance of clean, accessible, and properly managed data. This must be coupled with modernized contractual frameworks and legacy systems that facilitate rather than hinder technological advancement. Standardizing cloud infrastructure while maintaining flexibility for future needs is crucial for efficiency and interoperability.
Investing in human capital through comprehensive training programs and attractive career paths is essential for building and retaining the necessary talent pool. This investment must be balanced with realistic cost-benefit analyses that consider both the immediate and long-term implications of AI and ML initiatives.
Perhaps most importantly, the DoD must ensure its AI and ML initiatives align with warfighter needs. This alignment will ensure that technological advancements translate into tangible improvements in operational effectiveness and mission success.
The path forward requires patience, strategic thinking, and a willingness to address systemic issues. It demands a balance between embracing cutting-edge technologies and maintaining the robustness and security required for military operations. By adopting this comprehensive approach, the DoD can avoid the pitfalls of past initiatives and build an AI and ML ecosystem that truly enhances its operational capabilities.
Ultimately, the successful integration of AI and ML technologies has the potential to significantly enhance the DoD’s ability to meet current and future challenges. However, this potential can only be realized through careful planning, strategic implementation, and a continuous commitment to learning and adaptation. As the DoD moves forward, it must remain focused on its core mission while leveraging these technologies to create a more agile, efficient, and effective defense force for the future.
