Self-hosted AI: Balance innovation and security in government

Government agencies face strict rules that prevent them from using cloud technology for software development. This blocks their access to AI's transformative potential because most advanced AI solutions run in the cloud. The risks of sending data outside their networks and losing control over AI environments force them to find a more secure path.

Despite these challenges, ignoring AI entirely isn't realistic. Agencies must integrate AI into software development to support efficient software modernization. But how can they use AI tools to enhance productivity, improve security, and drive innovation without exposing themselves to the risks associated with cloud-based AI solutions?

Self-hosted AI models provide a strategic solution. By running and managing large language models (LLMs) and other advanced AI capabilities within their own secure infrastructure, whether in on-premises data centers or private cloud environments, agencies gain the control needed to leverage AI while maintaining strict compliance standards and advancing mission-critical applications.

Key benefits of a self-hosted AI strategy

After working with federal agency tech leaders for many years, I understand that a statement like "Let's just host it ourselves" might raise some eyebrows. It's not always straightforward, especially with a technology as new as AI. However, evidence suggests that federal agencies and defense organizations are ready for a different approach.

For example, the Pentagon is actively working on a "fast pass" approach to securing software components, aiming to onboard approved software more quickly by using existing standards such as Software Bill of Materials (SBOM), the NIST Secure Software Development Framework (SSDF), and other common attestation methods and risk assessments.

Meanwhile, the House Oversight and Government Reform Committee has been exploring ways to use IT modernization to enhance efficiency. And there's a broad groundswell of interest in finding ways to leverage AI in government.

Here are several examples from the U.S. military:

• The Defense Information Systems Agency is developing a new data strategy that integrates data, analytics, and AI into all aspects of defense operations through a secure, self-hosted platform.
• The Army is building two new self-hosted AI tools, CamoGPT and NIPR GPT, to support predictive maintenance, analysis of adversaries' communications, logistics optimization, and evaluation of different proposed courses of action.
• The Air Force Research Lab is developing an open-source platform, the Air and Space Force Cognitive Engine, a flexible, single IT platform for operationalizing AI within the Air Force.

Government organizations see clear advantages when they host LLMs within their own secure infrastructure:

• Data sovereignty: When working with sensitive national security information, the risks of external data processing and limited control over AI environments demand a more secure approach that keeps critical data within protected boundaries. Self-hosted environments ensure that level of security.
• Regulatory compliance: Federal agencies must adhere to complex regulatory frameworks, including the Federal Risk and Authorization Management Program (FedRAMP), International Traffic in Arms Regulation (ITAR), Federal Information Security Modernization Act (FISMA), and agency-specific mandates. Self-hosted environments provide the detailed control necessary to implement specific security controls, audit trails, and governance frameworks that meet these strict requirements.
• Better security: Self-hosted models dramatically reduce potential attack vectors by removing dependencies on external APIs and third-party infrastructure. Agencies maintain complete control over access management, network segmentation, and vulnerability patching within their AI systems.
• Custom solutions: Unlike standard cloud solutions, agencies can choose from a list of supported AI models using specialized datasets tailored to their unique use cases and environments. This enables the development of more effective, purpose-built AI solutions that directly support mission objectives, whether by enhancing intelligence analysis, optimizing resources, or strengthening cybersecurity. This customization also facilitates integration with legacy systems, a common challenge in the public sector.
• Cost control: While the initial setup requires an investment in infrastructure and expertise, self-hosted AI models can provide more predictable long-term cost structures compared to variable subscription-based cloud models. This approach offers greater flexibility for large-scale deployments, leveraging existing infrastructure and personnel. Plus, self-hosted AI can offer a secure environment for modernizing legacy systems while maintaining direct oversight of sensitive code.

Fostering innovation within a trusted framework

Running AI in a secure, self-hosted environment supports innovation within a foundation of trust and control. Agencies can adopt open-source AI advances while maintaining security, compliance, and performance standards. This flexibility allows government developers and data scientists to build critical applications with security and compliance as foundational principles rather than afterthoughts.

The examples above clearly demonstrate that the U.S. government — particularly the Department of Defense — is serious about embracing the potential of AI to make their work more effective, efficient, and innovative. This movement is already well underway.

For federal agencies, integrating self-hosted AI models into software development workflows is essential for managing the complex web of security regulations while fostering innovation. Self-hosting allows AI to reach its full potential throughout the software development lifecycle. This enhances operational effectiveness, strengthens security, and accelerates the creation of more intelligent applications to safeguard national interests in an increasingly complex digital environment.