From Hype to Reality: Real-World Lessons and Recommendations for AI in Military Applications
Name
MIT-LIN-151464.pdf
Description
Main Report
Size
1.46 MB
Format
Adobe PDF
Checksum (MD5)
38d14dc62f49303b3abe0b6874617afd
Author(s)
Lynch, Joshua
Niss, Laura
Date Issued
February 17, 2026
Abstract
The current use cases, limitations, and future capacity
of large language models (LLMs) as assistants to military
personnel remain an open question. This paper presents a case
study of an Airman’s interaction with and trust calibration of
LLMs over three months, both as an everyday assistant and
for development of ROMAD-AI, a tactical military application.
Through intuitive, AI-generated software development, an approach
relying on iterative code generation through natural
language prompting of LLMs from a technical novice rather
than human generated programming from a technical expert,
the research reveals significant gaps between industry curated
AI capability demonstrations and operational reality, requiring
systematic trust calibration and realistic scope management.
Outcomes are analyzed through operational and technical expertise
perspectives to provide practical guidance for both military
service members seeking effective AI integration and researchers
developing military-focused AI systems.
Subjects
AI, LLMs, military, vibe coding, application development, trust, calibration
MIT Department
Lincoln Laboratory
Persistent DSpace Link
