Task Force Lima chief gives first look at DOD’s vision for rapidly exploring the uncertain power of generative AI

More than 220 officials from across the private sector and academia registered to attend the Pentagon’s first-ever Task Force Lima Industry Challenge Day that kicked off Wednesday — where they’ll learn about certain “gaps” in the emerging and likely disruptive field of generative artificial intelligence that the U.S. military needs their help confronting, DefenseScoop has learned. 

Defense Department leaders launched Task Force Lima within the Chief Digital and Artificial Intelligence Office’s (CDAO) Algorithmic Warfare Directorate back in August to accelerate the enterprise’s understanding, assessment and deployment of generative AI. That nascent, buzzy realm broadly encompasses technology associated with large language models that produce (convincing but not always correct) software code and media based on prompts. And experts warn it’s rife with promising opportunities and serious challenges that are far from fully realized. 

In a recent interview, Navy Capt. M. Xavier Lugo, the CDAO’s new Task Force Lima mission commander, briefed DefenseScoop about how this first industry day and the other in-the-works events and elements fit into his team’s overarching, ambitious vision to rapidly navigate the uncertain and transformative power generative AI presents across the defense landscape, and to ultimately support the department’s ethical adoption of the tech.

“The mission from a task force perspective is really to develop, evaluate, recommend and monitor generative AI capabilities for the whole DOD and to responsibly ensure that the department capitalizes on their mission impact while integrating these technologies at scale. That sounds like a lot. So, what we did is we broke it into phases, with the first phase being learning everything that’s out there as much as we can,” Lugo explained. 

Lugo is a 28-year Navy officer with extensive experience in the Supply Corps. A mechanical engineer by degree, he’s been a coder since high school and is also an operations research graduate from the Naval Postgraduate School. In his previous assignment, he worked to modernize logistics-aligned information technologies for the entire Navy.

“I am very familiar with how [AI technology] actually works, and how it utilizes data,” he told DefenseScoop.

In his current capacity leading the new task force, Lugo meets with DOD’s Chief Digital and AI Officer Craig Martell very frequently. He and Martell — the former head of machine learning at Lyft — have “some geek sessions here and there” too, Lugo noted. They “don’t always agree on some of the theoretical stuff,” he added. 

But to both, encouraging diverse viewpoints in tech-driving pursuits is a good thing.

“From our perspective, because the use cases are military-related and because the workflows are all based on operational pieces, you do want military officers involved at some level. I have been lucky that I was involved from the sense of being the commander of it. But we bring that practical flavor to the research and technical pieces of AI. And also, just to be clear, CDAO as a whole is not a research entity. We’re not here to build models. We’re here to operationalize them. And that’s where a task force like Task Force Lima and the military officers in this command come in: We operationalize this technology,” Lugo told DefenseScoop.

DOD is a sprawling, distributed enterprise — “so getting to all the generative AI efforts out there is not an easy task,” the task force chief also noted.

So far though, his team has had “very good collaboration” from the services, combatant commands, and all DOD agencies. They are beginning to hone in on the categories where large language models, specifically as part of the generative AI-capable tools, are being considered for performance across the department.

Lugo’s team also has a short-term task to populate interim guidance on generative AI for the Pentagon.

“That has also been sort of coordinated. What I mean with ‘sort of’ is that it’s not not being coordinated, but that in parallel all the services and federal agencies have also provided guidance. So, what we did was, we’ve taken all of those multiple guidances and also included what we have learned so far, and we’re going to populate that soon. That’s one of the first steps so that everybody knows what their ‘left and right limits’ are with this technology, and so that we can all be at least in the same field as to what we are thinking of utilizing the tech,” Lugo said.

Task Force Lima, he added, has multiple “touch points” to carry out its mission.

“We have a battle rhythm of weekly, monthly and quarterly meetings,” he said.

And earlier this month, the cadre hosted its first government-only kickoff to engage internal stakeholders on their in-the-making plans and approach. Between 200 and 300 officials attended virtually and in person.

That “gives you a scale of the level of interest — and every single service was represented, plus the Coast Guard, plus multiple agencies, all the [Principal Staff Assistants]. I mean, it was very well covered from the perspective of which agencies and personnel were in it,” Lugo noted.

The team also recently released a new request for information via which it has invited the public to help the DOD “further understand and explore the risks and benefits of development, acquisition, and integration” of generative AI. 

Responses to that RFI are due by Oct. 8. 

“That is based on what we’ve learned so far. There are some gaps that we have noticed in the technology. And what that challenge is about is to tell [industry and academia] we are interested: ‘You’ve done a great job on the development and on the delivery of this technology but there’s some pieces that we need some help from you, because if you fix these gaps, we have more potential use of this technology,’” Lugo explained.

One example of such gaps is referred to as “hallucinations” — a phenomenon where a large language model essentially makes up untrue information or false facts that aren’t rooted in actual real data or events.

“So, how do we get around that? How do we measure it? How do we protect against it? What are some solutions out there — whether it is a single solution from a model per se, or whether it is a system with a system solution, or whether it is a mitigation process? Any of those or all of those, right, for that just one challenge. Come in and tell us your ideas on how we’re going to do that,” Lugo said.

Potential collaborators are learning more about this and other issues that need addressing at the industry day unfolding Wednesday. The task force will accept white papers — and then ultimately aim to work with partners through its Tradewinds initiative.

“While building the plane after we’re flying it — this is one of those pieces that has to be built over,” Lugo said.

Ultimately, Task Force Lima is deeply involved in both the Pentagon’s actual research and deployment of large language models, and in informing its creation of guidance, frameworks, workflows and policies to govern their evolving use.  

While publicly accessible models made by Open AI, Google, Microsoft and others already exist, the Pentagon has urged components not to enter government information into them for reasons it says are associated with operational security.

“We are not building at this moment in time and we have not decided whether we’re going to build DOD-specific models. However, what is happening is that models that are already built are being utilized in a containerized fashion — so disconnected from the wild, if you want to put it that way,” Lugo noted.

Multiple times in the interview, he emphasized that there are still many unknowns regarding if, in what ways, and for what purposes generative AI might be eventually deployed at scale by the military and DOD.

“Our mind is not necessarily made up yet — without doing the experimentation — as to whether we’re going to go ahead and say, ‘Yes, for sure, we’re going to implement this technology.’ That’s exactly where we are right now and that’s part of the reason we were stood up. This is new, this is out there with a lot of emphasis from industry — and so, how should we and how can we utilize this technology? Those are the two main questions,” Lugo said.

Ambitiously, Task Force Lima’s initial timeline to figure that out is only 18 months.

“Now, let me caveat that the reason we have a short timeline is because of the immediate nature of the technology,” Lugo noted.

Still, before his team stands down after that year-and-a-half passes, the plan is to provide a transition strategy with deliverables including a list of all the players and assets needed to carry on this generative AI-driving work. 

“This is extremely, extremely new technology — and we’ve got to have the humans on top of it all the time,” Lugo told DefenseScoop.