The Role
Design, build, and deploy on‑prem, air‑gapped AI/agent systems: integrate LLMs, MCP interfaces, RAG retrieval, and orchestration; optimize local CPU/GPU inference, harden Linux environments, run experiments, and collaborate across engineering and mission teams to move prototypes into operational capabilities.
Summary Generated by Built In
About The Role:
Innovative Defense Technologies (IDT), a leading defense technology company, is seeking an AI Implementation Engineer to be part of our Warfare Systems team and based out of our Arlington, VA or Mount Laurel, NJ location.
The AI Implementation Engineer will design and deliver engineering-focused AI solutions that move beyond demos into reliable, mission-relevant systems. This role is ideal for an engineer who has extensive experience with commercially available AI tooling/chat, hosting LLM servers and has demonstrated ability to build end-to-end capabilities including MCP integrations, RAG pipelines, tool-using agents, and production-grade AI workflows.
Clearance & Location Requirements:
- All applicants must be able to obtain/maintain an active Secret U.S. Security Clearance.
- This is an on-site position. Requiring at least 3 days in office, based out of our Arlington, VA location or Mt. Laurel, NJ location.
Key Responsibilities
What You Will Do:
- Design and Build AI Solutions for On-Prem systems in Air-Gapped environment: Design and implement end-to-end agentic AI systems that support planning, reasoning, tool use, and multi-step execution in real-world environments. Build modular, testable components that move from prototype to operational capability.
- Integrate Models and Tools for On-Prem systems in Air-Gapped environment: Develop integrations across LLMs, APIs, data sources, and Model Context Protocol (MCP) interfaces to enable intelligent agents to interact with external systems, retrieve context, and take action safely and reliably.
- Develop Retrieval Pipelines for On-Prem systems in Air-Gapped environment: Build and optimize Retrieval-Augmented Generation (RAG) pipelines that connect models to live knowledge sources, structured data, and enterprise content to improve factual grounding, contextual relevance, and response quality.
- Engineer Conversational and Agentic Interfaces for On-Prem systems in Air-Gapped environment: Create conversational systems and intelligent agents with memory, contextual awareness, adaptive decision-making, and support for multi-turn user and system interactions.
- Implement and Evaluate AI Workflows for On-Prem systems in Air-Gapped environment: Translate technical objectives into working pipelines, run experiments, evaluate agent behavior, and iterate on prompts, orchestration logic, retrieval quality, and system performance to improve reliability and usability.
- Architect local infrastructure to size, config, and optimize local CPU/GPU workloads, utilizing quantization techniques to maximize throughput, etc.
- Orchestrate disconnected environments, design and maintain offline model update pipelines, local package mirrors, etc.
- Scope and Define Requirements: Gather, document, and validate technical and functional requirements from project artifacts, stakeholders, and mission needs to ensure feasibility, completeness, and alignment with operational goals.
- Collaborate Across Teams: Work closely with engineers, technical leads, and mission stakeholders to integrate AI capabilities into broader software and system architectures. Participate in technical reviews, design discussions, and delivery planning.
- Support Technical Quality: Contribute to testing, debugging, and performance optimization of AI-enabled applications, including edge cases involving context management, retrieval failures, tool execution, and orchestration logic.
- Learn and Apply Emerging Practices: Stay current on advances in LLMs, agent frameworks, orchestration methods, and applied AI engineering practices, and bring that knowledge into practical system design and implementation.
- Communicate Technical Work: Clearly document architectures, workflows, assumptions, and implementation decisions so that solutions are maintainable, explainable, and transferable across teams.
Skills, Knowledge & Expertise
Who You Are (Required):
- Bachelor’s Degree in Information Technology, Computer Science, Computer Engineering, Electrical Engineering, Systems Engineering, Physics, Math, or equivalent full-time professional experience; Master’s Degree in Engineering or other technical field highly desired
- 5-10+ years of professional experience in software engineering, machine learning engineering, AI engineering, or related technical roles
- Proficiency in Python, including experience with core libraries such as NumPy and Pandas
- Deep hands-on experience with production local inference engines such as vLLM, SGLang, Triton Inference Server, or TensorRT-LLM
- Experience building software with one or more modern AI/ML frameworks such as PyTorch, TensorFlow, LangChain, LangGraph, Semantic Kernel, or AutoGen
- Experience with Linux and hardening (fapolicy/selinux/fips/etc)
- Experience with commercially available AI tooling/chat
- Experience with hosting LLM servers
- Experience with hosting different models (chat/embedding)
- Experience with distributed networking (reverse proxy/load balancing/firewalls/etc)
- Experience with containerization
- Ability to work independently on technical tasks while collaborating effectively in a team environment
- Ability to shift from one project to another in an agile work environment
- Strong leadership capabilities and skills
- Strong documentation skills
- Ability to travel up to 10% of the time, as needed
What Makes You Stand Out:
- Experience hosting LLM servers on local hardware
- Experience with various GPU architectures (NVIDIA preferred)
- Experience with AI Gateways
- Experience with LLM servers and model optimization (VRAM/layers/quantization/fasttensor/kv-cache/parallel/etc)
- Experience with LLM toolcalling
- Experience with RAG
- Experience with vector stores
- Experience with ModelContextProtocol
- Experience with container orchestration
- Experience managing and directing personnel while maintaining cost and schedule targets
- Problem-solving skills with the ability to navigate ambiguous situations
If you see yourself reflected in this role and are excited about the impact you could make, we encourage you to apply! If you know someone who may be a great fit, please feel free to share this opportunity with your network.
Pay Range*: $114,000 - $184,000 or $135,000 - $231,000
This amount may not be reflective of actual compensation that may be earned as pay is dependent on a candidate’s experience, skills, and education. The posted range does not include bonuses, commissions, tips, or other benefits. IDT is often looking to place multiple candidates at various levels. Therefore, more than one pay range has been included, commensurate with experience.
Job Benefits
Why Work at Innovative Defense Technologies (IDT):
IDT is a growing company with a vibrant, entrepreneurial culture. We are headquartered in Arlington, VA with additional offices in Fall River, MA; Mount Laurel, NJ; and San Diego, CA. At each location, our employees work together in a modern, snack-filled, and social office space, designing innovative solutions for our defense industry customers. We offer employees competitive pay and benefits including:
- Generous benefits package
- Competitive PTO
- Paid holidays
- 401(k) with immediate vesting and matching
- 9/80 optional schedule (2nd and 4th Friday off every month)
- Tuition Assistance Reimbursement Program
- Professional Development Resources
- Pre-Tax Commuter Benefits
- Organization-Wide Monthly Tech Connect Events
- Annual Employee Recognition Awards
- Regular Social Events and Catered Lunches
EEO Statement:
IDT is an Equal Opportunity employer.
About
Innovative Defense Technologies (IDT) is a leading defense technology company focused on rapidly delivering mission-critical software and systems solutions to the U.S. government. The company specializes in automation, digital engineering, and enabling the rapid integration of advanced capabilities into complex weapon and combat systems. These essential solutions empower the warfighter to field, operate, and sustain decisive capability at speed and scale. At IDT, our employees are developing advanced mission-critical outcomes by consistently delivering high-quality results, fostering innovation, applying rigorous problem-solving, and effectively collaborating across multifaceted teams and stakeholders. Grounded in these competencies, IDT translates complex customer priorities into robust software solutions.
Skills Required
- Ability to obtain/maintain an active Secret U.S. Security Clearance
- On-site, at least 3 days per week (Arlington, VA or Mt. Laurel, NJ)
- Bachelor's Degree in IT, Computer Science, Computer Engineering, Electrical Engineering, Systems Engineering, Physics, Math, or equivalent experience
- Master's Degree in Engineering or related technical field
- 5-10+ years professional experience in software engineering, machine learning engineering, AI engineering, or related roles
- Proficiency in Python (including NumPy and Pandas)
- Deep hands-on experience with local inference engines (vLLM, SGLang, Triton Inference Server, or TensorRT-LLM)
- Experience building software with modern AI/ML frameworks (PyTorch, TensorFlow, LangChain, LangGraph, Semantic Kernel, or AutoGen)
- Experience with Linux and hardening (fapolicy/SELinux/FIPS)
- Experience with commercially available AI tooling/chat and hosting LLM servers
- Experience hosting different model types (chat and embedding models)
- Experience with distributed networking (reverse proxy, load balancing, firewalls)
- Experience with containerization
- Ability to scope, document, and validate technical and functional requirements; strong documentation skills
- Ability to work independently and collaborate effectively in team environments; ability to shift between projects in an agile environment
- Strong leadership capabilities and ability to lead technical efforts
- Ability to travel up to 10%
- Experience hosting LLM servers on local hardware (GPU architectures, NVIDIA preferred)
- Experience with AI Gateways, LLM toolcalling, RAG, vector stores, ModelContextProtocol, model optimization (quantization/VRAM/layers/fasttensor/kv-cache/parallel)
- Experience with container orchestration
- Experience managing and directing personnel while maintaining cost and schedule targets
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
WHO WE ARE Innovative Defense Technologies -IDT- provides automated software testing, data analysis, and cybersecurity solutions for the DOD's mission-critical systems. Using patented Automated Test and ReTest (ATRT) technology, we support the entire software testing lifecycle from component testing, to system integration and testing, to the analysis of results. Our automated test and analysis solutions enable the rapid update and delivery of large, complex systems. Our technologies offer: -Rapid system configuration assessment and validation -Repeatable, consistent testing, analysis and reporting -On-demand regression tests in response to cyber threats -Statistical rigor in test and analysis -Reduced time and manpower At IDT, we work closely with our customers to tailor our products to support their systems and meet their goals. IDT has the expertise, strategies and processes in place to provide an automated solution for your next project. Let us help you dramatically accelerate test execution time, expand test coverage and improve software quality. OUR MISSION Our mission is to enable new software capabilities to be fielded faster and more affordably by reducing the amount of time and the cost spent on software testing. Through the use of our software testing technology, IDT is changing the way testing is being done for large, complex, software systems.
