The Role
Designs, develops, and maintains enterprise-scale Apache NiFi data flows for secure data onboarding, integration, transformation, and governance. Develops custom Java processors, controller services, APIs, and validation components; integrates databases, APIs, messaging systems, Kafka, and cloud storage. Optimizes clustered pipelines, implements metadata and security controls, supports Zero Trust requirements, troubleshoots complex data movement issues, and mentors engineers while contributing to architecture, standards, and documentation.
Summary Generated by Built In
Apache NiFi Engineer / Data Integration Engineer
Position Overview
Red Arch Solutions is seeking a highly skilled Apache NiFi Engineer to support enterprise data onboarding, integration, and transformation initiatives within a Zero Trust environment. The successful candidate will design, develop, deploy, and maintain complex Apache NiFi data flows that enable secure, scalable, and reliable movement of mission and enterprise data across disparate systems.
This role requires deep expertise in Apache NiFi architecture, custom processor development, enterprise integration patterns, metadata management, data transformation, and workflow automation. The engineer will work closely with Data Stewards, Enterprise Architects, Zero Trust Core Services teams, Data Catalog personnel, system owners, and mission stakeholders to onboard new data sources and ensure compliance with enterprise data standards and security requirements.
KEY RESPONSIBILITIES
Apache NiFi Development & Engineering
Experience
Demonstrated expertise with:
Experience developing custom:
Strong experience with:
Understanding of:
The ideal candidate is not simply a NiFi user, but a NiFi architect and developer who can build custom processors from scratch, solve complex integration challenges, optimize enterprise-scale data flows, and lead onboarding efforts across multiple systems and organizations. This individual should be the technical expert that other engineers rely upon for advanced flow design, custom processor development, and enterprise data integration solution
Salary Range for this opening $220-255K #CJ
Position Overview
Red Arch Solutions is seeking a highly skilled Apache NiFi Engineer to support enterprise data onboarding, integration, and transformation initiatives within a Zero Trust environment. The successful candidate will design, develop, deploy, and maintain complex Apache NiFi data flows that enable secure, scalable, and reliable movement of mission and enterprise data across disparate systems.
This role requires deep expertise in Apache NiFi architecture, custom processor development, enterprise integration patterns, metadata management, data transformation, and workflow automation. The engineer will work closely with Data Stewards, Enterprise Architects, Zero Trust Core Services teams, Data Catalog personnel, system owners, and mission stakeholders to onboard new data sources and ensure compliance with enterprise data standards and security requirements.
KEY RESPONSIBILITIES
Apache NiFi Development & Engineering
- Design, develop, test, deploy, and maintain complex Apache NiFi data flows supporting enterprise data onboarding and integration requirements.
- Engineer scalable and fault-tolerant ingestion pipelines capable of processing structured, semi-structured, and unstructured data.
- Develop reusable flow frameworks and integration patterns to accelerate onboarding activities.
- Optimize flow performance, throughput, latency, and resource utilization across clustered environments.
- Troubleshoot and resolve complex data movement, transformation, and processing issues.
- Develop custom Apache NiFi processors, controller services, reporting tasks, and extensions using Java.
- Create specialized processors supporting:
- Metadata extraction
- Schema validation
- Data tagging
- Attribute enrichment
- Security classification enforcement
- Data quality validation
- API integrations
- Workflow automation
- Extend NiFi capabilities to support mission-specific onboarding requirements.
- Develop shared processor libraries and reusable components.
- Integrate enterprise systems using:
- REST APIs
- SOAP Services
- JDBC/ODBC Connections
- Message Queues
- Kafka
- File-Based Transfers
- Cloud Storage Services
- Design data ingestion and routing strategies across multiple environments.
- Support onboarding of mission and enterprise data assets into approved repositories and data services.
- Implement metadata collection, tagging, and enrichment capabilities within data onboarding workflows.
- Support enterprise data catalog and governance initiatives.
- Ensure onboarding workflows comply with metadata standards, taxonomy requirements, and stewardship processes.
- Collaborate with Data Stewards and Data Owners to improve data quality and discoverability.
- Implement secure data movement patterns aligned with Zero Trust principles.
- Integrate authentication, authorization, encryption, and auditing capabilities into data pipelines.
- Coordinate with cybersecurity and Zero Trust teams to ensure compliance with enterprise security requirements.
- Support Attribute-Based Access Control (ABAC) and policy-driven data access models.
- Provide guidance and mentorship to junior engineers and onboarding specialists.
- Participate in architecture reviews, design sessions, and technical working groups.
- Contribute to engineering standards, best practices, and automation strategies.
- Develop technical documentation, standard operating procedures, and onboarding guides.
Experience
- 20 years experience. Experience working with users to help define their database use, data analytics, and the data user experience.
- 12+ years of data integration, ETL, or enterprise middleware experience.
- 8+ years of hands-on Apache NiFi development experience.
- Experience designing and supporting large-scale enterprise data ingestion architectures.
- Experience onboarding new data sources and integrating complex systems.
Demonstrated expertise with:
- NiFi Flow Design
- Process Groups
- Remote Process Groups
- Site-to-Site Communications
- Controller Services
- Parameter Contexts
- Flow Versioning
- NiFi Registry
- Clustered Deployments
- Provenance Tracking
- Back-Pressure Management
- Flow Performance Tuning
- Secure Flow Design
Experience developing custom:
- Java-based NiFi Processors
- Controller Services
- Reporting Tasks
- Custom APIs
- Schema Validation Components
- Metadata Extraction Processors
- Content Parsing Processors
- Security Enforcement Processors
- Data Quality Validation Processors
Strong experience with:
- Java
- Spring Framework
- JSON
- XML
- Avro
- Parquet
- REST APIs
- SQL
- PostgreSQL
- Elasticsearch
- Kafka
- Git
- Docker
- Kubernetes
- Linux
Understanding of:
- Zero Trust Architecture
- Data-Centric Security
- PKI
- TLS
- Identity and Access Management
- Data Governance
- Metadata Management
- Enterprise Architecture Principles
- Active TS/SCI clearance required.
- Bachelor's degree in Computer Science, Engineering, Information Systems, or related technical discipline.
- Experience supporting DIA, NSA, NGA, or other Intelligence Community customers.
- Experience integrating with Data Catalog platforms and metadata repositories.
- Experience with ontology, taxonomy, and semantic tagging implementations.
- Familiarity with Cloudera DataFlow, Hortonworks DataFlow, or large-scale NiFi deployments.
- Experience developing automated onboarding workflows.
- Knowledge of enterprise data governance and stewardship processes.
- Experience operating in highly regulated or classified environments.
The ideal candidate is not simply a NiFi user, but a NiFi architect and developer who can build custom processors from scratch, solve complex integration challenges, optimize enterprise-scale data flows, and lead onboarding efforts across multiple systems and organizations. This individual should be the technical expert that other engineers rely upon for advanced flow design, custom processor development, and enterprise data integration solution
Salary Range for this opening $220-255K #CJ
Skills Required
- 20 years of experience, including experience working with users to define database use, data analytics, and data user experience
- 12 or more years of data integration, ETL, or enterprise middleware experience
- 8 or more years of hands-on Apache NiFi development experience
- Experience designing and supporting large-scale enterprise data ingestion architectures
- Experience onboarding new data sources and integrating complex systems
- Expertise with NiFi flow design, process groups, remote process groups, Site-to-Site communications, controller services, parameter contexts, flow versioning, NiFi Registry, clustered deployments, provenance tracking, back-pressure management, performance tuning, and secure flow design
- Experience developing Java-based NiFi processors, controller services, reporting tasks, custom APIs, schema validation components, metadata extraction processors, content parsing processors, security enforcement processors, and data quality validation processors
- Strong experience with Java, Spring Framework, JSON, XML, Avro, Parquet, REST APIs, SQL, PostgreSQL, Elasticsearch, Kafka, Git, Docker, Kubernetes, and Linux
- Understanding of Zero Trust Architecture, data-centric security, PKI, TLS, identity and access management, data governance, metadata management, and enterprise architecture principles
- Active TS/SCI security clearance
- Bachelor's degree in Computer Science, Engineering, Information Systems, or a related technical discipline
- Experience supporting DIA, NSA, NGA, or other Intelligence Community customers
- Experience integrating with data catalog platforms and metadata repositories
- Experience with ontology, taxonomy, and semantic tagging implementations
- Familiarity with Cloudera DataFlow, Hortonworks DataFlow, or large-scale NiFi deployments
- Experience developing automated onboarding workflows
- Knowledge of enterprise data governance and stewardship processes
- Experience operating in highly regulated or classified environments
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Red Arch Solutions is a premier U.S. small business providing its customers with state-of-the-art tactical and strategic intelligence, systems, and software engineering solutions, solving some of the most pressing and unique intelligence community challenges related to National Security. Our employees are exceptionally skilled professionals; individuals who are dedicated to collecting, analyzing, creating and engineering the best solutions in support of critical missions. Red Arch Solutions takes pride in the success of our customers. The depth of our engineers’ experience combined with our breadth of exposure across the intelligence community enables us to provide value added, synergistic analytical and engineering support. Unique to Red Arch Solutions is the focus on its employees. Our guiding principle is “People First.” We offer employees career mobility and stability, work-life balance flexibility, and individualized career paths. The loyalty of highly trained, skilled professionals is our key corporate asset. Red Arch Solutions is an Equal Opportunity Affirmative Action Employer.
.jpeg)
