DualBird is building a next-generation data acceleration platform that brings hardware-level performance to cloud infrastructure with software-level simplicity.
We’re looking for a Infrastructure & Automation Engineer to build automated systems that validate our Spark acceleration layer in large scale.
The role
You will design and operate our CI/CD for our core framework. This is a SW role focused on building infrastructure and internal tools on AWS (EMR/EKS) to ensure data correctness, system stability, and performance parity.
What you’ll do
- Framework Development: Build an automation frameworks to build, validate, orchestrate, and release out SW suites.
- CI/CD Integration: Integrate automated tests into GitHub Actions CI pipelines, ensuring every release correctness and performance.
- System and CI Monitoring: Implement observability (Prometheus/Grafana/CloudWatch) to monitor our SW health.
- Cross-Team Collaboration: Work with low level system architechts to provide automated feedback loops for hardware-level optimizations.
What you bring
- 4–7+ years of experience: Proven track record in SW development, QA automation or DevOps within cloud or distributed data environments.
- Advanced Python: Ability to build complex, modular automation tools (Pytest, custom SDKs, Build tools, debug tools).
- Cloud & K8s: Hands-on experiennce with AWS (EMR, S3, EC2) and EKS (Kubernetes) networking/orchestration.
- CI/CD & Observability: Experience building automated release pipelines and monitoring distributed system as well as SW helath.
- Advantage: Spark & Data Lakes: Technical knowledge of Apache Spark (shuffles, memory tuning, executors) and Apache Iceberg.
- Linux/Systems: Strong CLI skills and comfort debugging failures across the OS and container layers.
Skills Required
- 4-7+ years of experience as a Data Engineer, Data Platform Engineer, or Automation Engineer
- Strong hands-on experience with Apache Spark and distributed data processing systems
- Experience building automated data workflows or system validation frameworks
- Solid programming skills in Python
- Familiarity with AWS-based data environments
- Ability to reason about system-level behavior, performance, and reliability
- Strong communication skills and comfort working across multiple engineering disciplines
What We Do
DualBird exists to solve the $7 trillion data infrastructure crisis. By 2030, enterprises will need nearly that much in new data center investment just to keep pace with AI and analytics demand. Traditional approaches: adding more GPUs, optimizing runtimes, expanding infrastructure: only postpone the bottleneck. The math doesn't work. We've built a cloud-native engine that fuses hardware and software into a single solution: delivering 10-100x faster performance and 50-90% cost savings through a simple plug-in. No infrastructure overhaul. No manual tuning. Just hardware-grade acceleration with software simplicity. Our approach leverages rewritable hardware in the cloud to bring specialized acceleration to data workloads instantly. Processes that once took hours now finish in minutes. Workloads once too expensive to run frequently – like AI model retraining – become practical and affordable. Our impact is already proven: Customer pilots are showing transformational results across analytics and AI infrastructure. Enterprises are processing data at speeds and costs previously impossible with general-purpose CPUs. Backed by Lightspeed Venture Partners, Bessemer Venture Partners, Uncork Capital, and Angular Ventures, we're a team of veterans from Amazon AWS and the semiconductor industry, building the infrastructure that will power the next era of data. Learn more at www.dualbird.io
