The Role
Designs, deploys, and operates production-scale data platforms in fully air-gapped environments. Builds Airflow and Spark pipelines for high-volume sensor, satellite, and third-party data; develops PostgreSQL, TimescaleDB, PostGIS, and DuckDB solutions; manages Docker-based deployments and MinIO storage; and handles data quality, observability, documentation, troubleshooting, and SLA compliance without cloud services.
Summary Generated by Built In
We're hiring a Data Engineer (7+ years) to design deploy independently, and operate production-scale data platforms in fully air-gapped, secure customer environments - no cloud dependency, with manual deployment and troubleshooting expected. The ideal candidate has hands-on experience with Apache Airflow, Spark, PostgreSQL, Docker, Python, SQL, and S3-compatible storage (MinIO), along with strong knowledge of geospatial and time-series data, and is comfortable owning infrastructure end-to-end without managed cloud services - from data pipelines to deployment to troubleshooting in fully isolated environments.
Requirements
- 7+ years of experience in Data Engineering and production-scale data platforms
- Design and manage Apache Airflow pipelines for high-volume sensor, satellite, and third-party data ingestion within isolated environment.
- Build and optimize Apache Spark workloads for batch processing, geospatial analytics, and large-scale aggregations
- Develop and maintain PostgreSQL, TimescaleDB, PostGIS, and DuckDB-based data solutions
- Implement efficient data ingestion, transformation, and bulk-loading pipelines without external connectivity
- Containerize and deploy applications using Docker and Docker Compose in air-gapped environments
- Work with S3-compatible object storage (e.g., MinIO) for on-prem data storage
- Proven experience deploying via Docker/Docker Compose in air-gapped or restricted-network environment.
- Comfortable working in Ubuntu-based environments with manual/local infrastructure management
- Strong proficiency in Python, SQL, Airflow, Spark, and PostgreSQL
- Experience with geospatial and time-series data
- Collaborate with customer teams to translate business requirements into data products
- Ensure data quality, lineage, observability, performance, and SLA compliance
- Troubleshoot production issues on-site/remotely without cloud tooling support
- Create and maintain technical documentation, data models, and operational run-books
- Strong communication, stakeholder management, and problem-solving skills
Benefits
- Comprehensive insurance coverage that gives you peace of mind, so you can focus on doing your best work
- Flexible work arrangements designed to support sustained productivity, personal well-being, and work-life balance
- Continuous learning and accelerated skill development through hands-on projects and mentorship from experienced industry leaders
- Global client exposure across 20+ countries, offering real-world experience with diverse markets and business environments.
- Opportunity to work on high-impact, large-scale projects that have collectively generated over $1B in measurable business value
- Competitive, market-aligned compensation packages that recognize performance, expertise, and long-term contribution
- Monthly demo days that celebrate innovation, showcase your work, and give you a real voice in what we build
- Annual recognition programs and performance-driven awards in a truly meritocratic environment
- Referral bonuses that reward you for helping grow a strong, like-minded team
- A strong problem-solving culture with opportunities to tackle meaningful, real-world challenges
- A positive, people-first workplace that supports happiness, balance, and long-term growth
Skills Required
- 7+ years of experience in data engineering and production-scale data platforms
- Hands-on experience with Apache Airflow
- Experience building and optimizing Apache Spark workloads
- Experience with PostgreSQL, TimescaleDB, PostGIS, and DuckDB
- Experience developing data ingestion, transformation, and bulk-loading pipelines in isolated environments
- Proven experience deploying with Docker and Docker Compose in air-gapped or restricted-network environments
- Experience with S3-compatible object storage such as MinIO
- Comfort working in Ubuntu-based environments with manual or local infrastructure management
- Strong proficiency in Python and SQL
- Experience with geospatial and time-series data
- Ability to translate business requirements into data products
- Experience ensuring data quality, lineage, observability, performance, and SLA compliance
- Production troubleshooting experience without cloud tooling support
- Ability to create technical documentation, data models, and operational runbooks
- Strong communication, stakeholder management, and problem-solving skills
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Spark Eighteen is a Delhi-based design and technology studio that builds scalable software products and digital solutions end to end. Its capabilities include AI and machine learning, web and application development, digital marketing, UI/UX and graphic design, content creation, and related brand-management services. The company works with founders, startups, and enterprises, combining global engineering and design talent to deliver impactful products and technology services.
