The Role
Design, build, and maintain scalable ETL/ELT pipelines, data platforms, warehouses, lakes, and models. Integrate structured and unstructured data, optimize distributed processing and SQL performance, and ensure data quality, governance, security, lineage, and observability. Lead cloud migrations, modernization, resilience, and incident response initiatives while collaborating with engineering, analytics, data science, risk, compliance, and product teams. Translate business requirements into reliable data solutions and mentor engineers.
Summary Generated by Built In
Job Title: Senior Data Engineer
Exp: 6-12 Yrs
Notice Period: 0-30 days
Mode of Work: Hybrid
Work Location: Hyd/Pune
We are seeking a skilled Data Engineer to design, build, and maintain scalable data pipelines and infrastructure that support analytics, reporting, and machine learning initiatives. The ideal candidate has strong experience with SQL, Python, cloud platforms, and modern data engineering tools. You will collaborate with data analysts, data scientists, and software engineers to ensure reliable, high-quality data is available across the organization.
Job Description:
- Design, develop, and maintain scalable ETL/ELT data pipelines.
- Build and optimize data warehouses, data lakes, and data models.
- Integrate data from multiple sources, including APIs, databases, and third-party systems.
- Ensure data quality, consistency, security, and governance.
- Monitor and troubleshoot data pipelines and resolve performance issues.
- Optimize SQL queries and database performance.
- Collaborate with cross-functional teams to understand data requirements and deliver solutions.
Requirements
Core Data Engineering
& Architecture
- Design
and build end-to-end data pipelines (ETL/ELT) for structured and
unstructured data
- Develop
scalable data platforms handling terabytes of data with high reliability
and low latency
- Strong
expertise in data modeling, warehousing, and lakehouse architectures
- Own
data quality, lineage, governance, and observability frameworks
- Hands-on
experience with Spark, Hadoop, Kafka, Flink (batch + real-time processing)
- Build
and optimize high-throughput, distributed data systems
- Experience
in streaming + event-driven architectures for large-scale financial data
- Deep
expertise in AWS / Azure / GCP (Data Lakes, Warehouses, Compute, Storage)
- Tools:
Databricks, Snowflake, Redshift, Synapse, BigQuery
- Pipeline
orchestration using Airflow, Prefect, or similar frameworks
- Strong
coding in Python, SQL, Scala
- Focus
on performance optimization, reliability, and production-grade systems
- Experience
with CI/CD, DevOps, and infrastructure-as-code
- Ability
to design architectures and make technology decisions at scale
- Strong
understanding of:
- Trade
lifecycle, transactions, risk, compliance, regulatory reporting
- Customer
360, payments, lending, capital markets data and Wealth Management
Business
- GenAI
/ LLM integration (RAG, embeddings, vector stores)
- Lead
legacy-to-cloud data platform migrations
- Drive
data re-engineering, system decomposition, and modernization initiatives
- Drive
resilience, failover, and incident response frameworks
- Work
across engineering, data science, risk, compliance, and product teams
- Ability
to translate business needs (risk, reporting, revenue) into data solutions
- Mentor
engineers and drive engineering best practices and standards
Benefits
Standard Company Benefits
Skills Required
- 6-12 years of professional experience
- Design and build end-to-end ETL/ELT data pipelines for structured and unstructured data
- Develop scalable data platforms handling terabytes of data with high reliability and low latency
- Strong expertise in data modeling, data warehousing, and lakehouse architectures
- Experience owning data quality, lineage, governance, and observability frameworks
- Hands-on experience with Spark, Hadoop, Kafka, and Flink
- Experience building high-throughput distributed data systems
- Experience with streaming and event-driven architectures for large-scale financial data
- Deep expertise in AWS, Azure, or GCP data lakes, warehouses, compute, and storage
- Experience with Databricks, Snowflake, Redshift, Synapse, or BigQuery
- Experience with Airflow, Prefect, or similar pipeline orchestration frameworks
- Strong coding skills in Python, SQL, and Scala
- Experience with performance optimization, reliability, and production-grade systems
- Experience with CI/CD, DevOps, and infrastructure as code
- Ability to design architectures and make technology decisions at scale
- Understanding of trade lifecycle, transactions, risk, compliance, and regulatory reporting
- Understanding of customer 360, payments, lending, capital markets, and wealth management data
- Experience integrating GenAI or LLM capabilities, including RAG, embeddings, or vector stores
- Experience leading legacy-to-cloud data platform migrations
- Experience driving data re-engineering, system decomposition, and modernization initiatives
- Experience with resilience, failover, and incident response frameworks
- Ability to collaborate across engineering, data science, risk, compliance, and product teams
- Ability to translate risk, reporting, and revenue requirements into data solutions
- Experience mentoring engineers and driving engineering best practices
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
DATAECONOMY is a global, cloud-first data and AI consultancy delivering enterprise-grade solutions through an innovative intellectual-property suite. Its work spans data and BI platform modernization, self-service AI, data mesh and fabric, master data management, governance, cloud enablement, digital engineering, knowledge graphs, and machine lakes supporting cybersecurity and financial-crime use cases for enterprise clients.
