Data Engineer

Posted 3 Days Ago
Be an Early Applicant
Noida, Gautam Buddha Nagar, Uttar Pradesh, IND
In-Office
Senior level
Big Data • Information Technology • Software • Consulting
The Role
Design, build, and maintain scalable Azure-based ETL/ELT data pipelines and Lakehouse/Data Warehouse architectures. Lead and mentor a team while remaining hands-on in Python, Spark, and SQL development. Ensure data quality, performance, reliability, security, and compliance. Collaborate with BI, analytics, product, and business teams to translate requirements into robust, production-grade solutions and continuously improve architecture and processes.
Summary Generated by Built In

Data Engineering & Architecture

  • Design, develop, and maintain scalable, high-performance data pipelines

  • Work extensively with Azure Data Factory and Microsoft Fabric

  • Build robust ETL/ELT frameworks using Python

  • Design and optimize Lakehouse / Data Warehouse architectures

  • Handle large-scale datasets efficiently (high volume and throughput)

  • Write and optimize complex SQL queries for performance and reliability

  • Integrate data from multiple sources including APIs, transactional systems, and external platforms

Leadership & Delivery (Hands-on)

  • Lead and mentor a team of data engineers while remaining actively involved in coding and solution design

  • Perform hands-on development for critical pipelines, complex transformations, and performance optimisation.

  • Conduct code reviews and enforce best practices, design patterns, and coding standards

  • Act as the technical owner for data engineering deliverables

Quality, Performance & Reliability

  • Implement data quality checks, validations, and monitoring

  • Optimize pipelines for performance, scalability, and cost

  • Ensure reliability, fault tolerance, and error handling in production systems

  • Follow data security, access control, and compliance best practices

  • Lead troubleshooting, root-cause analysis, and production issue resolution

Collaboration & Continuous Improvement

  • Work closely with BI, analytics, product, and business teams

  • Translate business requirements into scalable technical solutions

  • Stay up to date with modern data engineering tools, technologies, and techniques

  • Proactively suggest architectural and process improvement



Requirements
  • 3-6 years of experience in the Data Engineering field.

  • Strong hands-on experience in Python for data engineering, including building and maintaining production-grade, large-scale data pipelines

  • Advanced experience with Azure Data Factory and Azure-based data platforms for orchestration, integration, and scalable data processing

  • Working experience with Microsoft Fabric, including Lakehouse and data engineering workloads, along with a strong understanding of ETL/ELT and data warehousing concepts

  • Expert-level SQL skills covering complex query development, optimization, indexing, and partitioning for high-performance systems

  • Proven experience handling large-volume, high-throughput data and distributed processing environment.

  • Experience with analytics and visualization platforms such as Power BI

  • Knowledge of Delta Lake, Spark, and distributed data processing frameworks

  • Experience implementing CI/CD practices for data pipelines and data engineering workflows

  • Exposure to data governance, lineage, metadata management, and compliance-driven environments such as fintech or high-transaction systems

  • Hands-on leadership mindset with strong ownership and accountability for outcomes

  • Ability to mentor, guide, and grow junior engineers while leading by example

  • Clear and effective communication with technical and non-technical stakeholders

  • Strong problem-solving, analytical reasoning, and decision-making skills



Benefits
  • Working hours: 10:00 AM – 7:00 PM

  • Working days: 5 days a week (plus 1st & 3rd Saturdays working)

  • Medical Insurance coverage for employees

  • Provident Fund (PF) facility

  • Quarterly parties and yearly outings/trips for team bonding

  • Regular check-ins with leadership for growth and feedback

  • Recognition awards to celebrate high performance

  • Fun activities and team engagement sessions throughout the year



Skills Required

  • 3-6 years of experience in Data Engineering
  • Python for building and maintaining production-grade, large-scale data pipelines
  • Azure Data Factory and Azure-based data platforms experience
  • Microsoft Fabric experience, including Lakehouse and data engineering workloads
  • Strong ETL/ELT and data warehousing concepts
  • Expert-level SQL, including query optimization, indexing, and partitioning
  • Proven experience handling large-volume, high-throughput data and distributed processing
  • Knowledge of Delta Lake, Spark, and distributed data processing frameworks
  • Experience implementing CI/CD practices for data pipelines and workflows
  • Experience with analytics and visualization platforms such as Power BI
  • Exposure to data governance, lineage, metadata management, and compliance-driven environments
  • Hands-on leadership mindset; mentor and guide junior engineers
  • Clear and effective communication with technical and non-technical stakeholders
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
54 Employees
Year Founded: 2018

What We Do

ScaleupAlly is a fast-growing technology services and software development company that empowers businesses to thrive in the digital age. The firm provides on-demand access to experts in technology solutions development, specializing in custom software, mobile app development, BI and Big Data consulting, and web development, leveraging a collaborative intelligence approach to create industry-changing solutions for startups and leaders worldwide.

Similar Jobs

Capco Logo Capco

Data Engineer

Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Remote or Hybrid
India
6000 Employees

Capco Logo Capco

Senior Data Engineer

Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Remote or Hybrid
India
6000 Employees

ShyftLabs Logo ShyftLabs

Data Engineer

Blockchain • Database • Analytics
In-Office
Noida, Gautam Buddha Nagar, Uttar Pradesh, IND
110 Employees

Wahed Invest Logo Wahed Invest

Data Engineer

Software • Financial Services
Remote or Hybrid
India
191 Employees

Similar Companies Hiring

Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
178 Employees
Kepler  Thumbnail
Fintech • Software
New York, New York
6 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
60 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account