Sr.Data Engineer

Posted 13 Days Ago
Be an Early Applicant
CORP Colony, Tondiarpet, Chennai, Tamil Nadu, IND
In-Office
Senior level
Information Technology • Professional Services • Software • Consulting
The Role
Senior Data Engineer responsible for end-to-end design, development, optimization and monitoring of ETL/ELT data pipelines, data warehousing and data lake solutions. Duties include SQL optimization, building scalable big-data architectures (Kafka, Spark, Hadoop), implementing data security, performing root-cause analysis on large/unstructured datasets, and supporting cross-functional teams using cloud data platforms (Snowflake, Databricks, Redshift, BigQuery, Synapse).
Summary Generated by Built In

·       Development, maintenance, and enhancement of Data Pipelines (ETL/ELT) and processes with thorough knowledge of star/snowflake schemas

·       Developing complex SQL queries and SQL optimization

·       Hands on experience in PowerCenter/IDMC/IICS.

·       Development experience must be full Life Cycle experience including business requirements gathering, data sourcing, testing/data reconciliation, and deployment within Business Intelligence/Data Warehousing Architecture.

·       Designing and implementing data security

·       Monitoring and optimizing data storage and data processing

·       Use of Object function/object-oriented scripting languages including Scala, C++, Java and Python

·       Cloud data warehouse and data lake platforms (e.g., Snowflake, Databricks, Redshift, BigQuery, MS Synapse, MS ADLS, Apache Haddop, etc.)

 

·       Developing data services that are fit for purpose, resilient, scalable, and future proof, to meet user needs

·       Advanced SQL knowledge and experience working with relational databases and working familiarity with various databases.

·       Demonstrated understanding and experience using software and tools including ETL, relational SQL and NoSQL databases, big data tools like Kafka, Spark, and Hadoop

·       Experience building and optimizing ‘big data’ data pipelines, architecture, and data sets.

·       Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.

·       Strong analytic skills related to working with unstructured datasets.

·       Experience and knowledge of project management best practices and agile software development methodologies

·       A successful history of manipulating, processing, and extracting value from large, disconnected datasets.

·       Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.

·       Strong project management and organizational skills.

·       Experience supporting and working with cross-functional teams in a dynamic environment.

·       Understanding of data science concepts

·       Familiarity with AI/ML frameworks and libraries



Requirements
  • Procurement / Supplier data
  • Inventory and warehouse management data
  • Demand planning and forecasting
  • Production orders, BOMs, routings
  • Shop floor / MES data
  • Quality systems (defects, CAPA, nonconformance)


Skills Required

  • Develop, maintain, and enhance ETL/ELT data pipelines using star/snowflake schemas
  • Develop complex SQL queries and perform SQL optimization
  • Hands-on experience with Informatica PowerCenter, IDMC, or IICS
  • Full lifecycle development experience: requirements gathering, data sourcing, testing/data reconciliation, deployment
  • Designing and implementing data security controls
  • Monitoring and optimizing data storage and processing
  • Proficiency with object-oriented/scripting languages (Scala, C++, Java, Python)
  • Experience with cloud data warehouse and data lake platforms (Snowflake, Databricks, Redshift, BigQuery, Azure Synapse, Azure ADLS, Hadoop)
  • Experience building and optimizing big data pipelines, architectures, and datasets
  • Experience with ETL tools, relational SQL and NoSQL databases, and big data tools (Kafka, Spark, Hadoop)
  • Ability to perform root cause analysis on internal and external data and processes
  • Strong analytic skills working with unstructured datasets
  • Working knowledge of message queuing, stream processing, and highly scalable big data stores
  • Experience with project management best practices and agile software development methodologies
  • Experience supporting and working with cross-functional teams
  • Understanding of data science concepts
  • Familiarity with AI/ML frameworks and libraries
  • Domain data experience: procurement/supplier, inventory/warehouse management, demand planning/forecasting
  • Domain data experience: production orders, BOMs, routings, shop floor/MES data
  • Domain data experience: quality systems (defects, CAPA, nonconformance)
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
275 Employees
Year Founded: 2005

What We Do

Excelencia Consulting is a US-based, ISO 27001:2022-certified technology services provider specializing in Oracle solutions and Digital Engineering. The company provides on-site, offshore, and near-shore software technologies and ERP solutions to small and mid-sized enterprises globally. With a physical presence in the USA, UK, Canada, and India, Excelencia enables smarter operations and future-ready business transformations through its expertise in digital engineering services.

Similar Jobs

Magna International Logo Magna International

Asst. Manager - Logistics

Automotive • Hardware • Robotics • Software • Transportation • Manufacturing
Hybrid
Chennai, Tamil Nadu, IND
171000 Employees

TransUnion Logo TransUnion

Senior Analyst

Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Hybrid
Chennai, Tamil Nadu, IND
13000 Employees

Capco Logo Capco

Senior Project Program Portfolio Mgmt - Portfolio Manager

Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Remote or Hybrid
India
6000 Employees

CSC Logo CSC

Accountant

Fintech • Legal Tech • Software • Financial Services • Cybersecurity • Data Privacy
Remote or Hybrid
2 Locations
8500 Employees

Similar Companies Hiring

Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
178 Employees
Kepler  Thumbnail
Fintech • Software
New York, New York
6 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
60 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account