Principal Data Engineer (Azure)

Posted 4 Days Ago
Dallas, TX
Senior level
Big Data • Analytics • Business Intelligence • Big Data Analytics
The Role
As a Principal Data Engineer, you will design and build data ingestion pipelines, conceptualize high-performance data processing, and collaborate with multiple teams to deliver analytical solutions using Azure and big data technologies. You will also ensure the quality and scalability of your code while engaging with stakeholders across the organization.
Summary Generated by Built In

Description

Tiger Analytics is a global AI and analytics consulting firm. With data and technology at the core of our solutions, we are solving problems that eventually impact the lives of millions globally. Our culture is modeled around expertise and respect with a team-first mindset. Headquartered in Silicon Valley, you’ll find our delivery centers across the globe and offices in multiple cities across India, the US, UK, Canada, and Singapore, including a
substantial remote global workforce.

We’re Great Place to Work-Certified™. Working at Tiger Analytics, you’ll be at the heart of an AI revolution. You’ll work with teams that push the boundaries of what is possible and build solutions that energize and inspire.

Requirements

Curious about the role? What your typical day would look like?

As a Principal Data Engineer (Azure), you would have hands on experience working on Azure as cloud, Databricks and some exposure/experience on Data Modelling. You will build and learn about a variety of analytics solutions & platforms, data lakes, modern data platforms, data fabric solutions, etc. using different Open Source, Big Data, and Cloud technologies on Microsoft Azure.

● Design and build scalable & metadata-driven data ingestion pipelines (For Batch and Streaming Datasets)

● Conceptualize and execute high-performance data processing for structured and unstructured data, and data
harmonization

● Schedule, orchestrate, and validate pipelines

● Design exception handling and log monitoring for debugging

● Ideate with your peers to make tech stack and tools-related decisions

● Interact and collaborate with multiple teams (Consulting/Data Science & App Dev) and various stakeholders to meet deadlines, to bring Analytical Solutions to life.

What do we expect?

● Experience in implementing Data Lake with technologies like Azure Data Factory (ADF), PySpark, Databricks, ADLS,

Azure SQL Database

● A comprehensive foundation with working knowledge of Azure Synapse Analytics, Event Hub & Streaming
Analytics, Cosmos DB, and Purview

● A passion for writing high-quality code and the code should be modular, scalable, and free of bugs (debugging
skills in SQL, Python, or Scala/Java).

● Enthuse to collaborate with various stakeholders across the organization and take complete ownership of
deliverables.

● Experience in using big data technologies like Hadoop, Spark, Airflow, NiFi, Kafka, Hive, Neo4J, Elastic Search

● Adept understanding of different file formats like Delta Lake, Avro, Parquet, JSON, and CSV

● Good knowledge of building and designing REST APIs with real-time experience working on Data Lake or
Lakehouse projects.

● Experience in supporting BI and Data Science teams in consuming the data in a secure and governed manner

● Certifications like Data Engineering on Microsoft Azure (DP-203) or Databricks Certified Developer (DE) are
valuable addition.

Note: The designation will be commensurate with expertise and experience. Compensation packages are among the best in the industry.

Job Requirement

  • Mandatory: Azure Data Factory (ADF), PySpark, Databricks, ADLS, Azure SQL Database
  • Optional: Azure Synapse Analytics, Event Hub & Streaming Analytics, Cosmos DB and Purview.
  • Strong programming, unit testing & debugging skills in SQL, Python or Scala/Java.
  • Some experience of using big data technologies like Hadoop, Spark, Airflow, NiFi, Kafka, Hive, Neo4J, Elastic
    Search.
  • Good Understanding of different file formats like Delta Lake, Avro, Parquet, JSON and CSV.
  • Experience of working in Agile projects and following DevOps processes with technologies like Git, Jenkins & Azure DevOps.
  • Good to have:
  • Experience of working on Data Lake & Lakehouse projects
  • Experience of building REST services and implementing service-oriented architectures.
  • Experience of supporting BI and Data Science teams in consuming the data in a secure and governed manner.
  • Certifications like Data Engineering on Microsoft Azure (DP-203) or Databricks Certified Developer (DE)
Benefits

This position offers an excellent opportunity for significant career development in a fast-growing and challenging entrepreneurial environment with a high degree of individual responsibility.

Top Skills

Java
Python
Scala
SQL
The Company
Bengaluru, Bengaluru
5,000 Employees
On-site Workplace
Year Founded: 2011

What We Do

Tiger Analytics is a global leader in AI and Analytics, helping Fortune 1000 companies solve their toughest challenges. We offer fullstack AI and analytics services & solutions to empower businesses to achieve real outcomes and value at scale. We are on a mission to push the boundaries of what AI and analytics can do to help enterprises navigate uncertainty and move forward decisively. Our purpose is to provide certainty to shape a better tomorrow.

Our team of 4000+ technologists and consultants are based in the US, Canada, the UK, India, Singapore, and Australia, working closely with clients across CPG, Retail, Insurance, BFS, Manufacturing, Life Sciences, and Healthcare.

We are Great Place to Work-Certified™ and have been recognized by analyst firms such as Forrester, Gartner, Everest, ISG, HFS, and others. Ranked among the ‘Best’ and ‘Fastest Growing’ analytics firms lists by Inc., Financial Times, Economic Times and Analytics India Magazine.

In India, our offices are located in Chennai, Hyderabad and Bangalore.

Similar Jobs

Coppell, TX, USA
18996 Employees

Egen Logo Egen

Cloud Data Engineer

Artificial Intelligence • Big Data • Cloud • Information Technology • Machine Learning
Dallas, TX, USA
240 Employees

Capital One Logo Capital One

Data Science Manager, US Card

Fintech • Machine Learning • Payments • Software • Financial Services
Hybrid
Plano, TX, USA
55000 Employees
201K-230K Annually

Capital One Logo Capital One

Distinguished Machine Learning Engineer (Director IC)

Fintech • Machine Learning • Payments • Software • Financial Services
Hybrid
Plano, TX, USA
55000 Employees

Similar Companies Hiring

Energy CX Thumbnail
Utilities • Professional Services • Greentech • Financial Services • Energy • Consulting • Business Intelligence
Chicago, IL
55 Employees
MassMutual India Thumbnail
Insurance • Information Technology • Fintech • Financial Services • Big Data
Hyderabad, Telangana
InCommodities Thumbnail
Renewable Energy • Machine Learning • Information Technology • Energy • Automation • Analytics
Austin, TX
234 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account