The Role
Build robust, reusable, and scalable data pipelines using Azure Databricks and Spark. Analyze structured and unstructured data, support prototype integration into intelligence products, and deploy pipelines through Azure DevOps CI/CD. Collaborate across engineering teams, communicate data insights to stakeholders, create JIRA dashboards, participate in design and feature definition, and produce weekly project status reports and detailed project plans.
Summary Generated by Built In
- Collect progress updates from Squad on a regular basis.
- Consolidate updates into weekly project status reports and share with the Squad Leads for their review.
- You participate in early state design and feature definition activities.
- Responsible for implementing robust data pipeline using Microsoft, Databricks Stack
- Responsible for creating reusable and scalable data pipelines.
- You are a Team-Player, collaborating with team members across multiple engineering teams to support the integration of proven prototypes into core intelligence products.
- You have strong communication skills to effectively convey complex data insights to non-technical stakeholders.
- Develop and manage L3-level project plans with sufficient details for tasks, milestones, and dependencies
Requirements
Skills:
- Knowledge and experience with relational and non-relational databases.
- Strong analytic skills related to working with structured and unstructured datasets.
- Hands-on experience in Azure Databricks utilizing Spark to develop ETL pipelines.
- Strong proficiency in data analysis, manipulation, and statistical modeling using tools like Spark, Python, Scala, SQL, or similar languages.
- Strong proficiency in building JIRA dashboards based on tasks.
- Strong experience in Project Management will be considered as an additional advantage.
- Azure DevOps experience to deploy the data pipelines through CI/CD.
Qualifications and Experience:
- Minimum 5-7 years of practical experience.
- Bachelor’s degree in computer science, software engineering, information technology, or a related field.
- Azure cloud stack in-production experience.
Skills Required
- Minimum 5–7 years of practical experience
- Bachelor’s degree in computer science, software engineering, information technology, or a related field
- Production experience with the Azure cloud stack
- Experience with relational and non-relational databases
- Strong analytical skills working with structured and unstructured datasets
- Hands-on experience with Azure Databricks and Spark for ETL pipeline development
- Proficiency in data analysis, manipulation, and statistical modeling using Spark, Python, Scala, SQL, or similar languages
- Strong proficiency building JIRA dashboards based on tasks
- Experience with Azure DevOps for CI/CD deployment of data pipelines
- Strong project management experience
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
TechCarrot is a global IT services provider and digital-transformation partner, delivering cost-effective, customer-centric digital solutions to clients worldwide. Its 200-plus technology specialists provide services across multiple industries, aiming to create seamless customer experiences and support client growth. Founded in 2016 and headquartered in Dubai, the company serves the Middle East and Africa through UAE operations, APAC through India, and has also expanded into the United States and the Netherlands.
