The Role
Design, build, test, maintain, and monitor scalable Lakeflow Spark Declarative Pipelines using Event Hubs and medallion architecture. Gather requirements, transform data across pipeline layers, troubleshoot complex issues, and create technical documentation. The role requires proficiency in Python, SQL, PySpark, Spark SQL, Databricks, Azure, and Unity Catalog, with streaming and declarative pipeline experience preferred.
Summary Generated by Built In
Sr. Data Engineer
Data Engineer with 4+ years of experience in SQL, PySpark, ETL, ADLS, Azure Databricks, Unity Catalog, and Azure technologies.
Responsibilities- 3+ years of experience developing data pipelines using Python/PySpark, with proficiency in Spark SQL and Azure Databricks.
- 4+ years of experience with Azure data engineering technologies, including ADLS, Unity Catalog, Azure Data Factory, Azure Databricks, and Azure SQL/Synapse.
- 4+ years of experience with SQL.
- Experience in data modelling, including dimensional modelling, star schema and snowflake schemas.
- Proficiency in version control tools such as Git and project management tools such as Jira.
- Excellent verbal and written communication skills.
- UG: B.Sc, BCA, or B.Tech/B.E. in any specialization.
- Reliable internet connectivity is required.
- Experience working with financial data or finance-related datasets.
- Experience with Structured Streaming and Azure Event Hubs.
- Experience with Declarative Automation Bundles and Spark Declarative Pipelines (SDP).
- Azure certifications.
Skills Required
- Proficiency in Python
- Proficiency in SQL
- Proficiency in PySpark
- Proficiency in Spark SQL
- Proficiency with Databricks
- Proficiency with Microsoft Azure
- Proficiency with Unity Catalog
- Strong analytical and troubleshooting skills
- Attention to detail
- Proactivity and ownership
- Communication and teamwork skills
- Hands-on experience with Structured Streaming and Event Hubs
- Experience with Declarative Automation Bundles
- Experience with Spark Declarative Pipelines
- Experience with SDP-meta
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
InXiteOut is a strategic AI and data analytics partner for enterprises. It advises, architects, builds, and deploys production-grade AI solutions, data platforms, analytics systems, custom applications, and automation. The company helps organizations identify high-value use cases, turn complex data into actionable intelligence, and embed AI into daily workflows, enabling faster decisions, smarter operations, measurable business impact, and scalable growth across industries.
