Techstra is seeking an experienced Sr Azure Data Engineer with a minimum of 4-7 years' experience (ex: Data warehouse design, business intelligence reporting and Customer Information Systems) or in lieu of experience, 6-9 years equivalent combination of education and work experience.
From the day you join, you will hit the ground running surrounded by amazing people. Here is a look at some of the expectations and responsibilities for this position:
Responsibilities:
Design, test and build scalable ETL/ELT/Data pipelines using Databricks producing Data Products that support strategic priorities for our clients.
Integrate Databricks with cloud services
Ingest structured and unstructured data from multiple sources (e.g., Kafka, REST APIs, databases and cloud storage).
Write optimized Spark SQL or PySpark code for batch and streaming data transformations.
Implement data cleansing, validation, and enrichment processes.
Optimize jobs for performance, memory management, and cost-efficiency
Create Technical design documents
Develop Technical and procedural documentation, as required
Experience & Qualifications:
Databricks Platform
Proficient in using Databricks notebooks.
Delta Lake (versioning, ACID transactions).
Unity Catalog (data governance and lineage).
MLflow (for MLOps, if involved in machine learning workflows).
Azure Environment
Apache Spark
Deep understanding of Spark core concepts (RDDs, DataFrames, Spark SQL, Spark Streaming).
Performance tuning and optimization of Spark jobs.
Programming Languages
Python (PySpark): Essential for most Databricks tasks.
SQL: For data transformation and analysis.
Scala or Java: Sometimes required for advanced Spark use cases.
Data Engineering
ETL/ELT pipeline design and implementation.
Working with structured and unstructured data.
Batch vs. streaming data processing (Structured Streaming)
Integration with Collibra
DevOps & CI/CD
Git, Azure DevOps, or GitHub Actions.
Automated deployment of notebooks and workflows.
Data & Analytics Skills
Data modeling and warehousing (e.g., Star/Snowflake schema).
Working with BI tools (Power BI, Tableau).
Understanding of lakehouse architecture.
Security & Governance
Role-based access control (RBAC) and credential passthrough.
Data masking and audit logging.
Soft skills
Strong communication and collaboration (especially in cross-functional teams).
Experience writing technical design specifications for data integrations and data transformations with structured and unstructured data.
Problem-solving and debugging.
Experience with Scrum development methodology
At Techstra Solutions, we help top companies and brands achieve the business value of Digital and Talent Transformation. We believe there are three components in successful business transformation: Business Strategy, Technology and Talent. It is the coming together of these three disciplines that enables companies to take full advantage of opportunities. It differentiates us. Our approach is holistic and all encompassing. We consider the full picture as we guide our clients on this journey.
We are experts in transformation, business strategy, technology, innovation, and human capital management. We deliver our expertise through client consulting, innovative staffing solutions and software development. From strategy through implementation, we are dedicated to bringing our clients world-class business and talent solutions that fit strategic requirements and most importantly, deliver results.
Similar Jobs
What We Do
Techstra Solutions is a certified woman-owned consulting firm that provides strategy, technology and implementation support to large organizations undergoing digital and talent transformation. The rapid pace of technology innovation required to remain competitive, juxtaposed with the slow rate of change in many organizations, causes most to struggle.
To be successful companies must drive new ways of working including the use of technology/analytics and new skillsets (hard/soft) deep into their culture so that they are able to rapidly and continuously react to change. They must also apply new approaches that empower their employees, ensuring its sustainability.
At Techstra Solutions we focus on four key areas, otherwise known as the 4 A’s: Automation (Robotic Process Automation (RPA)/Intelligent Automation), Analytics (Reporting/Analytics/AI/ML), Application Modernization (Cloud, Mobility, etc), and Agile-based solutions (DevOps).
