We are looking for an "Sr Data Engineer/QA (Big Data Testing/Python Scripting) in Indore, MP."
Location: Indore, MP
Full Time
Experience: 5-10 Years
Must have:
- ETL/Data Testing/Azure Databricks + Azure Data Factory: 5+ years (Required)
- Strong SQL + Python/Py Spark: 5+ years (Required)
- Experienced in end-to-end of ETL (Data Pipeline) QA testing using various technologies in Azure cloud involving ingestion from varied data sources.
- Should be familiar with data warehousing target and intermediate structures ideally in environments using data lakes. Well versed in Databricks, SQL Database, Azure Data Factory, Azure Data Lake
- Must have hands-on experience in PySpark, SparkSQL, T-SQL
- Should be able to go through requirements, design specs and create appropriate test scenarios across various layers/stages of data pipelines.
- Should be able to write reusable PySpark and T-SQL script to validate data across layers.
- Experience of working in Databricks Delta Lake will be highly desirable.
- Ideally should have worked in a team using Agile methodology.
- Appreciation of data, its quality, and its usage for business benefit
- Very good communication and written skills
Nice to Have
• Experience in the Banking/Finance domain
📩 Interested candidates can apply or share their profiles with us at "[email protected]"
Top Skills
What We Do
Driving Innovation through Advanced Data and AI Services. We offer next-level data and AI services to help you transform your data into actionable insights for a competitive edge. To augment your teams, we offer full-time/contract resources & consulting, design and development services for turnkey projects in the following areas: 01 DATA MODERNIZATION Define Cloud strategy, architecture & roadmap Identify current landscape & catalog data sources Data warehouse design & setup Develop data governance & data quality framework Implement data management technologies Build data analytics capabilities Inculcate data driven culture A future-ready framework to serve all business use cases 02 DATA INTEGRATION & CLOUD MIGRATION Design & develop Optimized data models and data pipelines ETL/ELT workloads & monitoring Processes that scale up and down without performance problems Automate data migration between upstream/downstream (on-prem) systems and Cloud 03 DATA ANALYTICS & AI Data analytics consulting – use-case exploration and building analytics roadmap Data preparation and creating a single version of truth Dashboard & report development AI/ML model selection and development AI/ML model tuning and validation AI/ML model scaling, integration and deployment
