Exp: 5 to 12 Years
Job Description:
Please consider Python as ETL tool to read data from source perform all datatype & data transformation, error handling ,logging within python and load to data bricks managed delta tables.
Anyone we are hiring should be very good at python along with very good knowledge in Databricks & PySpark so that they can evaluate and perform transformation within Databricks as well. Candidate should be able to do almost everything in python including class ,objects, methods, dictionaries ,loops, Wheel files ,packages, should be able to connect to any source data including any DBs, streaming data.
Expertise in Azure services such as Azure SQL Database, Azure Data Factory, Azure Synapse Analytics, and Azure Data Lake
• The ideal candidate has extensive experience in data engineering, working with large-scale data infrastructures, and optimizing ETL processes
• Expertise in Azure services such as Azure SQL Database, Azure Data Factory, Azure Synapse Analytics, and Azure Data Lake
• At least 6+ years of experience in data engineering, with a focus on building data pipelines
• Proven experience with Spark, Scala, and other related technologies
• Familiarity with Could Warehouses, data lakes, storage formats.
• Strong knowledge of data modeling and ETL processes
• Experience leading data engineering projects.
• Proven experience with Databricks and Apache Spark
• Strong proficiency in Python
• Experience in Unix shell scripting and scheduling tools
Similar Jobs
What We Do
emids is healthcare's digital transformation leader, delivering business and tech solutions that help payers, providers and tech-enablers maximize technology to deliver care better.
As a global partner headquartered in Nashville, TN, emids helps bridge the critical gaps in accessible, affordable, high-quality healthcare by providing advisory consulting services, custom application development, and data solutions. Services include EHR application deployment and management, analytics, data integration and governance, software development and testing, and business intelligence.
