We are looking for an astute, proficient and qualified Data Engineer to assess, analyze and work with data concepts, use-cases & complex new data sources to provide business insights to customers and support the implementation & integration of the data sources into the platform.
Key Responsibilities
Functional
· Solve challenging problems, using python coding skills.
· Design, build and launch new data extraction, transformation & loading processes in production.
· Web crawling, data cleaning, data annotation, data ingestion and data processing.
· Reading and collating complex data sets.
· Creating and maintaining data pipelines.
· Continual focus on process improvement to drive efficiency and productivity within the team.
· Use of Python, SQL, ES, Shell etc. to build the infrastructure required for optimal extraction, transformation, and loading of data.
· Provide insights into key business performance metrics by building analytical tools that utilize the data pipeline.
· Support the wider business with their data needs on an ad hoc basis.
· Comply with QHSE (Quality Health Safety and Environment), Business Continuity, Information Security, Privacy, Risk, Compliance Management and Governance of Organizations policies, procedures, plans and related risk assessments.
Requirements
Requirements:
· Bachelor's degree in computer engineering, Computer Science, or Electrical Engineering and Computer Sciences.
· 3+ years of programming experience, solid coding skills in Python, Shell, and Java
· Good corporate capacity, good communication skills.
· Experience with Web crawling, cleaning.
· Experience with solution architecture, data ingestion, query optimization, data segregation, ETL, ELT, AWS, EC2, S3, SQS, lambda, Elastic Search, Redshift, CI/CD frameworks and workflows.
· Working knowledge of data platform concepts - data lake, data warehouse, ETL, big data processing (designing and supporting variety/velocity/volume), real time processing architecture for data platforms, scheduling and monitoring of ETL/ELT jobs
· PostgreSQL and programming (preferably Java, Python), proficiency in understanding data, entity relationships, structured & unstructured data, SQL and NoSQL databases
· Knowledge of best practice in optimizing columnar and distributed data processing system and infrastructure
· Experienced in designing and implementing dimensional modelling
· Knowledge of machine learning and data mining techniques in one or more areas of statistical modelling, text mining and information retrieval.
Ideally, you’ll also need
· In-depth market and domain knowledge
· A passion for constant improvement
· An innovative and creative approach to problem-solving
· Excellent communication skills
Benefits
Competitive salary
Class A Medical Insurance
Top Skills
What We Do
Optimiza, is a leading, regional Systems Integration and digital transformation solutions provider that supports its clients' pursuit of operational excellence and profitability.
Our IP solutions cover a wide spectrum of sectors and provide clients with highly secure, user-friendly, versatile, and seamless systems in a variety of work areas including document management, healthcare, insurance, accounting, HR, and banking.
With over 41 years of operational experience, hundreds of projects delivered, and intellectual capital that spans multiple industry sectors, Optimiza's team of over 400 experts is fully capable of integrating and delivering innovative consulting, business, and technology solutions with a commitment to excellence and client satisfaction.
