Data Engineering Consultant (Python,Pyspark OR Scala, Data Bricks)

Posted 2 Days Ago
Be an Early Applicant
Noida, Gautam Buddha Nagar, Uttar Pradesh, IND
In-Office
Senior level
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
The Role
Design, develop, test, document, and maintain scalable big data applications and cloud data platforms using Python, PySpark, Scala, Azure, Spark, Kafka, and Airflow. Build analytics rule engines, predictive algorithm frameworks, automation solutions, and AI-powered applications. Optimize performance, migrate applications, enforce code quality, support production systems, and collaborate across teams in an Agile environment.
Summary Generated by Built In
Requisition Number: 2385855
Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.
Primary Responsibilities:
  • Design, Code, Test, document, and maintain high-quality and scalable Big Data, application using PySpark and Scala Spark on Cloud solutions on Azure
  • Develop and implement analytics rules engines and Predictive Algorithm and Unit Testing framework eg PyTest
  • Research, evaluate, and deploy new automation tools, frameworks, and patterns to build sustainable Big Data platform for batch and streaming applications using Kafka
  • Identify gaps and opportunities for improvement of existing solutions
  • Ensure the code is meeting the desired quality checks using Sonar
  • Schedule the deployed pipelines using Airflow following proper dependency hierarchy of jobs
  • Write technical documentation and participate in production support
  • Performance Optimization and application migration experience
  • Builder Responsibilities:
    • Design, develop, and deploy AI-powered solutions using no-code, low-code, and advanced platforms, translating business needs into scalable applications that enhance products, workflows, and decision-making
  • Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

Required Qualifications:
  • Bachelor/master's degree in computer science or information technology or equivalent with 7+ years of work experience
  • Solid experience in programming languages like Python,Pyspark, Scala
  • Good experience in job schedulers like Airflow
  • Good experience in working on cloud environment, preferably Azure
  • Proven experience in cloud services like Azure Kubernetes, Azure Data Lake, Azure Databricks, Azure Container Registries, Azure Service Bus, Azure Key Vault
  • Solid experience and expertise in writing complex queries in SQL and PL/SQL
  • Experience in writing Shell scripting and Snowflake
  • Experience with Continuous Integration CI/CD pipelines using Jenkins, GitHub Actions
  • Experience working in an agile environment
  • Experience working across teams and proven track record in solution focused problem-solving skills
  • Expertise in BigData frameworks like Spark and good understanding of Hadoop concepts

At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission.

Skills Required

  • Bachelor's or master's degree in computer science, information technology, or equivalent experience
  • 7+ years of work experience
  • Programming experience with Python, PySpark, and Scala
  • Experience with Airflow or similar job schedulers
  • Experience working in cloud environments, preferably Azure
  • Experience with Azure Kubernetes, Azure Data Lake, Azure Databricks, Azure Container Registry, Azure Service Bus, and Azure Key Vault
  • Expertise writing complex SQL and PL/SQL queries
  • Experience with Shell scripting and Snowflake
  • Experience with CI/CD pipelines using Jenkins and GitHub Actions
  • Experience working in an Agile environment
  • Experience collaborating across teams and solving problems with a solution-focused approach
  • Expertise with big data frameworks such as Spark and understanding of Hadoop concepts

What the Team is Saying

Optum Compensation & Benefits Highlights

  • Leave & Time Off Breadth — PTO is generally described as decent or good, and many note it as a strong part of the package. Actual ability to take time off can depend on workload and team coverage.
  • Retirement Support — Offerings include a 401(k) with employer match and access to an employee stock purchase plan, which are highlighted as meaningful components of total rewards. These programs are consistently referenced among core benefits.
  • Parental & Family Support — Parental and caregiver leave, along with adoption assistance, are publicly highlighted and viewed as notable elements of the package. Availability can be role-specific, but these supports contribute to overall breadth where offered.

Optum Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Eden Prairie, MN
160,000 Employees
Year Founded: 2011

What We Do

Optum, part of the UnitedHealth Group family of businesses, is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together. At Optum, we support your well-being with an understanding team, extensive benefits and rewarding opportunities. By joining us, you’ll have the resources to drive system transformation while we help you take care of your future. We recognize the power of connection to drive change, improve efficiency and make a difference in health care. Join a team where your skills and ideas can make an impact and where collaboration is key to creating technology that produces healthier outcomes.

Gallery

Gallery
Gallery
Gallery

Optum Offices

Hybrid Workspace

Employees engage in a combination of remote and on-site work.

Optum has three workplace models that balance the needs of the business and the responsibilities of each role. These models, core on‑site (5 days/week), hybrid (4 days/week) and telecommute or fully remote, vary by country, role and location.

Typical time on-site: Not Specified
HQEden Prairie, MN
Metro Manila, Philippines
Cebu, Philippines
Davao, Philippines
Ann Arbor, MI
Atlanta, GA
Baltimore, MD
Bengaluru, India
Chennai, India
Dallas, TX
Detroit, MI
Dublin, Ireland
Hartford, CT
Houston, TX
Hyderabad, India
Jacksonville, FL
Las Vegas, NV
Letterkenny, Ireland
Louisville, KY
Madison, WI
Minneapolis, MN
Nashville, TN
New Delhi, India
Philadelphia, PA
Phoenix, AZ
Pune, India
Raleigh, NC
San Diego, CA
Washington, DC
Learn more

Similar Jobs

Optum Logo Optum

Consultant

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office
Noida, Gautam Buddha Nagar, Uttar Pradesh, IND
160000 Employees

Optum Logo Optum

Claims Representative Associate

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office
Noida, Gautam Buddha Nagar, Uttar Pradesh, IND
160000 Employees

Optum Logo Optum

Director AI/ML Engineering

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office
Noida, Gautam Buddha Nagar, Uttar Pradesh, IND
160000 Employees

Optum Logo Optum

Data Analyst

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office
Noida, Gautam Buddha Nagar, Uttar Pradesh, IND
160000 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account