Senior Engineer- Python Pyspark

Job Posted 25 Days Ago Posted 25 Days Ago
Be an Early Applicant
Bangalore, Bengaluru, Karnataka
Mid level
Artificial Intelligence • Big Data • Information Technology • Security • Software
The Role
The role involves writing and optimizing distributed data processing code using Python and PySpark, developing ETL processes, and ensuring performance and scalability of applications.
Summary Generated by Built In

Location: Bangalore, India

Thales people architect solutions that enable two-thirds of planes to take off and land safely. We create in-flight entertainment systems that engross 50 million fliers every year and we develop the avionics that control the world’s largest commercial aircrafts. Our simulators train the next generation of pilots for fighter jets, transporters and search and rescue helicopters. And, together, each and every member of our aerospace team makes a difference.

Present in India since 1953, Thales is headquartered in Noida, Uttar Pradesh, and has operational offices and sites spread across Bengaluru, Delhi, Gurugram, Hyderabad, Mumbai, Pune among others. Over 1800 employees are working with Thales and its joint ventures in India. Since the beginning, Thales has been playing an essential role in India’s growth story by sharing its technologies and expertise in Defence, Transport, Aerospace and Digital Identity and Security markets.

Essential Functions / Key Areas of Responsibility

  • Expert in Python, with knowledge of PySpark
  • Writing distributed data processing code with PySpark
  • Optimizing Spark jobs using DataFrames API and RDDs
  • Working with Spark SQL for querying large datasets (structured sql/mysql database)
  • Implementing UDFs (User Defined Functions) for custom transformations
  • Performance tuning (caching, partitioning, broadcast joins, threading, multiprocessing, Predicate Pushdown)
  • ETL development using Databricks workflows and Delta Live Tables
  • Data pipelines with Databricks Jobs
  • Streaming data processing with Structured Streaming ( event hub or AWS SQS)
  • Non structured file processing like parquet , columnar data format
  • Responsible for writing reusable, testable and efficient code — ensuring the application's performance, security and scalability.
  • Excellent problem-solving ability with solid communication and collaboration skills.
  • Provide technical documentation for system, features, and components
  • Follow and support agile methodologies and practices by actively participating in all SCRUM ceremonies

Minimum Requirements: Skills, Experience, Education, Technical/Specialized Knowledge, Certifications, Language.

  • Bachelor’s degree in Engineering, Computer Science or related study
  • Proven experience as a Python language with PySpark
  • Minimum 3+ years of experience
  • Understanding of data structures, data modeling and software architecture
  • Ability to write robust code in Python
  • Outstanding analytical and problem-solving skills
  • Experience with Git, JIRA, Confluence
  • Experience in working on Linux OS.
  • Excellent written and verbal communication.

At Thales we provide CAREERS and not only jobs. With Thales employing 80,000 employees in 68 countries our mobility policy enables thousands of employees each year to develop their careers at home and abroad, in their existing areas of expertise or by branching out into new fields. Together we believe that embracing flexibility is a smarter way of working. Great journeys start here, apply now!

Top Skills

Confluence
Databricks
Delta Live Tables
Git
JIRA
Linux
Pyspark
Python
Spark Sql
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
Arlington, VA
63,258 Employees
On-site Workplace

What We Do

Thales is a global high technology leader investing in digital and “deep tech” innovations – connectivity, big data, artificial intelligence, cybersecurity and quantum technology – to build a future we can all trust, which is vital to the development of our societies. The company provides solutions, services and products that help its customers – businesses, organisations and states – in the defence, aeronautics, space, transportation and digital identity and security markets to fulfil their critical missions, by placing humans at the heart of the decision-making process.

Similar Jobs

emids Logo emids

Sr. Data Engineer(Python+Pyspark+Databricks) || 6+ yrs ||RR/1162/2024

Healthtech • Information Technology • Other • Software • Analytics
Bangalore, Bengaluru, Karnataka, IND
2100 Employees

Cargill Logo Cargill

Senior Solution Integrity Analyst (SAP TM)

Food • Greentech • Logistics • Sharing Economy • Transportation • Agriculture • Industrial
Bengaluru, Karnataka, IND
155000 Employees

Atlassian Logo Atlassian

Senior Frontend Software Engineer

Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Remote
Bengaluru, Karnataka, IND
11000 Employees

Atlassian Logo Atlassian

Principal Backend Software Engineer

Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Remote
Bengaluru, Karnataka, IND
11000 Employees

Similar Companies Hiring

True Anomaly Thumbnail
Software • Machine Learning • Hardware • Defense • Artificial Intelligence • Aerospace
Colorado Springs, CO
131 Employees
Caliola Engineering Thumbnail
Software • Machine Learning • Hardware • Defense • Data Privacy • App development • Aerospace
Colorado Springs, CO
53 Employees
Red 6 Thumbnail
Virtual Reality • Software • Hardware • Defense • Aerospace
Orlando, Florida
113 Employees
Not Eligible
Save
By clicking Apply you agree to share your profile information with the hiring company.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account