Role Overview:
The Data Integration Engineer is responsible for integrating data from various sources, managing ETL/ELT pipelines on Databricks, and ensuring data pipeline scalability and compliance. Key skills include Databricks, Spark, Python, SQL, AWS, cloud computing, and big data technologies. The role involves collaboration with multiple departments and requires expertise in data integration, pipeline management, and automated monitoring strategies.
This is a remote position in India. We will only consider candidates currently in India and are not offering relocation assistance at this time.
About the role:
- Integrate data from a variety of source systems and knowledge to tailor integration approaches on a per-system basis.
- Liaise with Technical Data Steward and understanding business requirements for ingestion and integration.
- Document ingestion and integration flows and pipelines.
- Use Airflow to schedule and automate ingestion jobs.
- Manage metadata and master data in technical data catalog.
- Manage, maintain, and oversee ETL/ELT pipelines on the Databricks platform.
- Ensure ELT/ETL design meets required security and compliance guidelines, and ensure PII management, flagging, and risk assessment during ingestion and integration.
- Conduct pipeline monitoring.
- Ensure adherence to SDLC best practices.
About you:
- Bring 5+ years of experience in data engineering, ingestion pipelining, and ETL/ELT.
- Hands-on experience with and understanding of Spark/Scala, SQL, Python/PySpark, Databricks, ETL/ELT development using tools like Apache Airflow, omibo, data quality guidelines, CI/CD pipelines, Agile, and Git/version control.
- Good to have AWS cloud experience.
#LI-Hybrid
Company Overview
McAfee is a leader in personal security for consumers. Focused on protecting people, not just devices, McAfee consumer solutions adapt to users’ needs in an always online world, empowering them to live securely through integrated, intuitive solutions that protects their families and communities with the right security at the right moment.
Company Benefits and Perks:
We work hard to embrace diversity and inclusion and encourage everyone at McAfee to bring their authentic selves to work every day. We offer a variety of social programs, flexible work hours and family-friendly benefits to all of our employees.
- Bonus Program
- Pension and Retirement Plans
- Medical, Dental and Vision Coverage
- Paid Time Off
- Paid Parental Leave
- Support for Community Involvement
We're serious about our commitment to diversity which is why McAfee prohibits discrimination based on race, color, religion, gender, national origin, age, disability, veteran status, marital status, pregnancy, gender expression or identity, sexual orientation or any other legally protected status.
Top Skills
What We Do
McAfee is a global organization with a 30-year history and a brand known the world over for innovation, collaboration and trust. McAfee’s historical accomplishments are founded upon decades of threat and vulnerability research, product innovation, practical application and a brand which individuals, organizations and governments have come to trust.
