At OCLC, we build technology with a purpose: to connect libraries and make knowledge accessible worldwide, because we believe that what is known must be shared. Our teams work with complex global datasets, AI and machine learning, hybrid cloud solutions, and other technologies that connect people and organizations to the information they need. We value the power of unique perspectives and experiences to unlock innovation. At OCLC, your ideas matter, whether you have two years of experience or 20. You’ll learn, create, and problem-solve with technologists, product developers, librarians, researchers, marketing pros, and support teams around the world.
OCLC is consistently recognized as a best place to work by several independent programs. We recognize and reward people and results with a comprehensive Total Rewards package. This means competitive compensation that reflects your unique contributions—performance, experience, and skills—along with exceptional benefits, including best-in-class health coverage, retirement plans with generous company contributions, and a commitment to your overall well-being.
We know the best ideas don’t always happen at a desk. Take a walking meeting around our 100-acre campus or enjoy lunch on the patio. We’re committed to your success—both personally and professionally. Hybrid work environment: For many roles, three days a week on-site, with occasional additional days based on business needs.
Free use of our on-site fitness center, gym sports, group exercise classes, and game room
Onsite catering and cafeteria subsidized by OCLC
Health and wellness events
Work environments with individual and team spaces and the latest technology tools
Paid parental leave and adoption assistance
Tuition reimbursement and Public Service Loan Forgiveness eligibility
Company-subsidized pricing on local tickets and memberships
Join us in transforming how people everywhere access information and be part of a mission-driven team that makes a global impact.
The job details are as follows:As a Lead Software Engineer, you will be responsible for leading software development initiatives. You will independently design, develop, and test complex software programs and systems. You will also collaborate with team members, mentor junior engineers, and provide technical guidance to ensure the delivery of high-quality software solutions. You will also collaborate with product managers, designers, and other engineers to define, refine, and implement features and enhancements.Responsibilities:
- Own and evolve large-scale data flows end-to-end—batch processing, transformation, clustering, and publication of datasets and indexes—ensuring they are reliable, documented, and ready for modernization.
- Drive modernization of data-flow architecture: evaluate and introduce new techniques, patterns, and tooling; make build-vs-buy and technology choices; set direction others can implement against.
- Lead solution design for batch and distributed processing: data quality checks, indexing strategies, replay/idempotency patterns, and integration with existing Java/Spring services and APIs.
- Develop and tune Apache Spark workloads: implement batch jobs, diagnose failures, and improve performance through profiling, troubleshooting, and optimization.
- Build and maintain scripting and orchestration at scale using bash (required) and related automation; coordinate long-running jobs through enterprise scheduling and operational runbooks.
- Establish technical standards for batch and data-flow engineering—scripting patterns, monitoring, failure handling, and operational handoff—and influence practices beyond your immediate team.
- Collaborate with product managers, leadership, and engineering teams to align roadmaps with product needs and translate organizational goals into executable technical strategy.
- Troubleshoot and resolve complex production issues in data flows and batch systems; implement preventive, systemic improvements—not one-off fixes.
- Leverage and explore AI-assisted development tools (e.g., GitHub Copilot, Cursor, code generation, smart testing) where appropriate; help assess effectiveness and support adoption.
- Champion agile methodologies, lead technical and design reviews, and foster cross-team collaboration.
- Maintain awareness of security, data governance, and quality standards in an enterprise context.
Qualifications:
- Bachelor's degree in computer science or related discipline, or equivalent work experience.
- 7+ years of software development experience .
- Demonstrated professional strength in Java: shipping, maintaining, and evolving production services—not occasional or peripheral use.
- Strong bash scripting at scale in production data-flow and batch environments; comfort maintaining and extending large script-based systems.
- Hands-on experience with Apache Spark: developing batch workloads and troubleshooting/tuning performance and reliability (depth in patterns matters; specific cluster or vendor context can be learned on the job).
- Proven experience owning large-scale batch processing and data flows: design, operation, troubleshooting, and evolution—including publishing datasets or indexes at scale.
- Experience with batch orchestration and job scheduling in production (enterprise schedulers, dependency chains, failure recovery, and operational runbooks).
- Ability to work with messy, evolving data: inconsistent schemas, multiple sources, and changing requirements; design for robustness and incremental improvement.
Working Conditions: Normal office environment.
ADA/EAA: The above statements cover what are generally believed to be the principal and essential functions of this job. Specific circumstances may allow or require some people assigned to the job to perform a somewhat different combination of duties.
Skills Required
- Bachelor's degree in computer science or related discipline, or equivalent work experience
- 7+ years of software development experience
- Expert-level experience with large-scale data pipelines (ETL design and implementation, batch/streaming processing, publishing datasets/indexes)
- Experience with Big Data technologies and the Hadoop ecosystem
- Experience in cloud technologies (AWS, Snowflake)
- Linux/UNIX and Windows system administration experience
- Experience with modern deployment and CI/CD (e.g., GitLab, Ansible, containerization)
- CI/CD pipeline experience
- Experience owning critical pipelines or data systems end-to-end: design, run, troubleshoot, and evolve
- Excellent communication and documentation skills
