Woongjin, Inc.

Backend Engineer _ AI Gateway- Bilingual (Korean/ English)

Posted 2 Days Ago

Be an Early Applicant

Plano, TX, USA

In-Office

115K-128K Annually

Mid level

Information Technology • Professional Services

The Role

Build and operate a high-performance AI Gateway that securely routes, inspects, and governs enterprise traffic to LLM providers. Develop asynchronous Python and FastAPI services, streaming APIs, network protocols, cloud-native infrastructure, messaging systems, security integrations, observability, and CI/CD pipelines. The role involves Kubernetes deployments, scalable concurrent connections, identity and access management, AI/LLM workflows, cost optimization, and incident response. Korean-English bilingual ability is highly preferred.

Summary Generated by Built In

Company Description

For More Open Positions Visit us at:
http://recruiting.woongjininc.com/

Our Mission

WOONGJIN INC. is a rapidly growing team who provides a range of unique, exceptional, and enhanced services to our clients. We have a strong moral code that includes the service of goodness without expectations of reward. We are motivated by the sense of responsibility and servant leadership.

Job Description

We are building an AI Gateway — a high-performance, secure intermediary layer that routes, inspects, and governs all traffic between enterprise clients and LLM/AI service providers. This role requires deep expertise in network programming, async Python, and cloud infrastructure to design and operate a latency-sensitive, policy-driven gateway that handles massive concurrent streaming connections at scale.

What we are looking for

Async Python & Network Programming (3 + yrs) – Strong hands‑on experience with asyncio, aiohttp/anyio, low‑level TCP/UDP sockets, WebSocket & SSE servers, TLS handshake, connection‑pooling, keep‑alive, and zero‑copy buffering.
Streaming & Large‑Payload Handling – Ability to manage chunked transfer, bidirectional streaming, back‑pressure and high‑throughput data paths for real‑time LLM inference.
Protocol Knowledge – Working familiarity with HTTP/2 & HTTP/3 (multiplexing, HPACK), gRPC, and occasional custom protocol parsers.
FastAPI‑Based Backend (3 + yrs) – Design, develop and test production‑grade REST/Streaming APIs; pragmatic use of Pydantic, dependency injection and OpenAPI docs.
Containerization & Kubernetes – Docker, AKS/EKS/GKE deployment experience; HPA/VPA scaling, rolling updates, and service‑mesh (e.g., Istio/Envoy) basics.
Async Messaging & Event‑Driven Design – Production use of Redis Streams, RabbitMQ or Kafka for decoupled task queues, retries and back‑off logic.
PostgreSQL & Vault – Advanced query tuning, partitioning, PgBouncer pooling, plus HashiCorp Vault (dynamic secrets, transit encryption) for key management.
Cloud Networking & Security – L4/L7 load balancing, TLS termination, reverse‑proxy (NGINX/Envoy), mTLS between services, and automated cert lifecycle (ACME/Let’s Encrypt).
IaC & CI/CD – Terraform modules for multi‑env infra, plus GitHub Actions/GitLab CI pipelines with container image scanning and canary/blue‑green deployments.

Qualifications

Enterprise Security Gateways – Integration with CASB/DLP/WSS or similar edge‑security platforms; centralized logging & audit trails.
IAM & SSO – Configuring Azure AD/Entra ID, Okta, SAML 2.0 or OIDC SSO flows; managing Conditional Access & SCIM provisioning.
AI/LLM Pipelines – Building prompt‑engineering workflows, PII anonymisation, and monitoring Azure OpenAI (or comparable) usage, rate limits & cost.
API‑Gateway Operations – Custom webhook/REST integrations, circuit‑breaker patterns, rate‑limiting, auto‑retry and health‑checking.
Observability & FinOps – End‑to‑end monitoring with Datadog, Prometheus/Grafana or Azure Monitor; proactive cost‑optimisation and 24/7 incident response (SLA/SLO).
Bilingual Korean Highly Prefer

Additional Information

All your information will be kept confidential according to EEO guidelines.

*****NO C2C******

Skills Required

At least 3 years of hands-on experience with asynchronous Python and network programming, including asyncio, HTTP clients, sockets, WebSocket, SSE, TLS, connection pooling, keep-alive, and buffering.
Experience handling streaming and large payloads, including chunked transfer, bidirectional streaming, back-pressure, and high-throughput data paths.
Familiarity with HTTP/2, HTTP/3, multiplexing, HPACK, gRPC, and custom protocol parsers.
At least 3 years of experience designing, developing, and testing production FastAPI REST and streaming APIs.
Experience with Docker, Kubernetes, AKS, EKS, or GKE deployments, including HPA/VPA, rolling updates, and service mesh fundamentals.
Production experience with Redis Streams, RabbitMQ, or Kafka for asynchronous messaging and event-driven systems.
Advanced PostgreSQL experience, including query tuning, partitioning, and PgBouncer pooling.
Experience with HashiCorp Vault, dynamic secrets, transit encryption, cloud networking, load balancing, TLS termination, reverse proxies, mTLS, and automated certificate lifecycle.
Experience with Terraform modules and GitHub Actions or GitLab CI pipelines, including image scanning and canary or blue-green deployments.
Experience integrating enterprise security gateways such as CASB, DLP, or WSS, including centralized logging and audit trails.
Experience configuring Azure AD/Entra ID, Okta, SAML 2.0, or OIDC SSO, including Conditional Access and SCIM provisioning.
Experience building AI/LLM workflows, PII anonymization, and monitoring Azure OpenAI or comparable services.
Experience with API gateway integrations, circuit breakers, rate limiting, retries, and health checks.
Experience with Datadog, Prometheus/Grafana, or Azure Monitor, plus cost optimization and 24/7 incident response.
Bilingual Korean and English proficiency.