Software Engineer

About Skan AI

At Skan AI, we are pioneering the context engine for human and agentic execution, bringing context from enterprise operators, systems, and processes to power how the world's largest organizations execute their most complex, mission-critical work.

Backed by Dell Technologies Capital and other leading investors, we're the only company that can bridge the gap between AI's promise and enterprise reality.

About the Role

The Context Graph team develops the systems that turn large volumes of observed enterprise work into structured process knowledge. This role is responsible for turning research systems into production software that can be trusted at scale.

As a Software Engineer on the Context Graph team, you will own the engineering boundary between research and production, making systems reliable under real workloads, real data volumes, real deployment constraints, and real failure conditions.

Responsibilities

Productionize research and agentic systems

• Take research prototypes, algorithmic harnesses, and model-backed pipeline components and turn them into reliable production services.
• Define clear interfaces, contracts, failure behavior, and operational boundaries around rapidly evolving research code.
• Build integration and regression suites that allow research teams to iterate quickly without destabilizing existing production behavior.
• Review and harden code produced across both human and agentic development workflows.
• Establish patterns for moving experimental capabilities into the existing product line without creating fragile parallel systems.

Build and scale distributed services

• Design, build, and operate production Python microservices.
• Build asynchronous, message-driven services on RabbitMQ or equivalent systems, including retry semantics, idempotency, backpressure, and failure recovery across multi-stage pipelines.
• Develop and maintain workflows on our in-house orchestration engine, Temporal, and Airflow.
• Design and evolve data layers across PostgreSQL or MySQL, MinIO object storage, Redis, and high-volume intermediate artifacts.
• Build and scale the service layer around LLM and agentic pipelines, including throughput, concurrency, caching, latency, and cost.

Own correctness and regression resistance

• Build rigorous unit, integration, system, replay, load, and regression testing for complex pipeline behavior.
• Develop performance benchmarks for throughput, latency, resource utilization, and queue behavior.
• Trace regressions across long data paths spanning ingestion, gateways, orchestration, services, model calls, storage, and output generation.
• Establish validation gates that prevent changes from reaching production when correctness, performance, or reliability has degraded.

Build for heterogeneous deployment environments

• Build and maintain CI/CD pipelines covering build, test, artifact publishing, validation, and deployment.
• Write and maintain Dockerfiles, container images, Kubernetes manifests, Helm charts, and versioning strategies.
• Manage infrastructure as code with Terraform across GCP, Azure, and supported enterprise environments.
• Own environment configuration, secrets management, releases, rollbacks, and deployment troubleshooting.

Operate what you build

• Instrument services with high-quality logs, metrics, and traces, and build the dashboards and alerts needed to understand production behavior.
• Support GPU-hosted and other resource-intensive workloads, including scheduling, resource allocation, and cost control.
• Participate in production support and on-call for the services your team owns.

Requirements

• We screen for comparable experience and strong fundamentals, not exact tool matches. Equivalent technologies are welcome.
• Production Backend Experience. 2-5 years building and operating production backend systems in distributed or service-oriented environments.
• Strong Python. You write clear, tested, production-quality Python and can quickly understand unfamiliar code.
• System Design Fundamentals. You are comfortable reasoning about horizontal and vertical scaling, autoscaling, load balancing, capacity planning, caching, statelessness, concurrency, and throughput versus latency tradeoffs.
• Distributed Systems Fundamentals. You understand asynchronous and event-driven systems, queue-based load leveling, idempotency, retries, timeouts, backpressure, and graceful degradation.
• Testing and Software Quality. You have strong instincts around unit, integration, system, replay, performance, and regression testing.
• Performance Engineering. You can reason about bottlenecks across CPU, memory, storage, networking, queues, database access, and model-backed services.
• Message Queues in Production. Experience with RabbitMQ, Kafka, SQS, Pub/Sub, or equivalent event-driven infrastructure.
• Data Layer Fundamentals. Strong understanding of relational schema design, query performance, transactions, connection management, caching, and object-storage-backed data flows.
• Containers and Orchestration. You understand how services are packaged, scheduled, scaled, isolated, given resource limits, and operated across different deployment environments.
• Deployment and Infrastructure. Experience with CI/CD and infrastructure as code.
• Operability. You can debug live systems using logs, metrics, traces, dashboards, and raw system behavior.
• Engineering Judgment. You are skeptical of code that merely works once.

Preferred

• Workflow orchestration engines such as Temporal or Airflow, or experience building similar systems.
• High-throughput APIs, gateways, rate limiting, connection pooling, and latency tuning.
• Object storage as a primary data plane, including S3, GCS, or MinIO.
• LLM application infrastructure including model gateways, prompt caching, token and cost management, and model observability.
• Observability systems such as Grafana, Loki, Tempo, Prometheus, or Mimir.
• Multi-cloud or hybrid-cloud deployments across GCP, Azure, or enterprise-controlled environments.
• GPU workload scheduling and cost optimization.
• Experience building performance, replay, or regression harnesses for complex distributed systems.

Equal Opportunity

Skan AI is an equal opportunity employer committed to building a diverse, inclusive, and respectful workplace around the world. We welcome people from all backgrounds and provide reasonable accommodations throughout the hiring process.