The ASE Data Services workflow effort builds automation that is safe, reliable, observable, and resumable. This work requires an innovative spirit and an extraordinary degree of care and rigor in engineering. These workflows operate on production databases at massive scale, where a mishandled rolling upgrade or host replacement has real customer impact. You will support the workflow engine layer underlying Apples most critical database systems which power all of Apples internet services. You will design and build workers and workflow definitions, composing operations across existing execution layers: host and pod provisioning, cluster topology discovery, and our monitoring and alerting systems. You will migrate operational logic off home-grown workflow tooling onto a durable execution model with retry, signal, query, and full execution history. This role requires excellent communication, the ability to partner closely with developers, SRE, and platform teams, and a high degree of customer focus when engaging with the internal stakeholders who will run these workflows every day. As a distributed team, the ability to work effectively with colleagues based in other locations is essential; experience in this area is a plus. Prior experience building or operating workflow/orchestration systems, or operating distributed databases and storage systems at scale, is recommended.
Minimum Qualifications
BS or MS in Computer Science / related fields or equivalent work experience
5+ years in a Site Reliability Engineering or infrastructure-focused engineering role
Proficient in Java; proficiency in Go (golang) or Python is also valuable
Hands-on experience designing or operating workflow / orchestration / job-scheduling systems, or building substantial automation for stateful, long-running, failure-prone operations
Support of internet-facing production services and distributed systems via deployments, on-call, and incident management
Experience running large-scale infrastructure with a heavy reliance on automation tooling
Real operational experience managing services at scale on Kubernetes
Excellent troubleshooting and performance deep-dive analysis
Self-motivated and inquisitive, with an aptitude to learn new technologies quickly and effectively
Preferred Qualifications
Direct experience with workflow systems (or comparable durable-execution engines such as Temporal or Cadence): writing workflows and activities, reasoning about retries, signals, queries, heartbeating, and execution history
Experience operating or building tooling for distributed databases / storage systems (Cassandra, Redis/Valkey, Kafka, Solr, ZooKeeper, etcd)
Understanding of database concepts: consistency models, isolation levels, crash and recovery semantics
Operational experience deploying in and running on datacenter and cloud architectures: networking topologies, host placement strategies, and failure modes; design of multi-datacenter systems; failure domains; and wide-area networking
Performance engineering (design concepts, profile-guided optimization)
Familiarity with mTLS / certificate-based service auth and Kubernetes-native worker deployment
Experience replacing legacy or home-grown orchestration with modern, observable platforms
Comfort working in a distributed team across multiple sites and time zones