Temporal Technologies
Build invincible apps.
Staff Software Engineer, Visibility
Full-stack EngineerSoftware EngineerFull TimeRemoteTeam 51-200Since 2018H1B SponsorCompany SiteLinkedIn
Location
United States
Posted
38 days ago
Salary
$185K - $270K / year
5 yrs expEnglishAWSAzureCloudDistributed SystemsElastic SearchGoogle Cloud PlatformJavaGo
Job Description
• Re-architect Temporal Visibility at scale
• Lead the design and implementation of a new persistence layer for Temporal Visibility, informed by its real-world access patterns (high-volume writes, time-based queries, filtering, sorting, and pagination across long-running workflows).
• Evaluate and select the most appropriate storage technologies (e.g., ClickHouse, Elasticsearch, or complementary systems), clearly articulating tradeoffs around indexing models, consistency, cost, latency, and operational complexity.
• Design schemas, APIs, and query models that make Visibility both powerful and intuitive for customers.
• Plan and execute online migrations of live Visibility data from existing persistence stores to new backends, at scale and without customer downtime.
• Design dual-write, backfill, validation, and cutover strategies that prioritize correctness, observability, and rollback safety.
• Build tooling and automation to validate data integrity and performance throughout the migration lifecycle.
• Define and own SLOs for Visibility storage and query paths.
• Profile hot paths, design benchmarks, and lead systematic performance tuning efforts.
• Build operational playbooks, dashboards, and alerting that make the system understandable and debuggable for on-call engineers.
• Lead incident reviews and reliability improvements related to persistence and indexing systems.
• Break down large, ambiguous roadmap initiatives into concrete, executable phases.
• Author and steward design docs and RFCs through review with peers and stakeholders.
• Mentor and unblock other engineers working in the persistence and storage domain.
• Partner closely with Server, Cloud, and Developer Experience teams to land features end-to-end.
Job Requirements
- 5 or more years of experience as an "Arranger" and/or "Builder/Enhancer" of highly scalable distributed systems.
- Strong computer science fundamentals in distributed systems, including concurrency, consistency models, and failure modes
- Significant experience writing and operating concurrent production systems in Go, Java, or similar languages, at a high-end intermediate to expert level
- Experience writing concurrent code in production with languages like Go or Java or other applicable languages with skill level as "high end of Intermediate" and/or "Advanced" or "Expert" levels.
- Hands-on experience designing, operating, and tuning ClickHouse and/or Elasticsearch, ideally in self-hosted environments (managed services are a strong plus).
- Experience building and running services on AWS. Bonus: Azure and/or GCP experience.
- Demonstrated ability to lead large, multi-quarter technical initiatives, especially those involving core data infrastructure and live data migrations.
Benefits
- Unlimited PTO, 12 Holidays + 2 Floating Holidays
- 100% Premiums Coverage for Medical, Dental, and Vision
- AD&D, LT & ST Disability, and Life Insurance (Standard & Supplemental Available)
- Empower 401K Plan
- Additional Perks for Learning & Development, Lifestyle Spending, In-Home Office Setup, Professional Memberships, WFH Meals, Internet Stipend and more!
Related Guides
Related Job Pages
More Full-stack Engineer Jobs
Full-stack Engineer38 days ago
Full TimeRemoteTeam 51-200H1B Sponsor
Software Architect delivering technical architectures in a remote role for a digital consultancy
AzureCloudJavaScriptMicroservicesNext.jsNode.jsReactSitecore.NET
California + 3 moreAll locations: California, Colorado, Oregon, Washington
Full-stack Engineer38 days ago
Full TimeRemoteTeam 501-1,000Since 1999H1B Sponsor
Software Engineer role developing SaaS solutions in the remote Govtech industry.
ASP.NETJavaScriptSQL.NET
United States
Principal Full-Stack Software Engineer
AldeaContext is all you need. Intelligence that runs on understanding.
Full-stack Engineer38 days ago
Full TimeRemoteTeam 11-50Since 2023
Principal Full-Stack Engineer leading core development at Aldea Inc
AWSCloudDistributed SystemsDynamoDBGoogle Cloud PlatformJavaScriptNode.jsPostgresPythonReactRedisTypeScriptGo
United States
Full-stack Engineer38 days ago
ContractRemoteTeam 501-1,000H1B No Sponsor
Senior Full Stack Software Engineer developing publication management systems
AngularCloudJavaJavaScriptMicroservicesNode.jsNoSQLPythonReactVue.js
United States