LanceDB
Developer-friendly, open source database for multi-modal AI
Open Source Engineer
Location
United States
Posted
140 days ago
Salary
$180K - $235K / year
Bachelor Degree5 yrs expEnglishHadoopJavaPy TorchRayRustScalaSpark
Job Description
• Driving open-source community efforts to integrate the Lance format with Spark, Hive Metastore, Presto, Trino, Ray, and other data infrastructure systems
• Designing and maintaining efficient distributed Lance dataset operations
• Building efficient indices to enable predicate pushdown and accelerate queries in Spark, Ray, or Trino
• Working on table formats, data encodings, and various aspects of the Lance format in Rust
• Operating and improving internal data processing infrastructure
• Promoting the Lance format in open-source communities and at Big Data conferences
Job Requirements
- 5+ years of experience building high-performance databases, big data systems, or large-scale data services
- Deep understanding of internals of open-source Big Data or AI training systems (e.g., Hadoop, Spark, Flink, Ray, Iceberg, Delta Lake, Hudi, ClickHouse, Trino, Presto, PyTorch, or JAX)
- Strong experience with high-performance computing in Java or Scala
- Experience with Rust (or willingness to learn it)
- Proven ability to move fast, work independently, and collaborate with a high-caliber team
Benefits
- Medical, dental, vision, and life insurance
- 401(k) retirement plan
- Flexible Spending Accounts (FSA) and Health Savings Accounts (HSA)
- Commuter benefits
- Generous paid time off
Related Guides
Related Categories
Related Job Pages
More Engineer Jobs
Engineer141 days ago
Full TimeRemoteTeam 1,001-5,000Since 2001H1B Sponsor
Network Engineer II managing various network installations and troubleshooting
DNSFirewallsSwitching
Fire Protection Engineer II
Harrison AssociatesLeading the Field in Case Management and Expert Witness Services
Engineer141 days ago
Full TimeRemoteTeam 11-50Since 1985H1B No Sponsor
Fire Protection Engineer II designing systems to protect lives at HFA
United States
Engineer141 days ago
Full TimeRemoteTeam 1,001-5,000H1B Sponsor
Staff Observability Engineer driving Observability vision at Alteryx
AnsibleAWSAzureCloudGoogle Cloud PlatformGrafanaJavaKubernetesPrometheusPythonSDLCSplunkTerraform.NET
Engineer142 days ago
Full TimeRemoteTeam 11-50Since 2023H1B No Sponsor
Transpiler Engineer building systems that convert source code into human language.
Python
United States