DataPelago

Accelerate Any Engine on Any Data on Any Hardware

Data Processing Software Developer

Full-stack EngineerSoftware EngineerFull TimeRemoteTeam 51-200Since 2022H1B SponsorCompany SiteLinkedIn

Location

United States

Posted

142 days ago

Salary

Not specified

Bachelor Degree5 yrs expExperience acceptedEnglishLinuxRustSQL

Job Description

• Implement software to perform data processing operations efficiently on large data sets as part of an overall data processing engine. • Work closely with technical leads and other individual developers to develop this software and ensure data is processed accurately, efficiently, and reliably as part of the overall analytics engine. • Investigate and address issues in the software and develop necessary enhancements in an ongoing manner. • Working with cross-functional teams to deliver new algorithms and techniques in the core Datapelago GPU-powered, parallel and distributed execution engine • Developing enterprise-grade, highly reliable code and owning the end-to-end delivery of solutions ensuring smooth production rollouts • Designing, implementing, testing, and maintaining enhancements in different components of our engine. • Developing highly modular components and ensuring clean integration with the entire stack • Define algorithms, mechanisms, procedures, and policies for the required functionality. Document them, and secure review and approval from the technical lead(s) • Continuously analyzing and identifying areas for differentiation and improvements in the execution engine • Identifying bottlenecks in scaling and performance and implementing solutions • Investigate and resolve all bugs before and after the release of the software • Support troubleshooting and resolution of issues, incidents, and problems, working in collaboration with others as needed. • Document findings, resolutions, and identify required enhancements to avoid recurrence • Collaborating with other teams and team members and assisting with code reviews, design reviews, and development process improvements

Job Requirements

  • S. EE/CS or equivalent with 5+ years of experience or MS with 3+ years’ experience
  • 5+ years of experience developing database execution engine components including core database operations - joins, aggregations, sorts, analytic functions like window functions, etc., in an enterprise-class database serving large-scale data processing workloads
  • Experience developing modern analytic databases and familiarity with vectorized execution, parallel and distributed database algorithms, memory management, etc.
  • Good understanding of SQL and familiarity with Lakehouse architectures and technologies
  • Solid experience in developing performant code that minimizes overheads and latency and maximizes throughput
  • Experienced with the best practices in developing high quality, reliable software
  • Experience developing, evaluating, and troubleshooting high-performance software
  • Strong programming ability in C, C++. Rust experiences a strong plus
  • Strong development experience on Linux platforms

Related Job Pages

More Full-stack Engineer Jobs

Cloud Software Engineer – Observability Platform

ClickHouse

ClickHouse is an open-source, column-oriented OLAP database management system.

Full-stack Engineer142 days ago
Full TimeRemoteTeam 51-200Since 2016H1B Sponsor

Cloud Software Engineer designing observability systems for ClickHouse.

AWSAzureCloudDistributed SystemsGoogle Cloud PlatformGrafanaKubernetesPrometheusTerraformGo
United States
$115K - $185K / year

Senior Software Engineer

Magpie Literacy

Magpie Literacy is a research and design organization focused on unlocking the great reader in every child.

Full-stack Engineer143 days ago
Full TimeRemoteTeam 11-50H1B No Sponsor

Full Stack Senior Software Engineer developing systems for educational nonprofit

United States
$145K - $160K / year

Senior Software Engineer

Whisker

Home of the Litter-Robot, Feeder-Robot, and Litterbox.com

Full-stack Engineer143 days ago
Full TimeRemoteTeam 501-1,000H1B Sponsor

Senior Software Engineer extracting insights from electrical networks for fire prevention

ASP.NETAWSAzureCloudIoTPythonSQLTCP/IP.NET
United States

Software Developer, Avid/Adobe Premiere Plugins, Panel, REST API

DigitalFish

DigitalFish Delivers the Art and Science of Digital Storytelling

Full-stack Engineer143 days ago
ContractRemoteTeam 11-50H1B Sponsor

Software Developer creating Avid/Adobe Premiere plugins and APIs

JavaScriptMacOS
California
$50 - $80 / hour