Turn Change Into Value®

Senior Data Engineer

Data EngineerData EngineerFull TimeRemoteTeam 51-200Since 2002H1B SponsorCompany Site LinkedIn

Location

California + 3 more

Posted

173 days ago

Salary

Not specified

Bachelor Degree7 yrs expEnglishAmazon RedshiftAnsibleAWSCloudDynamo DBEC2HadoopHDFSJavaPy SparkPythonRDBMSScalaSparkSQLTerraformUnix

Job Description

• Design, develop, and operationalize large-scale enterprise data solutions using AWS services such as S3, Glue, Athena, Lambda, RedShift, EMR, Spark, and DynamoDB • Analyze, redesign, and re-platform on-premise data solutions from Cloudera Hadoop platforms to AWS native data stack • Design, develop, and deploy data pipelines from ingestion to consumption within a big data architecture using Java, Python, or Scala • Participate in discussions related to architecture, design, and product development to understand business requirements and translate them into technical solutions • Take responsibility for complete user stories through analysis, design, development, and testing as per project timelines • Develop and maintain scripts to automate batch jobs • Support and maintain existing applications, troubleshooting and resolving technical issues • Follow appropriate technical best practices and internal processes while complying with various information security controls • Maintain existing data solutions running on Hadoop stack using HDFS, Oozie, Impala, and Hive, handling some enhancements until the cloud migration.

Job Requirements

Bachelor’s Degree in Computer Science, Information Technology, or other relevant fields
Must be a US Citizen or a Green Card holder for over 3 years with the intent to become a US Citizen
7+ years of software development experience related to data engineering
Hands-on experience in Java, Python, or Scala with the ability to understand/write complex SQL queries
Proficient in AWS services such as S3, Glue, Athena, Lambda, RedShift, EC2, EMR, Spark, and DynamoDB using Java, Python, Scala, or PySpark
Experience deploying software solutions to the cloud platform through CI/CD in a DevOps model
General understanding of application and data security concepts with exposure to AWS IAM, CloudTrail, CloudWatch, AWS Config, Secrets Manager, and KMS
Hands-on experience with Hadoop technologies such as HDFS, Oozie, Impala, and Hive
Experience with Ansible, Terraform, or Cloud Formation scripts to develop or support Infrastructure as Code
Experience working with Hadoop-based Big Data architecture and solutions
Experience working in an Agile development environment using Agile tools like Jira or Rally
Proficiency with UNIX commands and shell scripts
Ability to effectively communicate, collaborate, and work in a team environment to deliver high-quality work independently
Background check which includes fingerprinting, drug testing, and a personal interview (applicant consent required)
Nice to have: experience with RDBMS and data warehouses
Nice to have: experience with Machine Learning technologies and data visualization tools

Benefits

Medical, dental, 401(k) match, flexible spending and more
Up to 27 days off a year (including your birthday!)
Remote work opportunities
Parental leave
Wellness benefits
Professional development, including our Communities of Practice, technology partnerships, sandbox and paying for certifications and trainings to improve their skills
Inclusive and diverse culture as a woman-owned organization