Costruiamo il Successo in Modo Efficiente!

Architect, Cloudera Spark – Big Data Developer, Spark

Software EngineerSoftware EngineerFull TimeRemoteTeam 51-200Since 2023Company Site LinkedIn

Location

United States

Posted

109 days ago

Salary

Not specified

Bachelor Degree3 yrs expItalianApacheDockerETLHbaseHDFSJenkinsKafkaKubernetesLinuxPythonScalaShell ScriptingSparkSQLUnixYarn

Job Description

• Operate on high-complexity data engineering and data analytics projects • Ensure scalability and security of data architectures in enterprise environments • Develop and optimize distributed data pipelines in enterprise contexts

Job Requirements

Big Data Architect with deep experience in distributed environments and Cloudera technologies
Proven experience with Big Data architectures based on Cloudera Data Platform (CDP) and Apache Spark
Strong knowledge of HDFS, Hive, Impala, HBase, Kafka and NiFi
Proficiency with YARN, Ranger, Knox, Atlas and data security and governance tools
Experience in data modeling and design of ETL/ELT pipelines
Knowledge of Scala, Python and SQL
Good understanding of microservices, containerization (Docker, Kubernetes) and REST APIs
Familiarity with Linux/Unix environments and advanced scripting
Experience with monitoring tools and performance tuning for Spark and Cloudera
Experience in Public Administration or regulated environments is a plus
Big Data Developer with solid experience in Cloudera and Apache Spark environments
At least 3 years' experience developing applications on Apache Spark (Core, SQL, Streaming)
Deep knowledge of the Cloudera ecosystem (HDFS, Hive, Impala, Oozie, NiFi)
Strong proficiency in Scala and Python
Experience managing and optimizing Spark jobs in clustered environments
Knowledge of Kafka for real-time ingestion
Familiarity with Git, Jenkins, CI/CD and DevOps best practices
Experience in query tuning, data ingestion pipelines and data transformation
Basic knowledge of Linux, shell scripting and distributed systems
Attention to detail and ability to work in structured environments
Good communication skills and a team-oriented attitude
Commitment to continuous improvement and adoption of quality standards