DATA ENGINEER FULL TIME

Company: Alltake
Location: New York
Posted on: October 26, 2024

Job Description:

Job Details: Job Description: We are looking for a skilled Data Engineer with 6-8 years of experience in data management and processing to join our growing team. The ideal candidate will have experience in working with big data ecosystems and cloud-based systems and will be responsible for building and optimizing data pipelines, as well as data architecture across various platforms. Responsibilities: Design, develop, and maintain scalable data pipelines and architectures for complex data processing. Work with big data technologies such as Hadoop, Hive, Pig, and Spark to manage large datasets. Develop and maintain robust data processing solutions in cloud-based environments (AWS, Azure, GCP). Perform data extraction, transformation, and loading (ETL) processes using SQL and other data query languages. Collaborate with business owners and stakeholders to understand business needs and provide data-driven insights. Automate repetitive tasks and optimize data processing using Unix shell scripting or Python. Basic Qualifications: 3 years of relevant experience in data engineering roles. Proven hands-on experience with big data ecosystems, including Hadoop, Hive, Pig, and Spark. Experience with cloud-based data systems such as AWS, Azure, or GCP. Strong knowledge of SQL for data extraction and complex query design. Familiarity with data modeling, ETL pipeline development, and optimization. Preferred Qualifications: Proficiency in Unix shell scripting and/or Python for automation. Strong problem-solving skills with the ability to troubleshoot and optimize processes. Excellent communication skills, with the ability to work closely with business stakeholders to define key business questions and develop data sets to answer them. Self-motivated and adaptable, with strong attention to detail

Keywords: Alltake, Philadelphia , DATA ENGINEER FULL TIME, Engineering , New York, Pennsylvania