YonkersRecruiter Since 2001
the smart solution for Yonkers jobs

Lead Data Engineer

Company: Hale Recruiting
Location: Yonkers
Posted on: June 23, 2022

Job Description:

An opportunity to work for an exciting Publicly traded Media company as a Lead Data Engineer. This individual will have carte blanche in orchestrating a multi-tier cloud data pipeline coupled with a layered integration platform. This individual will lead a team of production support engineers and operations for different data management platforms. Aa a lead Data Engineer you will be responsible for the roll-out of continuous optimization of the respective data management platform.
You will partner with privacy teams, cloud operations and development to execute and ensure operations scalability.
An excited, fun, and fast-paced environment.

Qualifications---5+ years of expertise in DataOps; proven proficiency with CI/CD tools and concepts (GitLab, and ELT tools like dbt)---Experience in working with Big Data including Data Pipeline, Event tracking, Streaming Data, and Data Lake architectures---5+years of demonstrated ability in programming languages such as Python, Spark, Java, SQL, etc.; 5+ years of validated capabilities in database technologies such as BigQuery, MySQL, Redshift, Snowflake, Hive, etc---Strong understanding of cloud architecture, specifically AWS, as it relates to data processing, data modeling, and orchestration (i.e., EC2, S3, Airflow, etc.)---Technology management experience of onshore and offshore development team---Experience in implementing user access, security, and user activity logging across data platforms---Willing and able to work flexible hours and be ''on-call'' as needed---Bachelor's degree or equivalent in Computer Science, Information Systems, Engineering, or other related fields

Responsibilities---The Lead Data Engineer, Data Ops is accountable for orchestrating the client's multi-cloud big data pipeline and integration platform and will lead a team of production support engineers and operations for our big data management platforms---This role will be key to the rollout and continuous optimization of client's Data management platform and partner with Development, cloud operations, and privacy teams to run and scale operations---Develop a Data CI/CD framework setting up standard methodologies across the data management platform in AWS and Google Cloud Platform---Lead all aspects of the technical flow of data from various collection sources through the final delivery of product---Building Technology operations, standard operating procedures across change and release management, and data pipeline operations to incorporate industry-standard methodologies---Identify, design, and implement DataOps process improvements: automate manual processes, optimize data delivery, redesign infrastructure for greater scalability, etc---Partner with data consumers across Analytics, BI, and data technology to improve speed and quality of insight generation---Implement automated data quality and integrity checks to identify and resolve data issues within the pipeline;---Analyze complex data schemas and relationships to implement information models---Implement monitoring tools, measure metrics, and improve performance for client's data assets and analytics services in the cloud---Plan and transition new products from Product Development into the Data Operations team---Represent data technology department considerations to broader the client's cloud operations discussions and ensure alignment with overall enterprise cloud strategy---Direct analytics and data consumption environments using serverless computing (e.g Athena, or similar)

Keywords: Hale Recruiting, Yonkers , Lead Data Engineer, Engineering , Yonkers, New York

Click here to apply!

Didn't find what you're looking for? Search again!

I'm looking for
in category

Log In or Create An Account

Get the latest New York jobs by following @recnetNY on Twitter!

Yonkers RSS job feeds