Sr Data Pipeline Engineer

Description

Primary Responsibilities

• Design, build, test, and deploy a data pipeline using standard tools, such as Apache Airflow, Jenkins, and various AWS services.
• Coordinate with other code authors to implement their Python scripts directly, or their desired functionality indirectly, in the pipeline.
• Maintain and improve the data pipeline as new requirements arise.
• Serve as a technical resource for ABC and other teams on appropriate subjects, e.g. pipelines, tooling, and development at scale.
• Participate in, and sometimes lead, team ceremonies: standup, planning, retrospective, backlog refinement, stakeholder meetings, and demos.
• Attend and participate in larger company sync-ups and presentations to stay current with and contribute to company procedures and priorities.
• Create and manage feature demos, prototypes, and documentation.
• Establish strong relationships with internal teams and specific customer staff.
• Role-Based Competencies
• Continuous Improvement: Continually focused on improving the responsiveness and quality of the solutions delivered.
• Flexibility/Adaptability: Adjusts quickly to changing priorities and conditions. Copes effectively with complexity and change.
• Intelligence: Learns quickly. Demonstrates ability to quickly and proficiently understand and absorb new information.
• Teamwork: Reaches out to peers and cooperates with supervisors to establish an overall collaborative working relationship.

Required Qualifications

• Minimum five (5) years of progressive experience in production workflow management and automation, e.g., CI/CD, infrastructure as code, and version control.
• Ability to multi-task and work independently while maintaining team involvement, organized and detail oriented, strong problem solving and analytical skills.