Data Pipeline Engineer (Remote in USA + Europe)

Posted 1 week, 4 days ago, 10 views
  • python
  • amazon-web-services
  • sql
  • etl
  • kafka-consumer-api

The programmatic team at TMP is seeking a Data Pipeline Engineer to support building new data products and services.
The Team


  • TMP Programmatic works on data services across the programmatic platform within TMP, and supports building customer-facing data visualization products


  • The team has extensive experience in ETL development and works with large scale data in real time
    About the Job


  • Providing technical support for all data pipeline environments

  • Set up, configure, maintain and enhance proper infrastructure to support a large scale data analytics environment

  • Evaluate the technical tradeoffs of every decision

  • Build and maintain ETL pipelines utilizing Python

  • Work with Cloud Computing Platforms (AWS), Kafka and other open-source technologies

  • Conduct data modeling, schema design, and SQL development

  • Ingest and aggregate data from both internal and external data sources

  • Collaborate with Product Owner and domain experts to recognize and help adopt best practices in reporting and analysis: data integrity, test design, analysis, validation, and documentation

  • Assist with the development and review of technical and end user documentation including ETL workflows, research, and data analysis

  • Work with Product team to define data collection and engineering frameworks

  • Responsible for daily integrity checks, performing deployments and releases


  • Own meaningful parts of our service, have an impact, grow with the company
    Desired Technical Qualifications


  • Excellent and proven knowledge of Python

  • Excellent and proven knowledge of SQL

  • Excellent and proven knowledge of streaming technologies, such as Kafka or Kinesis

  • Excellent and proven knowledge of queue-based infrastructures such as AWS SQS, SNS, etc.


  • Good knowledge of the following technologies is a bonus:

    • Postgres on Amazon RDS

    • Amazon Redshift

    • Docker




  • Proficiency in Git, JIRA, and Teamcity are a plus

  • 2-4+ years in a production environment a plus
    Flexible Location (Remote Hubs):


Berlin, Germany | Lisbon, Portugal | London, England | Paris, France | USA

😻
Please mention Remote Club as a website where you found this job. It helps us get more interesting job posts here.