Data Engineer

  • Engineering
  • Remote job

Data Engineer

Job description

About vidIQ

vidIQ helps YouTube creators and brands generate more views and subscribers, while saving time. With over 1 Million active weekly users, we are the #1 Chrome Extension for YouTube creators, with clients including Red Bull, Buzzfeed, PBS, TMZ, BBC as well as hundreds of thousands of the largest YouTube creators in the world. We’re backed by top Silicon Valley investors including Scott Banister and Mark Cuban. vidIQ is profitable with a fully remote team over 25 employees and growing.


Role & Responsibilities

vidIQ is seeking a highly-motivated Data Engineer with 3+ years of hands-on data engineering experience to join our growing team. The ideal candidate will be a go-getter with the ability to work independently. In this role, you will have oversight of partitioning data, building an ETL pipeline, data compaction, and AWS optimization. 


You must be highly collaborative and a self-starter who is able to work in a fast-paced environment. Strong communication skills are essential in this role, as it will be integral in communicating to the back-end team where and how to implement data integration and persistence. You will also communicate to management the volumes of data we are gathering, as well as communicate the data access points and how to use this data, to the team and management. 

Job requirements

Minimum experience:

  • 3+ years experience using Python for internal data pipelines (moving data inside AWS account)
    numpy, pandas
  • Experience with Scala - for external data pipelines (moving data from outside of AWS account into AWS account) FS2, http4s
  • Additional experience with DynamoDB, Lambda, Athena, S3, AWS GlueFamiliar with Spark (in the moment Scala only) preferred
  • Hands-on experience with data workflow orchestration (Airflow)