Our team’s mission is to build the world’s most useful agent, and we’re looking for a Data Engineer to build the pipelines and tools for collecting and analyzing a wide range of human data. You’ll work alongside world class AI researchers, engineers, and program managers to identify and implement the best processes for human data collection. This role is highly cross-functional, leveraging skills across data science, machine learning engineering, and project management to ensure our team collects the most effective agentic training data in a rapidly-evolving technological environment.
Key job responsibilities
* Work closely with researchers engineers to create robust data pipelines and data collection tools.
* Work closely with program managers to optimize data collection processes.
* Simplify and enhance the accessibility, clarity, and usability of large or complex datasets through the development of advanced dashboards and applications.
* Take ownership of the design, creation, and upkeep of metrics, reports, analyses, and dashboards to inform data collection projects.
* Develop and manage scalable, automated, and fault-tolerant data solutions using cutting-edge technologies such as Spark, EMR, Python, Redshift, Glue, and S3.
* Continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for datasets.
Key job responsibilities
* Work closely with researchers engineers to create robust data pipelines and data collection tools.
* Work closely with program managers to optimize data collection processes.
* Simplify and enhance the accessibility, clarity, and usability of large or complex datasets through the development of advanced dashboards and applications.
* Take ownership of the design, creation, and upkeep of metrics, reports, analyses, and dashboards to inform data collection projects.
* Develop and manage scalable, automated, and fault-tolerant data solutions using cutting-edge technologies such as Spark, EMR, Python, Redshift, Glue, and S3.
* Continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for datasets.