Overview
About the Role
We are seeking a highly skilled and motivated AI Infra Engineer to design, develop, and maintain our data platform specifically tailored for deep learning and computer vision applications. You will be responsible for building and optimizing our data infrastructure to support large-scale data collection, labeling, management, model training, evaluation, and continuous deployment. Your work will be critical in enabling our AI and computer vision teams to build and deploy state-of-the-art models efficiently and reliably.
About the Team
The AI and CV team at Caper (Instacart) innovates at the industry frontier across cloud and edge computing. The systems and algorithms built enable a magical shopping and checkout process in grocery stores. Our enthusiastic researchers and engineers are spread across different time zones but collaborate effectively on multiple exciting projects.
About the Job
Your responsibilities will include one or more of the following:
- Design, build, and maintain scalable and efficient data pipelines for collecting, processing, and storing large volumes of structured and unstructured data, specifically image and video streams and relevant metadata.
- Develop and integrate tools for data labeling and annotation, ensuring high-quality training datasets for deep learning and computer vision models.
- Collaborate with data scientists and machine learning engineers to build and optimize the infrastructure required for training and evaluating deep learning models at scale.
- Build and maintain CI/CD pipelines to seamlessly deploy machine learning models into production environments.
About You
Minimum Qualifications
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
- 5+ years of experience in data engineering, full-stack and/or infrastructure development.
- Proven experience with building and maintaining large-scale data pipelines (batching or streaming) for computer vision and/or machine learning applications.
- Familiarity with observability and monitoring tools (e.g. Datadog) and best practices.
- Familiarity with frameworks for large-scale data processing (e.g., Kafka, Spark, Airflow, Ray), storage (e.g., S3, Delta Lake), indexing and search (e.g. Elasticsearch).
- Experience with cloud platforms (e.g., AWS, GCP) and containerization technologies (e.g., Docker, Kubernetes).
- Strong problem-solving skills to work in a fast-paced, dynamic environment.
- Excellent communication skills to work collaboratively in a cross-functional team
Preferred Qualifications
- Experience building and/or integrating computer vision data collection, labeling and management systems.
- Experience in edge inference and optimization on Nvidia chipsets.
- Experience with deep learning frameworks (e.g., TensorFlow, PyTorch) and model management platforms (e.g., Kubeflow, MLflow, TensorBoard).
- Knowledge of computer vision and machine learning algorithms and models.
- Experience with frameworks and best practices for data security or compliance.