Software Engineer, Big Data
Role and Responsibilities
You will design, architect and implement big data pipelines to enable customer facing product capabilities and internal machine learning data platform. You will
- Design, architect and implement data pipelines streaming large amounts of video and metadata.
- Collaborate with data scientists in commercializing novel real-time analytics and product capabilities.
- Keep up with state of the art on best ways of architecting scalable big-data solutions.
- Architect the big data platform to enable data mining and rapid iterations of machine learning models.
- B. Tech, M. Tech or PhD in computer science, electrical engineering or a related area.
- Excellent programming skills – Python (required), Java, Scala.
- Excellent knowledge of data structures, algorithms and distributed systems.
- Experience in building large scale data platforms.
- Understanding of system performance and scaling.
- Strong knowledge of databases Postgres, MongoDB etc.
- Familiarity with developing and deploying solutions in AWS.
- Experience with developing micro-services and rest APIs.
- Ability to develop quick proof of concept solutions in a fast-paced startup environment.
- Flexible and can-do approach. Ability to adjust to changing priorities.
- Excellent written and verbal communication skills.
- Experience with Kafka/Kinesis, SQS, Lambda etc. is a plus.
- Experience in data visualization and storytelling dashboards.
- Familiarity with front-end and web-development concepts is a plus.
- Familiarity with ML, Statistics and Probability concepts is a plus.