AgileEngine is looking for a Senior Big Data Engineer to contribute to the Data Engineering team Marketplace and inventory listings to make car shopping an uplifting experience. The project is to work identifying data to capture Data normalization and adding new features to the existing platform and monitoring features to the pipeline. The ideal candidate’s prior work experience reflects high standards for engineering with proven capabilities.
What is required?
- 5+ years of experience programming in Java.
- 2+ years of experience in Big Data technologies.
- Experience in any of big data technologies: MapReduce, Spark, HBase.
- Proficient in SQL and experience with RDBMS/NoSQL databases.
- Experience working with Cloudera/Hortonworks/EMR distribution in AWS.
- Ability to self-manage tasks and be proactive in working with other teams to accomplish them while taking pride and ownership in their work.
- Team-player with strong collaboration and communication skills, who is able to respond positively to feedback.
- Bachelor degree (or Master) in Computer Science or related engineering field
What will you do?
- Design and develop efficient and scalable data processing pipelines using big data technologies (Hadoop, Spark, HBase, Kinesis, MapReduce, etc.) on large scale structured/unstructured data sets for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL/NoSQL
- Build complex workflows and orchestrate data dependencies
- Monitor and support data pipelines to honor internal and external SLA’s
- Work within standard engineering practices (i.e. SCRUM, unit/integration testing, design review, code reviews, continuous integration, etc.) to deliver product features with optimal efficiency for customers and clients
- Closely work with product owners & analysts to understand business and functional requirements and contribute to the design and prioritization discussions
- Working with a team of engineers where mentorship is valued
- Ability to learn and adapt to continually evolving technologies in the big data ecosystem
About the project?
Our client is a leading automotive digital marketplace that enables car buyers to connect to our nationwide network of Certified Dealers. They are building the industry’s most personalized and efficient car buying experience as we seek to bring more of the purchasing process online. Consumers who visit our marketplace will find a suite of vehicle discovery tools, price ratings, and market context on new and used cars – all with a clear view of what’s a great deal. When they are ready, our client will enable them to connect with a local Certified Dealer who shares in our belief that truth, transparency, and fairness are the foundation of a great car buying experience. As part of our marketplace, our client powers car-buying programs for over 250 leading brands, including AARP, Sam’s Club, and American Express. Nearly half of all new-car buyers engage with our client powered sites, where they buy smarter and drive happier.