AgileEngine is looking for a Senior Backend/Data Engineer to develop back-end/data engineering solutions. The ideal candidate’s prior work experience reflects high standards for engineering with proven capabilities. The candidate possesses the ability to take on important business challenges and execute them with precision and quality.
What is required?
- 4+ years experience in software development
- BS in Computer Science or equivalent practical experience
- Strong SQL knowledge
- Knowledge of SQL Server or other relational database management systems (Oracle, PostgreSQL, MySQL)
- Practical experience implementing Apache Spark jobs: knowing its features on partitioning, grouping, joins, importing the data into the cluster, exporting the data from the cluster
- Practical experience building ETL/ELT data pipelines from scratch or using existing frameworks like StreamSets DataCollector (SDC), Fivetran, Apache NiFi
- Practical experience of using Data Warehouse(s) with Snowflake or other DW-oriented databases (Google BigQuery, Redshift, etc.)
- Practical experience with one of the following languages: C#, Python, Java, Scala
- Intermediate English
What will be a plus?
- AWS basics
- AWS DMS
- Kubernetes basics
- Spark Operator
What will you do?
- Build Apache Spark jobs
- Build data pipelines(ETL flows) to connect different sources of data together
- Identification of performance bottlenecks
- Serve as a go-to Data Engineering consultant for other team members (Bloomberg Tax specifically requires that, since they are mostly SQL/RDBMS/.NET shop)
- Optimize SQL queries
- Producing high-quality code
- Communication with Engineering & Product management
- Alignment with existing development teams