Big Data Engineer Retraining Program

Essential functions

The program/project includes (but not limited):

• Uploader (Go written binary client with its backend API to transfer data from third-party into platform)

• Scheme registry (It is a metadata store with dataset schemas including schema management capabilities)

• Pipeline manager (Manage ETL pipelines: CRUD and execute. Triggers ETL pipeline on upload event Pipelines metadata storage)

• (Airflow DAG) data ingestion pipeline including data decryption, validation (optionally) and ingestion.

• Monitor data metrics. uploads and integrity. At least data size, num objects, uploader entity, path.

• Hone the soft skills necessary for effective collaboration with customer team members and stakeholders, including engineering managers.

Qualifications

• Practical experience as a Data Engineer or Backend Engineer

• Proficiency in Scala and Java

• Experience with Apache Spark for large-scale data processing

• Strong knowledge of SQL for querying and data manipulation

• Basic understanding of AWS or any other cloud service platform

• Strong focus on data processing, ETL pipelines, and database management

Would be a plus

• Familiarity with data modeling, distributed computing, and data integration techniques is also valuable.

• Python and PySpark

• Airflow

• Snowflake

• Knowledge of Apache Iceberg, Flink, Druid

• Kafka

• Data Lakes, Data Warehouse

We offer

• Opportunity to work on bleeding-edge projects

• Flexible schedule

• Benefits package - medical insurance, sports

• Corporate social events

• Professional development opportunities

• Well-equipped office

About us

Grid Dynamics (NASDAQ: GDYN) is a leading provider of technology consulting, platform and product engineering, AI, and advanced analytics services. Fusing technical vision with business acumen, we solve the most pressing technical challenges and enable positive business outcomes for enterprise companies undergoing business transformation. A key differentiator for Grid Dynamics is our 8 years of experience and leadership in enterprise AI, supported by profound expertise and ongoing investment in data, analytics, cloud & DevOps, application modernization and customer experience. Founded in 2006, Grid Dynamics is headquartered in Silicon Valley with offices across the Americas, Europe, and India.