Requirements
Bachelor’s degree in Computer Science, Information Systems, or other related technical field or equivalent work.Proven experience as a Data engineer or similar role, with at least 4 years of experience.Advanced SQL knowledge/experience - complex queries and query optimization.Proficiency in programming languages such as Python.Experienced in the development of data warehouses and proficient in data modeling.Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform.Experience in building data streams using technologies such as Kafka, Flink, or Spark Streaming is considered a plus.Strong problem-solving skills and attention to detail.The ability and willingness to learn new technologies on the job.Self driven, highly motivated, fast learner, ambitious and creative.
Description
Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and design appropriate solutions.Work closely with IT and infrastructure teams to deploy and maintain data storage and processing systems, including databases, data warehouses, and cloud infrastructure.Design, build, and maintain robust, scalable data pipelines to ingest, process, and transform data from various sources, including internal databases and external APIs.Optimize data pipelines for performance, scalability, and reliability, ensuring timely and accurate delivery of data.Implement data validation and quality checks to identify and rectify any issues in data pipelines, ensuring data integrity.Explore and evaluate new technologies, tools, and frameworks to improve data engineering processes and capabilities.Perform data modeling and schema design to support analytical and reporting requirements.Design and build data marts and business layers to facilitate efficient data access and analysis.