As a Senior Data Engineer at CodeValue, you'll play a crucial role in our data team, contributing to the development and maintenance of scalable data pipelines and systems. You'll be at the forefront of designing, implementing, and optimizing data solutions that drive our business objectives and empower data-driven decision-making processes. Collaboration with data analysts, data scientists, and other stakeholders will be essential to ensure data integrity, reliability, and accessibility.
Responsibilities:
Design, develop, and optimize data pipelines and architectures using a variety of technologies and tools including Python, SQL, Spark, Kafka, and Airflow.
Develop efficient data models to facilitate storage, retrieval, and processing, with a focus on schema design for NoSQL databases, data lakes, and data warehouses.
Ensure scalability of big data architecture to handle large data volumes and high traffic loads.
Implement data quality checks, validation processes, and data governance measures to ensure accuracy and consistency of data.
Monitor, troubleshoot, and debug data issues and performance bottlenecks.
Implement security measures such as encryption, access controls, and data masking to protect sensitive data.
Research and evaluate new data technologies and best practices to enhance data engineering processes and solutions.
