Data Engineer  

The Data Engineer will be responsible for designing, building, and maintaining robust data infrastructure and pipelines that support current and future business needs. This includes transforming raw data into actionable insights through scalable and secure pipelines, managing data assets effectively, and enabling seamless access to data for analytical and operational processes.

Duties and Responsibilities:

• Design, build, and maintain scalable, secure data pipelines and storage systems; ensure data quality through ETL processes and regular checks.
• Implement policies and practices to control, optimize, and secure data assets, ensuring data integrity and accessibility.
• Develop and maintain data models, structures, and databases to meet business needs; communicate data architecture effectively.
• Develop, test, and maintain scripts and programs to automate data processing and pipelines, adhering to industry standards.
• Create and operationalize data visualization solutions to simplify complex data for stakeholders and decision-making.
• Work with cross-functional teams to gather data requirements, optimize existing processes, and deliver ad hoc reports and insights

• Bachelor’s or Master’s degree in IT, Computer Science, Engineering, or any related field
• At least 3 years of experience in data engineering, data analytics, or related fields
• Proficient in Python and Java; other programming languages are a plus
• Hands-on experience with data manipulation tools (e.g., pandas, dplyr, Spark)
• Skilled in ETL (Extract, Transform, Load) processes for efficient data pipeline management
• Proficient in SQL and NoSQL for database interaction and management
• Experience with big data technologies (e.g., MongoDB, Spark, Hive, Snowflake, Redshift)
• Experience with cloud or server-based data processing (e.g., AWS, Azure, GCP)
• Comprehensive understanding of data manipulation tools (pandas, dplyr, Spark)
• In-depth knowledge of big data frameworks (Apache Spark, Hadoop)
• Familiarity with data warehousing services (AWS Redshift, Snowflake, etc.)
• Proficient in AWS Cloud Services (especially AWS Glue and AWS Lake Formation)
• Familiar with BI tools (Tableau, Power BI, QuickSight, Google Data Studio)
• Knowledge of data visualization libraries (Dash, Plotly, Matplotlib, ggplot, Folium)
• Understanding of machine learning libraries/tools (e.g., scikit-learn, caret, MATLAB) is a plus
• Knowledgeable in data governance, quality control, and security best practices