Senior Python Engineer for Data Retraining

Responsibilities

• Develop a strong foundation in Big Data by exploring core concepts, Hadoop infrastructure, real-world applications, data characteristics, and deployment trends
• Build familiarity with DevOps practices, including continuous integration and continuous deployment (CI/CD), to enhance software development and operational workflows
• Acquire essential data modeling techniques critical for managing and interpreting complex data structures in engineering and architectural roles
• Expand knowledge of Apache Spark by understanding its architecture, components, and functionalities such as Spark SQL, Spark ML, and Spark Streaming
• Gain expertise in Kafka fundamentals and tools like Kafka Connect and Kafka Streams for managing real-time data feeds and performing stream processing
• Explore Elastic Stack for real-time data analysis and visualization while understanding NoSQL databases to manage diverse data types
• Deepen understanding of data flow and pipelining with tools like NiFi and Streamsets to improve data collection, flow, and processing capabilities
• Learn orchestration and workflow management with tools such as Airflow and Jenkins to coordinate complex processes effectively

Languages