Data Engineer Lead
Bisviews
Published Date: • Remote
Description
Bisviews is seeking a highly skilled Data Engineer Lead to oversee the design and management of scalable data pipelines and infrastructures. The ideal candidate has over six years of experience in data engineering, including a minimum of two years in a leadership role supervising a team of 10 members. This role demands advanced expertise in Python programming and SQL, combined with a deep understanding of data pipeline engineering, especially within cloud environments like Google Cloud Platform. The Data Engineer Lead will be central to auditing existing data systems, driving improvements, and shaping a strategic roadmap for the data platform's evolution. The successful candidate takes ownership of building and maintaining both batch and streaming data pipelines, modeling data to support various business units like product, growth, and operations. They will also contribute to critical trust and safety initiatives such as fraud detection and fake review prevention. Apart from technical responsibilities, this role involves mentoring junior engineers and interns, promoting best practices in data quality, privacy compliance, and continuous monitoring of data workflows to ensure robust and secure data handling aligned with GDPR and NDPR regulations.
Responsibilities
- Audit and evaluate the current data platform to identify strengths, weaknesses, and areas for improvement.
- Develop a comprehensive, phased roadmap to enhance the data platform and streamline processes.
- Design, build, and maintain reliable batch and streaming data pipelines that operate efficiently at scale.
- Create and maintain dimensional and event-based data models to support product, growth, and operational metrics.
- Implement data quality checks, monitoring systems, and compliance protocols to uphold data privacy standards, including GDPR and NDPR.
- Support trust and safety initiatives by contributing to fraud detection mechanisms and fake review prevention strategies.
- Lead, coach, and mentor a team of data engineering interns and junior engineers to foster professional growth and high-quality work output.
- Conduct regular audits of existing systems to identify bottlenecks or inefficiencies and drive continuous improvements.
- Collaborate with cross-functional teams to align data engineering efforts with business objectives and operational needs.
- Explore and incorporate advanced tools and technologies such as Airflow, dbt, Pub/Sub, and Cloud Composer to enhance pipeline orchestration and workflow automation.
Experience
6 Years
Apply By
Required Skills
- Python
- PII handling
- Production Data Warehousing
- Data Privacy and Compliance (GDPR, NDPR)
- Data Quality and Monitoring
- Dimensional and Event-based Data Modeling
- dbt
- Airflow
- Google Cloud Platform (BigQuery, Cloud Storage, Pub/Sub, Cloud Composer)
- Data Pipeline Engineering (ETL/ELT)
- SQL
Nice to Have Skills
- Mentoring and leadership
- Trust and Safety Initiatives
- Pub/sub
- Dataflow
- Kafka
- TerraForm
- Looker
- Metabase
- Marketplace or Review Platform Knowledge