Enterprise Data Lake Platform Leader
Oversee Enterprise Data Lake development and operations to acquire, ingest and manage data from variety of Enterprise data sources. This will include partnering with data management capabilities including data ingestion, storage, metadata, data quality and data lineage for an extensible data lake.
Dev/ops leader that oversees the implementation and support of GP IT’s Enterprise data lake platform. As a key member of the leadership team this role contributes to the development and execution of the enterprise wide data strategy and is responsible for ensuring the platform and operating environment supporting this strategy are fully aligned, positioning the team to deliver compelling business value in support of data and analytics product teams across the enterprise.
What You Will Do In Your Role
- Build and lead a high-performing data engineering team. Inspire and deliver data engineering innovations that fuel the growth of business; coach team members and grow the skill sets.; Mentor and guide team members as needed
- Work with stakeholders to gather, understand, document and prioritize data sourcing requirements ensuring that the agile product team backlogs are refined and well understood by the teams; Ability to influence across different groups and levels in the organization
- Provide technical leadership in data engineering practices. Partner with architecture, security, data delivery teams, data scientists and business leaders to establish a strong relationship and trust for prioritizing and delivering data products on time and with a high degree of quality
- Managing cross dependencies across teams to ensure alignment, as well as adjusting team directions when gaps exist
- Accountable for the design, build and run of the data sourcing strategy to support downstream consumptionacross consumption methods (traditional BI and advanced analytics)
- Participate in product release and roadmap planning activities
- Manage production data pipelines and ensure SLAs are aligned to business needs
- Manage Infrastructure Capacity and resource allocation to optimize spend
- Participate in solution testing with stakeholders and development teams to ensure the integration solutions are working as expected and meeting all business objectives
- Prioritize projects across the team and allocate resources to meet business and team goals
- Partner with architecture to establish patterns for re-use, standards, and best practices for DataOps such that problem resolution has minimum latency and incidents are at a minimum
- Process the data into clean, accessible sets in alignment with enterprise taxonomy standards
- Establish structured sandbox environment (e.g., test and learn zone) to support experimentation where appropriate
- Interface with other technology teams to ingest, process, and store data from a wide variety of data sources using AWS data technologies
- Creation and support of real-time data pipelines & querying capability leveraging enterprise standard technologies (e.g., Kafka/Kinesis/SQS, Glue, Spark, Redshift/Spectrum and Athena)
- Partner with the enterprise data governance tools teams and federated data management teams to establish data dictionary and authoritative sources for the core data elements required to support Business processes and reporting
- Recommend technological solutions to improve data quality and data integration across the enterprise data platform
- Implement the appropriate governance structure to prescribe data handling, protection, and user provisioning controls
- Partner with enterprise data governance tools to support operational activities related to data management such as metadata capture, data quality, data security, data lineage, data inventory; Integrate data and metadata with enterprise tool
The Experience You Will Bring
- Bachelor's Degree with 10+ years management experience in building and leading data engineering teams
- 5+ years of industry experience in building & delivering highly performing data engineering solutions: Ingestion, Curation, Modeling, Persistence, Computation and Delivery
- Proven Track record of guiding teams through unstructured technical problems to deliver business impact. Demonstrated Ability to translate high-level business objectives into actions.Must be able to reference specific work product in relation to these skills.
- 5 years experience at running cross-functional relationships and communicating with leadership across multiple organizations
- 3 – 5 years experience in delivering data solutions and architecture using AWS services like SQS, Kinesis, S3, EMR, Spark, Athena, Redshift, Sagemaker, IOT Core etc.
- Demonstrated understanding of Data Architecture, Data Integration, Data Quality, Data Governance and Data Lineage concepts.Must be able to reference specific work product in relation to these skills
- 5 years experience in building data governance and security in data lake, implementing and supporting data catalog solutions.
- Demonstrated communication skills to collaborate with multiple Engineering, IT & business teams.Must have worked in a matrixed environment and provide examples of communication strategy leveraged
- 5 plus years experience working in Agile environments supporting multiple product owners with competing priorities.
What Will Put You Ahead
- Product management experience in Data Management areas (e.g. Data Governance/Data Quality/Data Integration, ETL, Big Data, Streaming Data) and/or equivalent experience as a Product Manager, Technical Lead, Enterprise Architect, Solutions Architect, and has handled multiple data from varied sources (raw, transactional, real-time, streaming, etc.)
- Extensive work in Enterprise data projects (ingestion, pipeline, integration, etc.)
- Experience working with a variety of database technologies.(e.g. relational, non-relational, timeseries, multi-dimensional…NoSQL, Postgres, Cassandra, MongoDB, Redshift, etc)
- Experience in working with both commercial and manufacturing operations data sources.
Salary and Benefits Commensurate with Experience.
Equal Opportunity Employer.
Except where prohibited by state law, all offers of employment are conditioned upon successfully passing a drug test.
This employer uses E-Verify. Please visit the following website for additional information: www.kochcareers.com/doc/Everify.pdf