Your future starts here
- Work collaboratively with technical team members to develop, deploy and production support data engineering jobs and pipelines – spark structured stream processing jobs across one or more streams, persisted to a data lake.
- Work with leads and architects to prototype designs and technologies.
- Design and develop data engineering jobs with an eye on quality attributes (and trade-offs) such as simplicity, reliability, scalability, extensibility, performance. Build or reuse common frameworks where possible.
- Set and adhere to coding and style guides to ensure consistency. Ensure code quality through mechanisms including code coverage, peer reviews, pair programming.
- Create and deploy streaming data quality checks with remediation runbooks. Adhere to data governance and compliance requirements and standards.
- Provide recommendations for and help curate technical backlogs for technical debt management and continuous improvement.
- 5+ years of total experience working with data and data pipeline technologies at scale.
- 3+ years of solid Python development experience – including: Robust unit and integration testing and Git driven CI/CD to production
- 2+ years of proven experience developing streaming data processing tasks using pySpark and SparkSQL
- Solid understanding of Spark, preferably Databricks Spark and Apache Delta
- Read streaming data from Kafka, Apache Delta or similar, merge /enrich /qualify data inline, capture operations metrics, persist to target, debug and optimize
- At scale and in production
- Experience leveraging public cloud data and analytics services. Azure preferred, but AWS/GCP also works.
- Nice to have: Super user experience with reporting tools such a PowerBI.
Required education and experience
- College/University Degree or equivalent.
More about G2O
We’re different than other companies at G2O. We blend the research and design, technology, and data expertise to deliver the solutions our clients crave — and we do all of this as one in-house team, from vision to execution. We’re also the largest company of our kind based in Ohio to do this — and have been evolving how we do it for 40 years.
Individually, we bring a wealth of experience from diverse backgrounds — personally and in business. We’re a diverse and passionate team of leaders and experts in technology, data, analytics, design, content, and more. But we think we’re best when we put our minds together. Each person brings something distinct to our team — some unique flavor to their background or their experience. That makes for stronger collaboration — and elevates the outcomes for our clients.