portal resources jobs companies p posthog senior data engineer

Senior Data Engineer 🔥


PostHog is an open-source product analytics platform. We provide product-led teams with everything they need to understand user behaviour, including funnels, session recordings, user paths, multivariate testing and more. PostHog can be deployed to the cloud, or self-hosted on existing infrastructure, removing the need to send data externally.

We started PostHog as part of Y Combinator's W20 cohort and had the most successful B2B software launch on HackerNews since 2012 - with a product that was just 4 weeks old. Since then, we raised $27m from some of the world's top investors [1], grew the team to over 30 and have shown strong product-led growth.

We’re now looking for a senior Data Engineer to join our Ingestion team. We have a community of over 20k+ developers using PostHog, mostly on the open source product, plus a 1,000+ Slack community and over 7,000 GitHub stars. And all of these numbers are going up, fast.

We hire globally, but are currently restricted to GMT -5 to +2 time zones.

What you’ll be doing:

We are looking for someone to take our ingestion pipeline to the next level. You will be working with our super talented Ingestion small team [2] to iteratively build out and shore up the functionality of our ingestion pipeline. A good chunk of this work will be focussing on our Plugins service [3]. This is the core of our data ingestion pipeline. It is responsible for transforming, augmenting, routing, and backfilling data to many different final destinations including the warehouse that we use to power PostHog, ClickHouse.

If in your spare time you love reading about Designing Data-Intensive Applications [4] and dream about producing and consuming large amounts of data from Kafka, then this is the spot for you!

If you like to see for yourself exactly the kind of projects you would be working on check out these:

  • Reworking our events schema in order to reduce joins at querytime [5]
  • Performing migrations on TBs of data (in someone else's datacenter) with zero downtime! [6]
  • You can read more about that last one (Async Migrations) here [7]
  • Plugin Server source code [8] (light reading 🍿 )

What we value:

  • We are open source [9] - building a huge community around a free-for-life product is key to our strategy.
  • We aim to become _the_ most transparent [10] company, ever. In order to enable teams to make great decisions, we share as much information as we can. In our public handbook _everyone_ can read about our roadmap, how we pay people, what our strategy is, and who we have raised money from. We also have regular team-wide feedback sessions [11], where we share honest feedback with each other.
  • We’re an all-remote company and writing things down is hugely important to us. We use asynchronous communication [12] to reduce time spent in meetings. We are structured for speed and autonomy - we are all about acting fast, innovating and iterating [13].
  • We are a #LI-remote company, which allows us to hire amazing people from all over the world, and foster an inclusive culture.

Requirements

  • Experience designing or operating large scale realtime or near realtime data pipelines
  • Operational knowledge and experience with Kafka at scale
  • Solid backend engineer skills

Nice to haves (if you don't have any of these you should still apply!):

  • Experience deploying realtime or near-realtime data pipelines to K8s environments
  • Working knowledge of the internals of Apache Flink [14] or other stateful streaming computation engines
  • Experience working with and operating a Data Lake / Lake House / Delta Lake [15] at scale.
  • Experience being a user and wearing your product analytics is always a huge advantage
  • Experience operating or being a user of ClickHouse or any other data warehouse
  • You enjoy geeking out about serializations and their tradeoffs

Benefits

What we offer in return:

  • 💰 Generous, transparent compensation [16] and employee-friendly equity [17] in PostHog
  • 🌴 Unlimited time off [18] with a 25 day minimum (in 2021 the team on average took 32 days off)
  • 🏥 Private medical insurance [19], including dental and vision (US and UK only)
  • 👵 👴 Pension/401k contributions [20] (4% matching)
  • 🍼 Generous parental, bereavement and child loss leave [21]
  • 📕 Training budget [22] and free books [23]
  • $200/month budget towards co-working or café working [24] and $250/month for team socials [25]
  • 🧠 Spill mental health chat [26]
  • 🤝 $100/month budget to provide support to open-source projects [27]
  • 💸 We'll be your first investor [28]
  • [29]🛫 Regular team off-sites (we went to Iceland in March) with carbon offsetting for work travel with Project Wren [30]

    We believe people from diverse backgrounds, with different identities and experiences, make our product and our company better. That’s why we dedicated a page in our handbook to diversity and inclusion [31]. No matter your background, we'd love to hear from you!

    Also, if you have a disability, please let us know if there's any way we can make the interview process better for you - we're happy to accommodate!


  1. https://posthog.com/handbook/strategy/investors
  2. https://posthog.com/handbook/people/team-structure/ingestion
  3. https://posthog.com/docs/user-guides/plugins
  4. https://www.amazon.com/Designing-Data-Intensive-Applications-Reliable-Maintainable-ebook/dp/B06XPJML5D
  5. https://github.com/PostHog/meta/pull/39
  6. https://github.com/PostHog/posthog/issues/7054
  7. https://posthog.com/blog/async-migrations
  8. https://github.com/PostHog/posthog/tree/master/plugin-server
  9. https://posthog.com/handbook/company/values#we-are-open-source
  10. https://posthog.com/handbook/company/culture#extremely-transparent
  11. https://posthog.com/handbook/people/feedback#full-team-feedback-sessions-1
  12. https://posthog.com/handbook/company/culture#we-write-everything-down
  13. https://posthog.com/handbook/company/culture#iteration
  14. https://flink.apache.org/
  15. https://delta.io/
  16. https://posthog.com/handbook/people/compensation
  17. https://posthog.com/handbook/people/compensation
  18. https://posthog.com/handbook/people/time-off
  19. https://posthog.com/handbook/people/benefits#private-health-insurance
  20. https://posthog.com/careers
  21. https://posthog.com/handbook/people/time-off
  22. https://posthog.com/handbook/people/training
  23. https://posthog.com/handbook/people/training
  24. https://posthog.com/handbook/people/spending-money#work-space
  25. https://posthog.com/handbook/people/benefits#team-socials
  26. https://posthog.com/careers
  27. https://posthog.com/handbook/people/benefits#support-open-source-projects
  28. https://posthog.com/handbook/people/benefits#well-be-your-first-investor
  29. https://posthog.com/handbook/people/benefits#well-be-your-first-investor
  30. https://www.wren.co/
  31. https://posthog.com/handbook/company/diversity

Other openings you might be interested in

More remote jobs

Let us send you new openings similar to Senior Data Engineer straight to your Inbox. Weekly or Daily. 7-day free trial 💌

The ability to work remotely increases employee happiness by 20 percent.