portal resources jobs companies s splitgraph senior software engineer

Senior Software Engineer


We're building the Data Platform of the Future

Join us if you want to rethink the way organizations interact with data. We are a developer-first company, committed to building around open protocols and delivering the best experience possible for data consumers and publishers.

Splitgraph is a seed-stage, venture-funded startup hiring its initial team. The two co-founders are looking to grow the team to five or six people. This is an opportunity to make a big impact on an agile team while working closely with the founders.

Splitgraph is a remote-first organization. The founders are based in the UK, and the company is incorporated in both USA and UK. Candidates are welcome to apply from any geography. We want to work with the most talented, thoughtful and productive engineers in the world.

Open Positions

Data Engineers welcome! The job titles have "Software Engineer" in them, but at Splitgraph there's a lot of overlap between data and software engineering. We welcome candidates from all engineering backgrounds.

Senior Software Engineer - Backend (mainly Python) [1]

Senior Software Engineer - Frontend (mainly TypeScript) [2]

Apply to Job [3] ← (same form for both positions)

What is Splitgraph?

 Open Source Toolkit

Our open-source product, sgr, [4] is a tool for building, versioning and querying reproducible datasets. It's inspired by Docker and Git, so it feels familiar. And it's powered by PostgreSQL, so it works seamlessly with existing tools in the Postgres ecosystem. Use Splitgraph to package your data into self-contained data images that you can share with other Splitgraph instances.

 Splitgraph Cloud

Splitgraph Cloud is a platform for data cataloging, integration and governance. The user can upload data, connect live databases, or "push" versioned snapshots to it. We give them a unified SQL interface to query that data, a catalog to discover and share it, and tools to build/push/pull it.

Learn More About Us

  • Listen to our interview on the Software Engineering Daily podcast [5]

  • Watch our co-founder Artjoms present Splitgraph at the Bay Area ClickHouse meetup [6]

  • Read our HN/Reddit posts (one [7] two [8] three [9] four [10])

  • Read our blog [11]

  • Read the slides from our early (2018) presentations: "Docker for Data" [12], AHL Meetup [13]

  • Follow us on Twitter [14]

  • Find us on GitHub [15]

  • Chat with us in our community Discord [16]

  • Explore the public data catalog [17] where we index 40k+ datasets

How We Work: What's our stack look like?

We prioritize developer experience and productivity. We resent repetition and inefficiency, and we never hesitate to automate the things that cause us friction. Here's a sampling of the languages and tools we work with:

Python [18] for the backend. Our core open source [19] tech is written in Python (with a bit of C [20] to make it more interesting), as well as most of our backend code. The Python code powers everything from authentication routines to database migrations. We use the latest version and tools like pytest [21], mypy [22] and Poetry [23] to help us write quality software.

TypeScript [24] for the web stack. We use TypeScript throughout our web stack. On the frontend we use React [25] with next.js [26]. For data fetching we use apollo-client [27] with fully-typed GraphQL queries auto-generated by graphql-codegen [28] based on the schema that Postgraphile [29] creates by introspecting the database.

  • **PostgreSQL [30] for the database, because of course.** Splitgraph is a company built around Postgres, so of course we are going to use it for our own database. In fact, we actually have three databases. We have auth-db for storing sensitive data, registry-db which acts as a Splitgraph peer [31] so users can push Splitgraph images to it using sgr [32], and cloud-db where we store the schemata that Postgraphile uses to autogenerate the GraphQL server.

  • **PL/pgSQL [33] and PL/Python [34] for stored procedures.** We define a lot of core business logic directly in the database as stored procedures, which are ultimately exposed by Postgraphile as GraphQL endpoints [35]. We find this to be a surprisingly productive way of developing, as it eliminates the need for manually maintaining an API layer between data and code. It presents challenges for testing and maintainability, but we've built tools to help with database migrations and rollbacks, and an end-to-end testing framework that exercises the database routines.

  • **PostgREST [36] for auto-generating a REST API for every repository.** We use this excellent library (written in Haskell [37]) to expose an OpenAPI [38]-compatible REST API for every repository on Splitgraph (example [39]).

- Lua (luajit [40] 5.x), C, and embedded Python [41] for scripting PgBouncer [42]. Our main product, the "data delivery network", is a single SQL endpoint where users can query any data on Splitgraph. Really it's a layer of PgBouncer instances orchestrating temporary Postgres databases and proxying queries to them, where we load and cache the data necessary to respond to a query. We've added scripting capabilities to enable things like query rewriting, column masking, authentication, ACL, orchestration, firewalling, etc.

Docker [43] for packaging services. Our CI pipeline builds every commit into about a dozen different Docker images, one for each of our services. A production instance of Splitgraph can be running over 60 different containers (including replicas).

Makefile [44] and docker-compose [45] for development. We use a highly optimized Makefile [46] and docker-compose so that developers can easily spin-up a stack that mimics production in every way, while keeping it easy to hot reload, run tests, or add new services or configuration.

Nomad [47] for deployment and Terraform [48] for provisioning. We use Nomad to manage deployments and background tasks. Along with Terraform, we're able to spin up a Splitgraph cluster on AWS, GCP, Scaleway or Azure in just a few minutes.

Airflow [49] for job orchestration. We use it to run and monitor jobs that maintain our catalog of 40,000 public datasets [50], or ingest other public data into Splitgraph.

Grafana [51], Prometheus [52], ElasticSearch [53], and Kibana [54] for monitoring and metrics. We believe it's important to self-host fundamental infrastructure like our monitoring stack. We use this to keep tabs on important metrics and the health of all Splitgraph deployments.

Mattermost [55] for company chat. We think it's absolutely bonkers to pay a company like Slack to hold your company communication hostage. That's why we self-host an instance of Mattermost for our internal chat. And of course, we can deploy it and update it with Terraform.

Matomo [56] for web analytics. We take privacy seriously, and we try to avoid including any third party scripts on our web pages (currently we include zero). We self-host our analytics because we don't want to share our user data with third parties.

Metabase [57] and Splitgraph [58] for BI and dogfooding [59]. We use Metabase as a frontend to a Splitgraph instance that connects to Postgres (our internal databases), MySQL (Matomo's database), and ElasticSearch (where we store logs and DDN analytics). We use this as a chance to dogfood our software and produce fancy charts.

- The occasional best-of-breed SaaS services for organization. As a privacy-conscious, independent-minded company, we try to avoid SaaS services as much as we can. But we still find ourselves unable to resist some of the better products out there. For organization we use tools like Zoom [60] for video calls, Miro [61] for brainstorming, Notion [62] for documentation (you're on it!), Airtable for workflow management [63], PivotalTracker [64] for ticketing, and GitLab for dev-ops and CI [65].

- Other fun technologies including HAProxy [66], OpenResty [67], Varnish [68], and bash. We don't touch them much because they do their job well and rarely break.

Life at Splitgraph

We are a young company building the initial team. As an early contributor, you'll have a chance to shape our initial mission, growth and company values.

We think that remote work is the future, and that's why we're building a remote-first organization. We chat on Mattermost [69] and have video calls on Zoom. We brainstorm with Miro [70] and organize with Notion [71].

We try not to take ourselves too seriously, but we are goal-oriented with an ambitious mission.

We believe that as a small company, we can out-compete incumbents by thinking from first principles about how organizations interact with data. We are very competitive.

Benefits

  • Fully remote

  • Flexible working hours

  • Generous compensation and equity package

  • Opportunity to make high-impact contributions to an agile team

How to Apply? Questions?

Complete the job application [72]

If you have any questions or concerns, feel free to email us at [email protected] [73]

Location

? WorldwideSee more jobs at Splitgraph


  1. https://www.notion.so/splitgraph/Senior-Software-Engineer-Backend-2a2f9e278ba347069bf2566950857250
  2. https://www.notion.so/splitgraph/Senior-Software-Engineer-Frontend-6342cd76b0df483a9fd2ab6818070456
  3. https://4o99daw6ffu.typeform.com/to/ePkNQiDp
  4. https://www.github.com/splitgraph/splitgraph
  5. https://softwareengineeringdaily.com/2020/11/06/splitgraph-data-catalog-and-proxy-with-miles-richardson/
  6. https://www.youtube.com/watch?v=44CDs7hJTho
  7. https://news.ycombinator.com/item?id=24233948
  8. https://news.ycombinator.com/item?id=23769420
  9. https://news.ycombinator.com/item?id=23627066
  10. https://old.reddit.com/r/datasets/comments/icty0r/we_made_40k_open_government_datasets_queryable/
  11. https://www.splitgraph.com/blog
  12. https://www.slideshare.net/splitgraph/splitgraph-docker-for-data-119112722
  13. https://www.slideshare.net/splitgraph/splitgraph-ahl-talk
  14. https://ww.twitter.com/splitgraph
  15. https://www.github.com/splitgraph
  16. https://discord.gg/eFEFRKm
  17. https://www.splitgraph.com/explore
  18. https://www.python.org/
  19. https://www.github.com/splitgraph/splitgraph
  20. https://github.com/splitgraph/Multicorn
  21. https://docs.pytest.org/en/stable/
  22. https://github.com/python/mypy
  23. https://python-poetry.org/
  24. https://www.typescriptlang.org/
  25. https://reactjs.org/
  26. https://nextjs.org/
  27. https://www.apollographql.com/docs/react/
  28. https://graphql-code-generator.com/
  29. https://www.graphile.org/postgraphile
  30. https://www.postgresql.org/
  31. https://www.splitgraph.com/docs/publishing-data/push-data
  32. https://www.github.com/splitgraph/splitgraph
  33. https://www.postgresql.org/docs/current/plpgsql.html
  34. https://www.postgresql.org/docs/current/plpython.html
  35. https://www.graphile.org/postgraphile/functions/
  36. https://postgrest.org/en/v7.0.0/
  37. https://www.haskell.org/
  38. https://github.com/OAI/OpenAPI-Specification
  39. http://splitgraph.com/mildbyte/complex_dataset/latest/-/api-schema
  40. https://luajit.org/luajit.html
  41. https://docs.python.org/3/extending/embedding.html
  42. https://www.pgbouncer.org/
  43. https://www.docker.com/
  44. https://www.gnu.org/software/make/manual/make.html
  45. https://docs.docker.com/compose/
  46. https://www.splitgraph.com/blog/makefile
  47. https://www.nomadproject.io/
  48. https://www.terraform.io/
  49. https://airflow.apache.org/
  50. https://www.splitgraph.com/blog/40k-sql-datasets
  51. https://grafana.com/
  52. https://prometheus.io/
  53. https://www.elastic.co/
  54. https://www.elastic.co/kibana
  55. https://mattermost.com/
  56. https://matomo.org/
  57. https://www.metabase.com/
  58. https://www.splitgraph.com
  59. https://en.wikipedia.org/wiki/Eating_your_own_dog_food
  60. https://www.zoom.us
  61. https://miro.com/
  62. https://www.notion.so
  63. https://airtable.com/
  64. https://www.pivotaltracker.com/
  65. https://about.gitlab.com/
  66. http://www.haproxy.org/
  67. https://openresty.org/en/
  68. https://varnish-cache.org/
  69. https://mattermost.com/
  70. https://miro.com/
  71. https://www.notion.so
  72. https://4o99daw6ffu.typeform.com/to/ePkNQiDp
  73. mailto:[email protected]

Other openings you might be interested in

Senior Java Backend Engineer - Remote

Senior Java Backend Engineer - Remote

This position requires U.S. residency. Summary SemanticBits is looking to hire a talented senior software developer who can develop modern digital health services implemented as highly scalable Java services deployed on the AWS cloud in Docker cont

yesterday
Back End Senior Software Engineer (Springfield, VA)

Back End Senior Software Engineer (Springfield, VA)

Please Note: Position is Fully Remote. We are looking for someone who has a passion for solving new and interesting problems. We welcome what you bring: new ideas and fresh perspectives that can help grow our team and our platform-both internally and

today
Remote Back-End Senior Software Engineer

Remote Back-End Senior Software Engineer

Please Note: Position is Fully Remote. We are looking for someone who has a passion for solving new and interesting problems. We welcome what you bring: new ideas and fresh perspectives that can help grow our team and our platform-both internally an

this week
Senior Software Engineer Americas

Senior Software Engineer Americas

Company Description Shopify’s mission is to make commerce better for everyone. From building a new product feature for our commerce platform, to helping a merchant troubleshoot an issue over the phone, we want to empower our ecosystem through our wo

today
Senior Software Engineer

Senior Software Engineer

Superside is growing fast and we're currently looking to expand our engineering team with a bunch of talented and product-driven engineers. We are looking for both front-end, full-stack, and back-end engineers. Read more to learn about Superside, the

today
Senior Software Engineer (New York, NY)

Senior Software Engineer (New York, NY)

Our team is looking for a senior or principle backend software engineer to join our core product team as an individual contributor and help build & scale Rho’s commercial banking technology. We require a team member who can be a key contributor to ou

today
Senior Software Engineer

Senior Software Engineer

Our team is looking for a senior or principle backend software engineer to join our core product team as an individual contributor and help build & scale Rho’s commercial banking technology. We require a team member who can be a key contributor to ou

today
Mid/senior software engineer

Mid/senior software engineer

Headquarters: London URL: https://petsapp.com/careers At PetsApp we're doggedly determined to make veterinary care purrfect. We provide a platform for veterinary clinics that allows them to better engage with pet owners and make sure their patients

this week
Senior Staff Software Engineer

Senior Staff Software Engineer

Company Description Shopify’s mission is to make commerce better for everyone. From building a new product feature for our commerce platform, to helping a merchant troubleshoot an issue over the phone, we want to empower our ecosystem through our wo

today
More remote jobs

Other jobs at Splitgraph

One job in the last 60 days · 1 in total · avg 0.51 jobs/mo · 177 job visits

Senior Software Engineer

Senior Software Engineer

WE'RE BUILDING THE DATA PLATFORM OF THE FUTURE Join us if you want to rethink the way organizations interact with data. We are a developer-first company, committed to building around open protocols and delivering the best experience possible for dat

8w ago
Splitgraph