Dad. Technical Author, Data Engineer. Data practitioner (20y) • Writing at ssp.sh since 2015. Focused on the craft of data engineering & storytelling. 📚 vault.ssp.sh • 📖 @dedp.online ❯ #dataengineering, #opensource, #writing, #obsidian, #neovimhttps://blacksky.community/profile/ssp.shhttps://blacksky.community/profile/ssp.sh/post/3mpdszjqov32nNew blog post: The Process of Smart Note-Taking. Connecting your Zettelkasten, owning them in plain text, and publishing what you learn in public. https://www.ssp.sh/blog/process-smart-note-taking/28 Jun 2026 11:00 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mpdszjqov32nhttps://blacksky.community/profile/ssp.sh/post/3mpdswwkbek2vTIL—to send git changes for an article you made, or code changes, you can just send a simple HTML report that visually shows all the changes. Just install the `diff2html-cli` and run: > git diff | diff2html -i stdin -F changes.html https://www.ssp.sh/brain/git-diff-report/28 Jun 2026 10:58 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mpdswwkbek2vhttps://blacksky.community/profile/ssp.sh/post/3mpdnbffyn22tI've added a new full-text search that searches both the second brain and blogs in one singular search. Plus, you can filter by dates, e.g., `This year` or `Last 2 years`, and toggle between Blog or Brain. Try it, hit `ctrl+k` (or cmd+k) and search ssp.sh. [contains quote post or other embedded content]28 Jun 2026 09:17 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mpdnbffyn22thttps://blacksky.community/profile/ssp.sh/post/3mpb76z2jrn2cI have notes from 2007 that still open in any editor on the planet. That is the whole pitch for plain text. Every device, including ones long gone and ones not yet invented, can read and edit plain text. Will Notion be around in twenty years? Maybe. Will Markdown? Almost certainly.27 Jun 2026 10:00 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mpb76z2jrn2chttps://blacksky.community/profile/ssp.sh/post/3mp7aflwsps2aBefore and after 🙃 [contains quote post or other embedded content]26 Jun 2026 15:16 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mp7aflwsps2ahttps://blacksky.community/profile/ssp.sh/post/3mp72ygmr2k2mIf I were to update the entry page at ssp.sh, what direction should I go? Here are 3 Claude Opus 4.8 suggestions based on my taste. Any good, wdyt?26 Jun 2026 13:39 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mp72ygmr2k2mhttps://blacksky.community/profile/ssp.sh/post/3mp72jmuxd72iI vibe-coded a markdown email client called Neomd in a language I do not know well. What I learned: agents are brilliant when the project is small and the spec is clear. They start drifting when the codebase grows past a few thousand lines and the architecture is loose. https://www.ssp.sh/brain/vibe-coding https://www.ssp.sh/brain/vibe-coding/26 Jun 2026 13:31 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mp72jmuxd72ihttps://blacksky.community/profile/ssp.sh/post/3mp5a3lc5nt2eA curated list of many open-source data engineering projects collected over the years. For data engineers and developers who like to get their hands on some hands-on projects, hand-picked with ❤️. https://www.ssp.sh/brain/open-source-data-engineering-projects/25 Jun 2026 20:05 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mp5a3lc5nt2ehttps://blacksky.community/profile/ssp.sh/post/3mp3ypvhh5226If you are a writer, what's your favorite physical notebook and pencil? I like best: Leuchtturm1917 Notebook, B5, Plain, Softcover. But as expensive as it is, the 2nd one is cheap here at the local store. Important that it's plain to mind map or bullet list, sketch, or do anything. https://www.ssp.sh/brain/paper-notebooks/25 Jun 2026 08:20 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mp3ypvhh5226https://blacksky.community/profile/ssp.sh/post/3mp3tei3awa27Query data where it lives without moving it. That's the promise of federated query engines. Trino, Presto, Dremio, DataFusion let you run SQL across multiple data sources in a single query. https://www.ssp.sh/brain/federated-query-engines https://www.ssp.sh/brain/federated-query-engines/25 Jun 2026 06:45 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mp3tei3awa27https://blacksky.community/profile/ssp.sh/post/3mp24ac4blc25Amazing talks so far. If you don't have time yet, and want to read the notes, I'm taking some as we go. https://www.ssp.sh/brain/duckcon-7-ams/ [contains quote post or other embedded content]24 Jun 2026 14:18 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mp24ac4blc25https://blacksky.community/profile/ssp.sh/post/3mozxcru5xk2mOperationalizing data orchestration is hard, and that's a pity, as it's at the heart of your data platform's operating system. But what's even harder is deploying it across multiple environments, or upgrading the infra, or only the pipeline. How does it sync with the data stack versions we use?24 Jun 2026 12:50 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mozxcru5xk2mhttps://blacksky.community/profile/ssp.sh/post/3mozg6m5wk22eThe work from a writer and data engineer, on the road, or just around home. Sometimes working from home, sometimes around the 'Coffee Break Rhythm', and sometimes wherever I am. Where do you do your best work? Find all images: https://xcancel.com/sspaeti/status/206967582003819739024 Jun 2026 07:43 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mozg6m5wk22ehttps://blacksky.community/profile/ssp.sh/post/3moxluld76k26Today's Office.23 Jun 2026 14:20 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3moxluld76k26https://blacksky.community/profile/ssp.sh/post/3mowsgqih2l2iDuckLake is a new lakehouse format from the DuckDB folks. The pitch is short: keep your Parquet files, but put all the metadata in a SQL database instead of in a pile of Avro and JSON files on object storage. www.ssp.sh/brain/ducklake https://www.ssp.sh/brain/ducklake/23 Jun 2026 06:45 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mowsgqih2l2ihttps://blacksky.community/profile/ssp.sh/post/3movgubr2622pWhat a great interview, really a ton to learn from. Don't watch it if you don't want to learn Zig. Andrew is very convincing 😀. https://www.youtube.com/watch?v=iqddnwKF8HQ22 Jun 2026 17:45 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3movgubr2622phttps://blacksky.community/profile/ssp.sh/post/3mouveaabxg25If you want to get started with data engineering, look no further. I get asked constantly, «How to get started with data engineering?». That's why I created a single-entry point. This starter note includes backlinks and an interactive graph to help you learn more and dive into the full DE vault.22 Jun 2026 12:32 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mouveaabxg25https://blacksky.community/profile/ssp.sh/post/3moudgeyud22d> I just had to retire another phrase from my writing. The "It's not X, it's Y" construction. This is what Marc wrote. And as a fellow writer, I thought about it a lot. To me, I like the Tim Ferriss metaphor for photographers22 Jun 2026 07:11 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3moudgeyud22dhttps://blacksky.community/profile/ssp.sh/post/3mosjsqc5pc2dWalks are better when they are walks. The most underrated skill I have built in the last few years is the ability to do nothing. I walk in nature, no headphones. For a long time I listened to audiobooks on walks, but it always felt like more work, like a productivity tax. https://www.ssp.sh/brain/turn-off-shut-down-re-energize https://www.ssp.sh/brain/turn-off-shut-down-re-energize/21 Jun 2026 14:00 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mosjsqc5pc2dhttps://blacksky.community/profile/ssp.sh/post/3moplwlgomp2bMost writing advice gets this backwards. We talk as if you think first, and then write down what you thought. That is not how it works. If you are thinking without writing, you only think you are thinking. That line is from Leslie Lamport, and one of the most useful things I have read about writing20 Jun 2026 10:00 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3moplwlgomp2bhttps://blacksky.community/profile/ssp.sh/post/3moneooig4q2dData analytics, as a distinct field, mostly exists because dbt was so successful. Before dbt, this work happened already. It just lived in BI tools, in Excel, in the heads of domain experts and power users on the business side. dbt made it a profession by giving it a tool, workflow, & vocabulary.19 Jun 2026 12:45 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3moneooig4q2dhttps://blacksky.community/profile/ssp.sh/post/3momxsuvtpc2fA new social network W (wsocial.news), built on AT Proto. You only get access after you've been verified. Slogan: > a social media platform made in Europe, for the world. Governed by European law, data hosted on European infrastructure, and designed for conversations among humans around the world.19 Jun 2026 08:54 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3momxsuvtpc2fhttps://blacksky.community/profile/ssp.sh/post/3molecc4nec2fPollmd: A minimal newsletter poll that works anywhere with just plain links. https://q.ssp.sh/18 Jun 2026 17:32 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3molecc4nec2fhttps://blacksky.community/profile/ssp.sh/post/3moka3yztyk2xIn 2025, AI in data engineering was mostly slop on top of slop. The agents themselves are great. Almost every wrapper above them was not. For 2026, my prediction and hope is the same thing: less hype, more value. Stop putting AI in the name. Treat it as a tool, not the product.18 Jun 2026 06:45 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3moka3yztyk2xhttps://blacksky.community/profile/ssp.sh/post/3moj42qrh5b2yI have been keeping a running timeline of data engineering acquisitions since 2022. It is getting hard to keep up. https://www.ssp.sh/brain/data-engineering-acquisitions https://www.ssp.sh/brain/data-engineering-acquisitions/17 Jun 2026 20:00 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3moj42qrh5b2yhttps://blacksky.community/profile/ssp.sh/post/3moig73s7lc2lStarted a new Reddit community r/dataengineeringvault. Join if you like my content & genuine news from the open-source data engineering ecosystem. I'll share the latest, but I'm happy to get your blogs. I actively encourage links/blogs (not AI-generated). https://sh.reddit.com/r/dataengineeringvault/.17 Jun 2026 13:28 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3moig73s7lc2lhttps://blacksky.community/profile/ssp.sh/post/3mohx2xply22zQuack is a new protocol that enables client-server architecture for DuckDB over TCP/IP and HTTP(S), announced at AI Council 2026. It allows one DuckDB instance to act as a server and another as a client, with remote database attachment, query pushdown (computing on the server vs. the client). https://www.ssp.sh/brain/quack-protocol-duckdb17 Jun 2026 08:58 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mohx2xply22zhttps://blacksky.community/profile/ssp.sh/post/3mof7666sqw2wData engineering spent fifteen years building orchestrators — Airflow, Dagster, Prefect, Kestra. We are now doing the same thing for AI agents. Agor from Max the original creator of Airflow. Agent Teams from Anthropic, native to Claude Code. Omnigent from Databricks. Superset-sh, herdr, Gastown.16 Jun 2026 06:45 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mof7666sqw2whttps://blacksky.community/profile/ssp.sh/post/3mof3qzz7qk2cMade the URL shorter and added a result page to see the result directly. See the survey result below at q.ssp.sh/result/init [contains quote post or other embedded content]16 Jun 2026 05:43 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3mof3qzz7qk2chttps://blacksky.community/profile/ssp.sh/post/3moeatkyb322sI created a minimal, anonymous, non-tracking survey for my newsletter that works in Markdown with DuckDB Quack. I can simply answer as many questions on the fly, and for a survey, with this, I can query the results from client DuckDB via Server, as shown below.15 Jun 2026 21:42 +0000at://did:plc:edglm4muiyzty2snc55ysuqx/app.bsky.feed.post/3moeatkyb322s