Been playing with #Deno and #FreshJS to build a small app for #LegendsOfRuneterra, and the developer experience has been amazing so far.
#DenoDeploy is freaking amazing. It's the closest I've ever gotten to push and forget. I usually like to #SelfHost, but for this project I didn't want to bother with all that.
Deno will be on the top of the list for whenever I want to work on a side project. (Now just need to think how I can champion for it at work...)
The Nazistack reimbursements are still in limbo (I’m going to try to talk to or at least tele-chat with an actual Substack proto-human this week to get this resolved), so everybody gets a Bonus Drop until that’s fixed.
Bluesky is going GA “this month” (I have two invite codes at the end of this Drop if you want to jump the line), and in preparation for the “big time”, they’ve also made it possible to view individual posts/threads sans-authentication.
So, today, we’re going to write some Deno-flavoured JavaScript and create both a CLI thread unroller as well as stand up a free thread unroller REST API (using virtually the same code).
Bluesky Threads & JSON
If you hit a given Bluesky post/thread, such as this one with Developer Tools open in an inconito session you can see a few useful items:
The “view source” version of the HTML has a handy <noscript> section with the bare minimum post and author information.
One XHR request for the JSON of the author info: <https://public.api.bsky.app/xrpc/com.atproto.identity.resolveHandle?handle=gregsargent.bsky.social>
Another XHR request for the JSON of the entire thread <https://public.api.bsky.app/xrpc/app.bsky.feed.getPostThread?uri=at%3A%2F%2Fdid%3Aplc%3A2amnkge5a6hplfwyesmxqkfl%2Fapp.bsky.feed.post%2F3khjvpz5cfc2e>
Please check those out before continuing.
“Um…hrbrmstr…that thread JSON is complete. Why do we need an unroller?”
Glad you asked!
That thread JSON is absolutely complete. Too complete. It has the author’s main thread, then all the nested and layered-in replies from the author and folks engaging with the content. The majority of references are also in “raw” Blueskly/ATProto form, meaning we need to do some work to turn any attached JPEG identifiers into fully href’d <img src=…> equivalents, and fish out any truncated URLs.
We’ll get to some of that in a moment.
Why Deno?
A thread unroller will be useful in CLI form, and we can turn Deno scripts into executables with a single command. An unroller API endpoint would also be pretty handy. Both idioms could use with a bit of caching, and Deno (unstable) has a built-in key/value store that “scales to the cloud” without you having to know the gory details.
I’m going to ask you to head to Deno Deploy and create a free account. You will not need to spend any coin to work with the resources in this Drop and deploy an API service to Deno. If you recall a previous Drop where we created our own edge service, the code will work there, too, minus the Deno key/value caching part (I need to modify the Supabase Docker setup to run Deno in “unstable” mode to make that work and just haven’t had time).
CLI Time
You can find the code for the CLI version of the Bluesky thread unroller on Codeberg. It has JDDoc comments and is also lightly annotated, so we won’t spend a ton of time working through some basic HTML and JSON parsing. We will focus (a bit) on this function which I’ve also put below:
function extractReplies(thread, authorDid) { const replies = []; const cid = thread.post.cid function traverseReplies(replyArray, pcid) { if (!Array.isArray(replyArray)) { return; } for (const reply of replyArray) { if (reply.post.author.did === authorDid) { if (reply.post.record.reply.root.cid == cid) { if (reply.post.record.reply.parent.cid == pcid) { replies.push({ uri: reply.post.uri, text: reply.post.record.text, embed: extractEmbeds(reply.post.record.embed, authorDid), facets: extractFacets(reply.post.record.facets) }); if (reply.replies && reply.replies.length > 0) { traverseReplies(reply.replies, reply.post.cid); } } } } } } traverseReplies(thread.replies, thread.post.cid); return replies;}
This function traverses the nested replies structures of the thread and uses the metadata of the original post ID and author ID, plus subsequent direct reply IDs, to ensure we’re just seeing what the author has directly contributed. If you need more, then the full, ugly, nested JSON is what you should work with.
As we traverse the replies, we extract the “embeds” — which are, for now, just images — and facets, which can be hashtags or URLs (we only focus on URLs).
Search the file for kv to see three touch points required to add a caching layer (which will speed-up subsequent requests for the same thread).
Take a peek at the included justfile to see how to call it on the CLI
In the dashboard for that project, you can find instructions on how to wire-up the global key/value store (if you want to).
Making It Real
The new “unrolled” JSON is still “just JSON”, so I whipped up an Observable Notebook that shows one way to visualize the thread.
Make sure to delete the project if you do not want to accidentally move from free to “paid” due to this project.
I have an alternate version of this running on my self-hosted “edge” service, since I’ll likely shut down the Deno Deploy one if it starts to cost real money:
Here are the aforementioned Bluesky codes (first-come/first-served):
bsky-social-zurbs-csz73bsky-social-r737p-bkppg
Remember, you can follow and interact with the full text of The Daily Drop’s free posts on Mastodon via @dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev ☮️
Ryan Dahl's weekly #Deno video update just dropped that includes a preview of the next Deno release which will have the ability to import wasm modules, blobs (images), plain text and urls for things like CSS files. He also previews the new website, and says they are actively working on JSR. "What is that," he asks, and answers "I don't know" with a smile. My guess is JavaScript Repository, an alternative to #npm. @deno_land https://youtu.be/5nv2zhic6Jk?si=QJrbEznW4I6WQFb0
Turns out my suspicion was correct: #Deno is fantastic for periodic jobs on my VPS. I love me some #TypeScript, and the whole import … from “https://url” concept makes it trivial to get started. npm compatibility is a big plus, of course.
Choose to try #pino for the #NodeToDenoChallenge and worked first try. Would be interesting to dig deeper into all of Pino's functionality to see if it holds up but a promising start nonetheless. #deno#denoland
1️⃣ pick a node project
2️⃣ run it with #deno
3️⃣ screenshot the output (success or failure)
4️⃣ share on twitter with #NodeToDenoChallenge
5️⃣ be eligible for prizes 🎁
With npm: specifiers, you have access to over 2 million modules on npm. Here's an example using cheerio, a popular package for parsing HTML and extracting data using jQuery selector syntax.
If you are not a Helix user, ignore this post — just some configuration gibberish. If you are a Helix fan, save yourself a hassle and copy/pasta 🍝 this one!
Our friends at @openai released a Deno-native TypeScript library supporting all their latest API offerings (GPT-4 Turbo, DALL-E 3, and more). Check the 🧵 for a full tutorial.
⚡️ Loads fast on Deno Deploy / Netlify / @supabase edge functions
👩💻 Great TypeScript DX in VS Code
Detecting changes in Deno KV with kv.watch makes it easier to build real-time applications like newsfeeds, analytics, multi-user collaboration tools, and more.
#Deno team video master Andy Jiang just dropped a fact-filled vid on YouTube about the fmt utility built into the @deno_land runtime. One fun fact he points out is that the Deno formatter is about 20x faster than Prettier. Find out about that and the various options for 'deno fmt' here:
#Deno head honcho Ryan Dahl in his weekly YouTube update announces v1.39 improvements due December 14 including test coverage output as HTML and WebGPU returning to the runtime with performance improvements. @deno_land https://youtu.be/wVEKhM11CBk?si=GOcYau37MWIUl8KN
Build Real-time Applications with the new "watch" API in Deno KV (deno.com)
Detecting changes in Deno KV with kv.watch makes it easier to build real-time applications like newsfeeds, analytics, multi-user collaboration tools, and more.