We have two new speakers. One of them will be talking about #Drupal and #CloudFlare and the other oneis the creator of a $NodeJS behavior driven automation testing tool based on #Cucumber and #puppeteer
Though the plugin still has some rough edges and missing features, overall I think it is working pretty well for what I wanted. When I post on joelencioni.com, I can see my post in my Mastodon feed pretty quickly—though sometimes the formatting isn’t the best but I suppose that’s just the way it is. And, commenting is bi-directional: replies from Mastodon show up as comments on my blog, and replies to those comments from my blog show up as replies on Mastodon. Great!
The ActivityPub plugin watches for when the client sends a request HTTP header that is asking for content with the mime type of “application/activity+json”. If that type of content is requested, then instead of responding with the web page, it will respond with some JSON data meant for machine consumption instead of human consumption. This is how the syndication works, and that all seems fine.
However, I have been bumping into an issue due to the way this all works together with the CDN I chose for page caching, CloudFlare.
This is what one of my blog posts looks like when the ActivityPub JSON is served instead of the web page.
The problem is that CloudFlare will cache the first version of the page that is requested and serve that up to everyone going forward, regardless of the type of content being requested.
Thankfully, I found a new approach to try. Using CloudFlare workers, I can program the CDN to vary the content based on this header with this bit of code:
export default { async fetch(req) { const acceptHeader = req.headers.get('accept'); const url = new URL(req.url); if (acceptHeader?.indexOf("application/activity+json") > -1) { url.searchParams.append("activitypub", "true"); } return fetch(url.toString(), { cf: { // Always cache this fetch regardless of content type // for a max of 5 minutes before revalidating the resource cacheTtl: 300, cacheEverything: true, }, }); }}
This tells CloudFlare to look at the accept header, and if it has “application/activity+json”, it will add “activitypub=true” to the request query string (the part of the URL after the question mark) behind the scenes, which effectively makes it a different URL. This allows the different content to be cached and served up differently, which I think should solve the issue for me for good. If you still see this problem, please let me know!
For a very small instance with only a couple of concurrent users a CDN might not make much difference. But if you take a look at your web server logs you’ll quickly notice that every post / like / vote triggers a storm of requests from other instances to yours, looking up lots of different things. It’s easy to imagine how quickly this would overwhelm an instance once it gets even a little busy.
One of the first web performance tools people reach for is to use a CDN, like Cloudflare. But how much difference will it make? In this video I show you my web server logs before and after and compare them.
The short answer is – before CDN: 720 requests. After CDN: 100 requests.
Usually just turning on a CDN with default settings will not help very much, you’ll need to configure some caching rules or settings. By watching your server logs for a while you’ll get a sense for what needs to be cached but check out mine for a starting point:
Beware of caching by URI Path because often fediverse software will return different data depending on the Accept header that the requester sets. For example, on PieFed and Lemmy instances a request by a web browser to /post/123 will return HTML to show the post to someone. But when that same URL is requested with the Accept: application/ld+json header set, the response will be an ActivityPub representation of the post! You don’t want people getting activitypub data in their browser and you don’t want to be serving HTML to other instances. Once you spot a URL you want to cache, use a tool like Postman to set the Accept header and make a fake ActivityPub request to your instance and see if you get back HTML or JSON.
Another problem that can happen is that often a response will vary depending on whether the viewer is logged in, or who is logged in. If you can figure out how to configure the CDN to pay attention to cookies or whatever headers are used for Authentication by your platform then you might be able to cache things like /post/*… I couldn’t.
The things I’ve chosen to cache by URI Path above are ones that I know don’t vary by HTTP header or by authentication.
Although we can’t use URI Path a lot of the time, we can cache ActivityPub requests by detecting the Accept: allocation/ld+json header:
https://join.piefed.social/wp-content/uploads/2024/02/caching_activity2-1024x811.pngThis will cache all ActivityPub requests, regardless of URL. People browsing the same URLs as those used by ActivityPub will be unaffected as their requests won’t have the special HTTP header. I used a short TTL to avoid serving stale data when someone quickly edits a post straight after creating it.
There seems to be a deep vein of optimization here which I’ve only just started to dig into. These changes have made a huge difference already and for now my instance is under very little load so I’ll leave it there for now…
I’ve added The Love Burn, happening this weekend in Miami, to the dust app (https://dust.events?app). The backend is now built using #Cloudflare workers, D1 for database and R2 for static storage. I’ve now got imports from Airtable and Google Maps KML format. End result is theme camps are on the map with GPS directions to restrooms.
So, does anyone know how to contact #Cloudflare when for some reason their #DNS simply... doesn't list my domain anymore? Others do and my authoritative DNS does as well.
Figured out a pretty low-effort way to take payments in a Chrome extension - no server required. Don't have to give up an extra 5% to ExtensionPay either 😄
I was going to write a blog post after work today, instead I'm arguing about Cloudflare's broken "are you a human" security check with Zenva's tech support.
Same problem on the Better Business Bureau's website.
Clearly Cloudflare is NOT helping these people get their stuff configured properly.
Maybe I'll write a blog post about Cloudflare being broken. Works in Chrome or from my iPhone, of course.
If Substack is perfect for your needs then use that. Your problem with substack prolly isn't who else uses it, but rather, that you yourself are calling a proprietary, privacy disrespecting deprecated monolithic silo a "Perfect solution".
Instead of doing what's right, and for the right reasons, you eschew dogfooding on #FOSS when you should be championing it, and call a professional data mining haven perfect, when it is anything but.
Well, you're already on the Fediverse, so you should know better, but I'll dispense with the lecture now and point out a few good FOSS solutions that are Fediverse powered (and one that isn't, but still rocks as a publishing platform) for you:
Option #1, #WriteFreely, which you can find over at its git repo under https://gitHub.com/writefreely/writefreely.
Option #2, deploy yourself a #WordPress site, Then install the #ActivityPub plugin - the latest release publishes into the Fediverse and allows any Fediverse account to reply/comment threads natively - like I'm responding now. It also allows anyone on the Internet to join the discussions as well. WordPress has many options for subscriber lists, Etc., as well as #paywalled#digital_downloads, if you like.
Option #3, #Mitra is a Fediverse publishing platform that currently supports paid subscriptions for Authors: https://mitra.fediverse.observer/list - pick one that has open registrations or self-host yourself, like all of the other solutions here :)
If you're really talking about maintaining subscribers lists, but especially Having a subscriber list and building it up, then most ignorant folks would recommend HubSpot - but they would be wrong, because you can get the same powerful inbound marketing solution / #CRM, only better, for #FREE (That's a bare minimum savings of over $500/month)!!! So install #Mautic and let it do what it does, which you can get here: https://www.mautic.org/download/source-code and then after that, use it in conjunction with the following FOSS application that was tailor made for exactly what you're asking for...
#Ghost is FOSS, and in conjunction with an inbound marketing platform like Mautic is the perfect dynamic duo - like Batman and Robin. But even better, is that I'm going to point you towards a #HowTo that is an actual cookbook #tutorial written by someone expressing the same lamentations as yourself, and here's the exact solution they've provided for you:
By the way, your Mautic server also integrates directly with#MailGun (or Sendgrid, SendinBlue, SparkPost, etc.) to complete your transactional email system that will tell you when each and every recipient received, viewed (and or how long) your emails, as well as how many times they looked at those emails, with a bunch of other tools as well.
I hope that helps, and I'm very glad that you came to your senses about not using a privacy disrespecting, proprietary closed source solution like Substack - besides, registering your own domain name would have hidden the fact that you were using substack anyway, so it's about YOU doing the right thing the right way. Please choose your software in the future based upon the freedoms and ethics it offers in serving you and your customers. There's evil people everywhere, and the smart ones are using FOSS too - not substack.
It's good to receive feedback that helps people determine information that has value to others. It helps us focus on topics with merit.
There are a couple of additional things I'd like to address though, as briefly I can, considering I'm a rather loquacious sort ;)
I think it was @frogzone that brought up the general controversies that typically do follow #Cloudflare around. I have privacy conscious friends on both sides of that widening chasm...
In general it tends to be the developer sorts that although are cautious, reserved usually, when passing around compliments where Cloudflare is concerned, they're also the pragmatists where performance and dare I say security is concerned, and are often quite willing to turn to Cloudflare (specifically, as a #CDN).
With respect to security concerns, it is true that incorporating a CDN does provide a level of obfuscation of the IP spectrum, that is often cited as a major reason by hosting providers for the customer to incorporate/subscribe to CDN services (more often than not, Cloudflare - because they offer better kickbacks (er.... incentives) to hosting providers.
Then there's the hard core privacy concerned folks. #Last_Mile delivery performance considerations typically being much less of a compelling reason to use, let alone pay, for a CDN like Cloudflare to be injected into the website admin's #DNS. This is because, and let's be real here folks, most websites don't generate anywhere near the levels of traffic that their Nginx or Apache Servers can easily serve up, and for folks on the other side of the world from the particular website, a few milliseconds on a clear day is negligible.
Now, if you're running a very busy site, like... Etsy, or even really popular sites with thousands of requests per minute then you can really benefit by spreading your cache around the globe on super fast CDN services. Even a site that receives on average 1 request per second (60 per minute - and that's pretty respectable traffic) doesn't really benefit enough from the #caching related benefits of a CDN to mark a compelling case - the Last Mile Delivery, however, to Oslo, Norway, from a website in Melbourne, Australia... that can indeed improve perceived response by 250ms (2.5 seconds) or so.
So, just like these so-called VPN services, like NordVPN, etc., there needs to be an effort to educate the consumer as to the actual benefits expected for specific matters - some may be important considerations for the consumer, while others may just be a tech support person in a boiler room trying to reach that bonus number for the month... I've seen waaaay too many people purchase services they really didn't need or would receive much benefit from, and many support desk personnel upselling customers with things they probably shouldn't have.
Now, there's another thing I didn't mention - #Denial_of_Service attacks... Good ole #DDoS campaigns. Well, first of all, one should check with their hosting provider - whether they have the benefit of protections against such attacks, and then, weigh the added benefit of using something like Cloudflare to do the same job (are you paying for protection that you might need twice?).
I personally would probably not have included Cloudflare as part of the #HowTo. It can be added at anytime, but some folks swear by it, so it's not that I'm on the fence about Cloudflare, it's just that I look at it more from the engineering and security perspective, with an eye specifically focused on the veracity of any perceived needs by the customer. And I'm not super fond of turning all of that DNS control (and valuable #metadata) to some third party.
I realize that may have only served to raise more questions, so I'll just say that this is why you pay your trusted IT support professionals who make all of their money on labor they've billed you for, to sit down and discuss what you may or may not need, and especially, why 👍
Brenden Eich was invoked by @marathon - and I too, concur that It is only right to measure technology based on it's own merit and capability - without regard to superfluous and unrelated matters of personal politics.
When haters start fomenting hatred, disparaging everyday, average people for their informed choice of technologically capable software relevant to the task at hand, I like to remind those vile, adolescent, sniveling children that they're literally denigrating things like Brave Browser and Soapbox (the platform I'm authoring this post on), while at the same time availing themselves of the full compliment of features that #JavaScript's technology affords them - JavaScript, invented by#Brenden_Eich...
And they have my blessings to completely swear off and forgo ever using JavaScript again - but they won't, will they? Why? Because they're filthy, hateful, hypocrites consumed by their own criminal commiserations.
You can also see all of what I have to offer or check out my #blog on my janky little website run on a free #OracleCloud server and hosted on #CloudFlare
I don't understand why #Mozilla didn't switch to Quiche, a full QUIC + HTTP/3 stack entirely written in #Rust and actively maintained by #Cloudflare.
I picked Quiche in 2021 to implement DNS-over-HTTP3 (DoH3) on Android, and it was small and easy to embed into the existing C++ codebase of the Android DNS resolver.
Ich weiss, das zu mindestens #Okta mal unsicher war, doch wie sieht es bei den anderen aus und nutzen die auch die aktuellste #Technik für ihre #Dienstleistung'en? Das ist ja viel versprochen aber nicht garantiert, da Closedsource oder nicht?
«#Identity & #Access Management – Die 9 besten IAM-Tools:
Diese Identity-und-Access-Management (#IAM) -Tools schützen Ihre Unternehmens-Assets auf dem Weg in die Zero-Trust-Zukunft.»
🧵…[ENG] It's worse than previously assumed. Apart from how many #web services are dependent on it, this has a very big impact
#Cloudflare Breach: Nation-State #Hacker's Access Source Code and Internal Docs @cloudflare has revealed that it was the target of a likely nation-state #attack in which the threat actor leveraged stolen credentials to gain unauthorized #access to its #Atlassian server and ultimately access some documentation and a limited amount of #sourcecode
Trump Team Targets European Union for Punishing Trade Measures (www.bloomberg.com)