Every time I post a link to my site on Mastodon, it locks up for like 30 seconds or so.
My theory is that because I have a quite a lot of followers (👋 hii!) the post hits a lot of instances and they all poll my site at the same time, effectively DDoS'ing it for a lil bit.
One day I'll run a packet capture and prove this theory…
@kushal with #varnishcache, UDS and abstract sockets on linux are the preferred way to access local resources (and across containers in the same pod) because, besides being more efficient and potentially more secure, their use avoids running out of ephemeral tcp ports. concrete examples are tls off-/onloading with #haproxy and recursive requests from varnishd to itself.
I enabled Brotli compression on the CDN which serves the main BBC websites (www.bbc.co.uk. www.bbc.com etc.) outside the UK this morning.
Over ~4 hours, we're seeing a mean of ~20% better compression (smaller responses) via Brotli & ~95% of responses being Brotli now.
I've not had time to look in detail at performance but there doesn't look to be a significant change (LMK if you see diferent!).
(the spikes are breaking news events linking to a large "live" pages) #Brotli#WebDev#BBC
@tdp_org removing a-e: is a weird kink of fastly, but yeah, their #varnishcache fork point is ancient. with current filters, we can easily do "gunzip br".
thank you for the interesting update!
Linux/devops/web question. Are there open alternatives to cloudflare's "tunnel thingy"?
Say that I want to host my website https://samuels.bitar.se on a raspberry pi at home but I don't want to expose my home ip, how would I solve this? I know I can do it with Cloudflare but I don't want to use Cloudflare. Can I setup like an nginx proxy on a VPS and do like an ssh tunnel or something? Or are there other solutions?
I learned abour zrok but I don't know if that will work.
@samuel if your content is (partly, semi) static, using a caching reverse proxy can help reduce bandwidth used on your origin uplink and improve "speed" (reduce latencies, increase bandwidth) for your users. some options are #apachetrafficserver#freenginx and #varnishcache, only to name a few. my recommendation should be obvious, but you have the choice and a lot of options.
as a pure (non caching) forwarding tcp or http proxy #haproxy is a good choice.
vmod-dynamic, our #varnishcache module for dynamic backends from #dns (A/CNAME and SRV records) has received some bug fixes and, in particular, workarounds specific to Varnish-Cache 7.4, for which a new https://github.com/nigoroll/libvmod-dynamic/tree/7.4 branch has been created.
I rebuilt my #varnish server tonight using an Ansible playbook I'm writing on an updated Debian 12 Triton image. While working on it, I noticed the log was going crazy. I checked the connections (netstat -nat | wc -l) and found there were >4300 open at that moment. This was just someone with a modest follower count boosting one of my posts.
Before Varnish, this would have taken my system down for 20 minutes or so. I'm curious how far the new service can go. /1
Who's using Apache Traffic Server as an HTTP reverse proxy in a large-ish scale way?
Interested to hear any opinions and/or loves/frustrations with it...Eyeing it up as a potential successor (one day, would be a chunk of work) to our NGINX-based in-house CDN. So that'd mean it'd go into 3+ datacentres, each configured as clustered machines, serving 10s of thousands to millions of RPS across 10s of domains as a reverse proxy. https://trafficserver.apache.org/
@bagder iirc it was two years ago that someone promised to implement http3 for #varnishcache "this year". they even had the features planned in a public repo.
we are all doing it wrong. #vaporware is the solution.
@selea layer4 (syn flood, file descriptor exhaustion): mostly a non issue nowadays because ram is cheap enough.
tls: rate limiting works (eg with #haproxy ) or techniques along the #fail2ban idea : if an ip hits you too hard, filter it efficiently in the kernel
http: here my best recommendations are all based around #varnishcache because i work on it, but alternatives do exist. i will focus on what i know to be most helpful. 🧵