Recent events have spurred some great discussion of fediverse safety and the ways that we can improve it. I'm thrilled to see the growing interest in this problem, but there's one specific angle that concerns me. Specifically, the push for allow-list federation.
First, some background information. Federation between instances is usually controlled by restricting which remote instances (identified by domain name) can send or receive messages from the local instance. In the majority of fedi servers, this is implemented with a "deny list" - a list of instance domains that should be blocked. When using a deny list, all unlisted domains can communicate by default.
This is in contrast with an "allow list", which specifies the domains that should not be blocked. When using an allow list, all unlisted domains are blocked and can not communicate. Deny-list federation is open except for specific routes, and allow-list federation is closed except for specific routes. There are other approaches, but these are by far the most common.
Now, moving on to my actual point. I've recently seen some discussion of using allow-list federation as means to improve user safety. This idea, while very well-intended, is unfortunately flawed and could easily cause more harm than it prevents.
This idea suggests that by limiting federation to known, approved domains, then it becomes easier to block sources of abuse, hate, and illegal content. This is actually quite effective, and some instances are already using allow lists to great effect. They experience minimal contact with dangerous instances, and those that slip through are easy to identify and block. It's a great strategy for specific communities. Unfortunately, however, this approach cannot scale.
Allow-list federation works well for a minority of instances, but only because the majority do not use it. This is due to the network effect. Just as a centralized social network can collapse if there are too few connections between users, a federated social network can collapse if there are too few connections between nodes (instances).
With open or deny-list federation, all nodes are connected to all others. This forms a very strong network - so strong that small and even single-user nodes are possible. The open federation ensures that there are always sufficient connections to keep the network useful.
The same is not true of allow-list federation. In this mode, the network nodes are only connected to a limited subset of others. As the percentage of allow-list nodes increases, the number of connections decreases exponentially. The resulting network is weak and unstable enough to collapse under pressure. A bigger problem, however, is not the network weakness but rather the loss of small instances.
In my experience, it's rare to see a federation allow-list with more than 1,000 entries. Fediverse index websites currently show around 25,000 active instances, which means that most allow-list instances are connected with only 4% of others. We can assume that the 4% is biased towards medium-to-large instances because of their increased visibility. After all, an instance cannot allow-list another node until they've encountered it through a mutual. Larger instances have more mutuals, and thus a greater chance to be discovered.
Over time, this creates an imbalance where larger instances benefit from a richer and broader network than smaller nodes, who have limited reach and reduced federation. Any new instance must somehow maintain a user base while faced with network isolation. This becomes a significant barrier to entry for the fediverse.
This barrier effect is bad enough, but there's another angle to consider. Fedi is large enough to have a constant churn of small instances starting up and shutting down. Currently, more instances are forming than closing, which leads to positive growth in the number of nodes - a good thing for network health. But if it was harder to start a new instance? That ratio would drop or even reverse, causing the network to shrink instead of grow. Reduced network size increases the network effect, which expands the inequality, which strengthens the barrier, which decreases the growth, which reduces the network size. A classic feedback loop.
This process would be devastating for fedi. The cycle would continue until there's no new instances at all, by which point we'd have lost the rich ecosystem of tiny communities that make fedi unique. Large instances can never provide a truly safe space, and often foster a community much like Twitter - problems included. The small, diverse instances are what make this network special. If we lose that, then we lose the fediverse.
@hazelnoot hard agree, there are already so many barriers to setting up your own instance, technical, costs etc. putting up one more barrier of having to navigate whatever system gets you into the allow list exclusive club would make it even less appealing a prospect to people.
@HaruEb@hazelnoot allowlist would make some sense if there was some mechanism to notify admins and mods of a new instance for review (to check if it isn't dedicated spam/harassment/illegal material, etc). Or at least some way to add a new entry on request when an user sees a broken thread or boost. I'm running an allowlist instance on the side and it's honestly barely usable.
@affine@HaruEb separate but related to the above topic, I've been researching ways to compute "local trust" or reputation values for remote instances, users, and posts. This would allow for a hybrid approach, where anything "new" is held in limbo while the source is analyzed against some configurable set of rules. My current thinking is along these lines:
Every object from an unknown instance should be reviewed, but there should be an option to exclude "known" good / bad instances to reduce unnecessary reports.
I'm unsure whether local instances should be included.
All output of the trust algorithm should include a confidence value. For example, a keyword filter may return a "block" result for a post containing an unreclaimed slur. But if that's the only notable finding, then the result should be considered "low confidence" and placed at the top of a queue for moderator review. Maybe it's a bigot harassing minorities, but maybe it's a minority user attempting to reclaim the slur. That distinction can only be resolved by human staff.
Nothing should be blocked "silently". Even a high-confidence action should result in a queue entry for staff to review.
This must be a highly-adjustable algorithm. Instance staff must be able to adjust weights, heuristics, and actions. Staff must also be able to disable the default actions and emulate allow-list or deny-list federation.
To prevent email-style centralization, the weights and decisions must not be federated. Each instance needs to compute its own trust value for each other instance, according to local staff decisions and available metadata.
Staff should be able to see the current trust value for any known instance, user, or object (post).
Staff should have access to the exact indicators that led to a particular decision. This can't be a mysterious "closed box" algorithm making arbitrary decisions.
Staff must be allowed to override the algorithm! Human moderators will always know more than a computer.
Staff should also be able to "pin" a decision for an entire instance or user. The algorithm can keep computing trust values, but it must only take action based on the pinned decision.
I've got a lot more thoughts, but they'll have to wait until I've had more time to research and refine the framework. This is still just an idea bouncing around in my head.
Add comment