box464, 4 months ago

@evan Don’t forget about oEmbed data as well (embedded players for all kinds of things - peertube, funkwhale, etc). It’s currently pulled as part of the link preview generator logic. Many times coming through from the Twitter:player meta tag if no oEmbed exists.

https://box464.com/posts/mastodon-preview-cards/

renchap, 4 months ago

@evan 👀 interested in the replies, as we are thinking about how to federate link previews to avoid having each instance fetch it (and DDOS the link’s server)

evan, 4 months ago

@renchap I'll definitely write up a proposal for it.

benpate, 4 months ago

@evan Here’s my take, hope it helps?

https://github.com/benpate/sherlock

Sherlock is a #Golang library that assembles any data/metadata it can find on a URL (including #WebFinger, #RSS, #OpenGraph, and #IndieWeb #MicroFormats ) and returns an #ActivityStream back to its caller. There’s composable add-ons for caching and other custom rules.

Overall, mapping to ActivityStreams was pretty easy. Sherlock is the key component in #Emissary that helps it participate in many different social webs.

evan, 4 months ago

@benpate nice!

smallcircles, 4 months ago

@evan @benpate @trwnh

There's a #FediverseIdea on this topic that you might add comments / new insights to:

https://codeberg.org/fediverse/fediverse-ideas/issues/26

trwnh, 4 months ago

@evan hot take, why not just use the opengraph namespace/iris directly?

if the goal is to reduce duplication of data, i'd probably recommend the following

• og:title = as:name
• og:type = og:type (no way to map this one)
• og:url = as:url
• og:image = as:image

for optional ones:

• og:description = as:summary

all the other ones i'd just use ogp.me iris directly

evan, 4 months ago

@trwnh that's about what I'd expect as a mapping. Have you seen it used to create Page objects, for link previews?

trwnh, 4 months ago

@evan i have only ever seen link previews implemented out-of-band due to trust issues :/

evan, 4 months ago

@trwnh you mean, on the receiving servers?

trwnh, 4 months ago

@evan yes, and it creates a sort of "thundering herd" unintentional ddos for web sites on underpowered providers. there have been ideas on how to solve the trust issue, like randomly verifying some small subset and tracking reputation, or like deferring to a semi-centralized pre-trusted "link preview provider" that is shared between several instances, or even simply ignoring the issue and depending on people to report misleading links. (i prefer the provider approach)

evan, 4 months ago

@trwnh another option is accepting the sender-provided metadata to start, and then verifying on a schedule that recognizes that thundering herd problem -- say, a randomized delay between 1 and 300 seconds, with an exponential backoff. And I definitely think having a trusted shared caching server can help immensely.

thisismissem, 4 months ago

@evan @trwnh I'm in favour of third-party providers that give a standardised API for going from link URL to Page or similar representation & provides hashes of the returned object allowing easy verification with the provider

evan, 4 months ago

@thisismissem @trwnh doesn't have to be either-or. Can be also-and. Use sender-provided metadata, and possibly re-verify with a trusted API.