rwg,
@rwg@aoir.social avatar

I've said this before, but I will say it again: those of you doing research on the fediverse using #scraping techniques or #APIs, keep in mind that this place is not 1 thing, but 10s of thousands of communities, each with their own terms/codes of conduct. So just grabbing all the posts you can likely violates many communities' norms or terms.

1/2

rwg,
@rwg@aoir.social avatar

Moreover, if you release crawlers into the fedi, that translates to real costs for fedi admins who pay for the bandwidth your crawlers consume.

@rra and I are writing an article about these very issues.

2/2

  • All
  • Subscribed
  • Moderated
  • Favorites
  • random
  • magazineikmin
  • rosin
  • Youngstown
  • mdbf
  • ngwrru68w68
  • slotface
  • khanakhh
  • ethstaker
  • everett
  • kavyap
  • thenastyranch
  • DreamBathrooms
  • tacticalgear
  • osvaldo12
  • JUstTest
  • InstantRegret
  • Durango
  • cisconetworking
  • modclub
  • cubers
  • GTA5RPClips
  • tester
  • normalnudes
  • Leos
  • provamag3
  • megavids
  • anitta
  • lostlight
  • All magazines