I am thinking of breaking #Postgres and #ElasticSearch to a separate server instead of increasing my current VPS resources.
I'll have two servers. My main, which will run Mastodon, and another that will have Postgres and ElasticSearch on it.
Will I benefit by doing this or should I just move ElasticSearch to its own server? What is the minimum CPU/ram I can get for the second server & what should I get?
Behind search for masto is a resource-hungry #java application called #elasticSearch. Sure you might be able to get Masto to run on a #raspberrypi4, but you won't be running elastic search with it.
Having recently moved my instance to a much more powerful system, I now run ES, and WOW what a difference being able to search post makes! Soooo many times I wanted to reference a toot that had scrolled by but had no way of finding it. Now I do!
Even though I've put some serious resource limits on ES, it's bar none the most resource intensive service running (out of around 25).
Please consider sending a few bucks to the #MastodonAdmin of your instance. Better yet, if you can afford it, sign up for a monthly donation. They need it.
ES|QL is a great new query language for #Elasticsearch, and my team is building integrations into the client libraries. Here's an overview of our #PHP integration that lets you turn your results into PHP objects.
After reading a massive tome about #ElasticSearch earlier this week I realised it was complete overkill and just used the full-text capabilities of #PostgreSQL instead.
Currently PieFed has 46,000 posts and results are fast. It'll be interesting to see how well it copes when there are more posts. Anyone want to make a guess when it'll bog down?
Great to see more people catching on to ClickHouseDB. We’re using ClickHouse at @honeybadger to power our upcoming logging/observability tool (Honeybadger Insights).
We’re also benchmarking a replacement backend for #Elasticsearch. Looks like quite a performance gain so far!
Will hopefully have more to share soon, but in the meantime we discussed this on the latest episode of @FounderQuest. Give it a listen:
I'm planning to write an updated homelab guide on my blog this year but I think I'm about to rebuild some parts for a new purpose 😅
It might be time to try out OpenCTI given what I do in my lab should be representative of what I do during < dayjob >. That also means I need to tear down Wazuh and configure an ELK stack instead (resource constraint). #Homelab#ELK#CTI#ThreatIntel#elasticsearch
Q for people who have used Elastic Search, esp. for a Mastodon instance: how should I configure it to use less memory (while still having enough)? Right now it seems to eat as much as it wants (~4 GB)...
It's a single-user instance, so the total data size it reports is 40 MB now.
Delighted to let everyone know that https://fediverse.au has been successfully upgraded to 4.2.5 of #Mastodon software (thank you, Claire!).
Additionally, #ElasticSearch has been installed for a better user experience.
Fediverse AU is exclusively for #research organisations and #university schools, centres and #faculties in the Australasian region - as most Mastodon instances don't allow these sorts of corporate accounts.
Please let me know if you experience any issues or glitches.
So, apparently I have ElasticSearch now? I had to hard reset my server in the middle of indexing because it ran out of memory and came to a complete halt. (See https://ma.fellr.net/@fell/111375959444602138)
It seems to be working though? I don't care if it hasn't indexed everything as long as it indexes new posts.
Remember: if your server has the new Search enabled, you should decide if you want to be included in it or not (Settings > Public Profile > Privacy & Reach) and tick (or not) the “Include public posts in search results” box.
It’s disabled by default so please enable it to make Mastodon more searchable and useful!! #MastoTip#ElasticSearch
I took some time to work out the exact #formatting that #Elasticsearch demands. If you think this might be useful, go ahead and boost. If you find any errors or would like to suggest additions, please also post a reply!
Hopefully this can help prevent a lot of frustration and trouble people might encounter and give them a better Mastodon experience!
Well, I finally got myself up to #Mastodon v4.1.9, and now to begin the v4.2.0 upgrade. Not going to bother with #Elasticsearch setup for now, though would probably go for #OpenSearch probably anyways.
I'm currently running the Elasticsearch update after upgrading to Mastodon 4.2.0, and it was running really fast until it got to "PublicStatusesIndex" — now it's still importing documents, but it's really slow. It says 406 docs/s, w/ 54M to go, and the ETA keeps getting longer. Anyone else experience this?