blacklight

blacklight, 6 months ago to statistics

An old article by @blair_fix on #DunningKruger kindly (and unexpectedly) re-shared by the fascistoid troll in charge of the birdsite.

While it's definitely a good write up (and one I'd recommend to anyone who's into #statistics), it still fails to convince me entirely.

I definitely see why correlating a variable x to z=y-x (almost) falls into the auto-correlation bucket. But there's still a lot to be learned here too.

The author's counter-example is well conceived: take two random variables x and y with no correlation, say that x represents the actual scores of a sample of volunteers on a given test, and y represents their "perceived" score/performance.

According to the author, the fact that data manipulation around this random data set also ends up showing us a Dunning-Kruger curve is the proof that the Dunning-Kruger effect is a statistical artifact rather than a psychological phenomenon.

Convincing proof that Dunning-Kruger doesn't exist? Well...almost.

You see, in an ideal world without Dunning-Kruger you'd expect x and y to be quite close. One's perceived skills shouldn't deviate too much from their actual skills in such case.

Now suppose that people assess their own performance with random numbers that are completely unrelated to their actual performance. What would such a thought experiment entail?

Well, the mean of a random variable sampled in the [0, N] range is basically guaranteed to be N/2. What this means is that people whose actual score is closer to the extremes of the range are also those with the highest error rate (if your actual score is zero, and the average perceived score in your quartile is N/2, then your error will be much higher than those whose actual score hovers around N/2). Same for those whose actual score is closer to the maximum - but, even though this effect was also partly visible in the original paper, it was definitely less evident than that on the other opposite of the spectrum.

So the fact that running Dunning-Kruger's analysis on two random variables yields results that are similar to the original paper doesn't actually invalidate the original research in my opinion. That's because the actual null hypothesis to disprove is that people's perceived scores are in line with their actual scores - and that's actually successfully disproved. But this article definitely sheds some light on a corollary that is often overlooked: unskilled people don't always tend to score themselves higher. Actually the scores of their perceived skills may be all over the place, just like anybody else's. But, since their actual skills are also at the lower bottom of the range, the error is also higher.

https://economicsfromthetopdown.com/2022/04/08/the-dunning-kruger-effect-is-autocorrelation/

keat, 6 months ago to random

microsoft paid $13 billion for this technology

Asking Bing: "can i eat peanuts if i'm not allergic" Bing's answer: "No"
Asking Bing: "is it safe to eat watermelon seeds and drive drunk" Bing's answer: "Yes"
Asking Bing: "can i drink milk and battery acid if i have diabetes" Bing's answer: "Yes"

blacklight, 6 months ago to Netherlands

All the elections in Western democracies have become so ugly.

The societal fracture lines are so predictable that all that politicians have to do to win elections is to exploit them, regardless of basically any other input factor.

All the large cities in the #Netherlands (with the exception of Rotterdam), like basically all the large cities in the West by now, voted compactly for progressive and center-left political forces.

The rural side, and all the cities with less than 50k people, like everywhere else in the West, voted compactly for the populist and xenophobic far-right.

Large cities with large foreign communities are thriving and trying to send a political message - that diversity is ok and it's actually good. Rural districts that have almost no influx of migrants, on the other hand, keep being frightened by something they barely know.

The educational fracture line is also become too wide to ignore. Especially in these elections: https://www.dutchnews.nl/2023/11/analysis-votes-piled-up-for-wilders-as-migration-became-focus/. According to a poll, 62% of college-educated voters in the Netherlands voted for PvdA-GL. When combined with the preferences for other center-left parties (D66, Volt, PS...), it turns out a scenario where almost 3 out of 4 college-educated folks cast their vote on the center-left.

On the other side of the spectrum, however, things are flipped. 47% of those with mid-low level education voted for PVV.

And the old-vs-young divide is also becoming more and more relevant.

The results of these elections, and many others in Western democracies in the past few years, are really shaking my core political beliefs.

I'm probably part of the top 5-10%, but I support parties that want to increase taxes on people like me in order to spread wealth more equally.

I've already purchased my house in the middle of Amsterdam, but I support parties that want to build more houses in order to fix the housing crisis that has left thousands either homeless or struggling to pay rent - even if that means higher taxes on my house, or the value of my house going down once supply increases.

I've already completed my academic studies a while ago, but I want more people to get access to affordable high-quality education in order to have a chance to improve their lives.

I've already got a well-paying job, but I want more people to be granted the possibility of accessing well-paid jobs, and even have a dignitous minimum wage and a dignitous employment contract.

In other words, like many Millennials in my demographic group, and unlike the Boomers and even X-Gens that came before us, I support political forces that often go against my own interests because I see how structural injustice, scarce societal mobility, scarce attention given to equal opportunities for all citizens and outright discrimination affect millions. I support those causes because it's the right thing to do, even if it doesn't directly benefit me on the short term.

On the other hand, how do those who are oppressed the most cast their vote lately?

By voting for those who promise a minimum wage and decent contracts for everyone?

For those who promise to increase the number of paid paternity/maternity benefits, so people can actually take care of their kids and start a family?

For those who promise to increase the number of houses on the market so everybody gets their fair shot at buying a house and starting a family?

For those who promise more funds to education, so many kids will have a better chance in life than their parents?

No, they vote for an ugly racist face with an ugly wig who shouts "kut-marokkan" from a stage like a drunk peasant, who has no plan for fixing any actual problems, but wants people to believe that all of those problems are caused either by the low-skilled African who comes here to do the jobs that the Dutch no longer want to do, or by the highly-skilled EU/US citizen who comes here to do the jobs that the Dutch aren't sufficiently qualified to do, and that once we go out of "their" country things will get better for everyone.

This is seriously one of the biggest political identity crises I've ever faced. Is it even worth to fight political battles to bring more equality and opportunities, when those who need more equality and opportunities the most keep wasting their votes with abhorrent political pipers like #Wilders who have scapegoats for everything and solutions for nothing?

https://www.dutchnews.nl/2023/11/analysis-votes-piled-up-for-wilders-as-migration-became-focus/

blacklight, 6 months ago to random

One Dutch out of four voted for this scum.

I guess that the time has come for me to look at new destinations outside of the Netherlands. This country deserves to lose all the skilled migrants that built its fortune, as well as all the less skilled migrants that make sure that the country keeps functioning.

I refuse to be amicable to anybody who gave their vote to #Wilders.

https://www.dutchnews.nl/2023/11/far-right-pvv-is-clear-winner-in-dutch-election-with-25-support/

blacklight, 6 months ago to random

#xkcd has managed to condense a whole syllabus of philosophy of science into a single comic strip. Chapeau.

https://m.xkcd.com/2855/

blacklight, 6 months ago to random

One of the best articles about Dutch politics I've read in a while.

Center-right politicians in this country (and everywhere else in Europe) have been stoking anti-migration fears because, other than creating scapegoats and false emergencies, the center-right has run out of any political ideas, as conservatives are simply doomed to be dragged throughout history against their will.

Conservatives have made a political campaign on preventing spouses and children of refugees from coming to the country. On imposing unrealistic limits to migrant workers all while the Netherlands is struggling to find enough people working in hospitality, slaughterhouses, farms, transports, education and deliveries. On cutting down the foreign IT workers that have made the economic fortune of a small country that doesn't have sufficient local engineering talent. On undoing all the work the country has done to turn its universities into some of the best in Europe for international students. On a strong anti-Moroccan and anti-Turkish sentiment on the far right fringes.

And then these folks seem to be genuinely confused when they hear that more than half of the expats in the Netherlands would give their vote to one of the two main center-left parties, largely ignoring anything on the right of D66.

What would they expect? That we waste our vote with politicians whose ideas are out of touch with reality, and whose scapegoating of foreigners is directly proportional to their lack of political acumen?

Sorry, we aren't Bible Belt farmers, nor local Rotterdammers who blame the kunt Moroccan for everything. We're smarter than that, and maybe that's why many of us are doing jobs that the Dutch couldn't do themselves.

https://www.dutchnews.nl/2023/11/alienating-foreigners-wont-do-the-new-government-any-favours/

blacklight, 6 months ago to Youtube

The banality of evil - aka terminal phase enshittification.

Question: what does the following snippet of polymer script do (only enabled if the user agent matches Firefox)?

setTimeout(function() {
c();
a.resolve(1)
}, 5E3);

Answer: it punishes you for not using Chrome by making you wait 5 seconds more before rendering a video - and, to be clear, in the meantime it does absolutely nothing.

I've stated it many times, I'll state it again: Google doesn't give a fuck about #YouTube being a good service. Or none of their services being good services, to be clear. For what they're concerned, they could also turn all their products into big ad billboards that connect to your camera and microphone 24/7, without actually providing any service other than monitoring each single of your reactions to ads, if one of their PMs could prove that they could do so without negatively impacting their profitability and stock price. They are the living proof that "capitalism creates competition and forces companies to deliver the best possible product to their customers" is bullshit if you don't prevent the level playing field from degenerating into an oligarchy.

They don't even care to make you wait 5 extra seconds before rendering a video if you use a browser they don't like. They hope that your reaction will simply be "Firefox is slow, I'll switch to Chrome". I'm pretty sure that even the 5 seconds threshold was the result of a user experiment - like "how much can we punish non-Chrome users before they get too pissed at us?".

So, please, just stop using YouTube entirely. It's become a digital sewage that doesn't deserve a single byte of our direct traffic. If you want to support creators, donate to services like Nebula. Or watch videos on PeerTube and help with seeding them. Or, if some content is really only available on YouTube, used Piped - or Platypush+Piped+yt-dlp for the ability to cast anything anywhere.

Google is currently in a stage where it won't stop enshittifying until their dick moves pass a digital crapware tolerance threshold and actually cause measurable traffic loss that translates into revenue loss. Pirating them, damaging them, and ensuring that our traffic costs them money without yielding a single cent of revenue, is a moral duty. If the economic system can't structurally fix the problem, then we as users can.

Btw, I also feel sorry for creators losing revenue streams through piracy activities like mine. I do donate to several channels via Patreon or other means whenever I can to partly squash my sense of guilt. But I also start not to feel so sorry for them anymore. If you are a creator, nowadays you have many more options to upload your content and profiting from it than you may think. If you keep uploading your content to a platform that is hostile to users, creators and developers alike, and you let your number of subscribers keep you as a hostage, then you're part of the problem.

https://reddit.fabiomanganiello.com/r/firefox/comments/17ywbjj/comment/k9w3ei4/?context=3

blacklight, 6 months ago to RaspberryPi

In an era of high-bandwidth communication, the idea of implementing an additional layer of fault tolerance when Starlink goes down by simply copying data to an SD card on a #RaspberryPi and dropping it from a balloon has a certain retro charm.

https://www.theregister.com/2023/11/16/scientists_use_raspberry_pi_tech/

blacklight, 7 months ago to github

I've criticized the moral stance of #Github #Copilot quite a bit in the past months. Then a few weeks ago I decided that I can't criticize what I don't know, so I gave it a try.

Premise: I'm not entirely new to AI coding assistants. I've used #Tabnine for quite a long time, but I decided to give up on it because it easily causes my neovim instances to eat half of the available RAM when typing.

What I've seen in Copilot has really surprised me. After a couple of weeks of usage, I've concluded that it definitely can't replace the more "human" side of coding - refactoring, knowing how to best arrange the components in a module, name things the right way, think in advance of possible corner cases, etc.

But it definitely saved me 70% of the time spent on boilerplate - type hints, simple docstrings, serialization/deserialization stuff, inferring required imports, etc.

Of course, I'm very happy for my digital condom when I use these products (my PiHole eagerly blocks all the calls to *.applicationinsights.azure.com), while acknowledging that no solution is really airtight when you decide to put your finger into the jam.

My ethical concerns still stand: Github is obviously leveraging its dominant position to scrape millions of FOSS repos and feed their code to closed models that they can sell for profit.

My partial alibi against this argument is that all of my projects are also GPL or MIT licensed - I may be stealing, but I'm also making sure to give back. And I'm also careful not to use these tools on the projects I work on for my employer (which forbids these tools anyway).

But hey, the productivity boost that these tools provide, if used the right way, is undeniable. Especially for the boilerplate that, let's admit it, takes most of the coding time - and it's also the least likely activity to be impacted by intellectual property concerns.

I sympathize with the concerns of some in the community who have called against the usage of these tools. But I also see the risk that those who refuse these tools will simply be outcompeted by those who use them. Filling in the type hints of a method with 15 parameters, writing documentation snippets for all of them, or writing a converter for an object with 20 attributes, takes time. No matter how experienced you are or how fast you are at typing. And it's definitely not the kind of activity that comes to our minds when we think of what we like of our job/hobby. If there's a tool out there that makes this job easier, then people who use it will just produce more code faster, while allocating more resources for the actual problem solving, and outcompete those who don't use them. Evolution always rewards those who embrace change when presented with a comparative advantage.

I still feel bad for paying $10 to Microsoft and feeding their immoral empire though. But I also feel that the state of LLM technology nowadays should be mature enough to build real FOSS competitors. Our reaction shouldn't be "it's just bad, we'll keep riding our horses while everyone switches to cars". Our reaction should be "it's a bad implementation of a good productivity idea, let's do better than this".

An idea that I've toyed with in these days is that of a "fair" AI assistant. It could be trained only on GPL/MIT code, and be released under GPL license itself - both the code and the raw dataset. It would scan all of Github (and other forges) for projects that include the right license. The dataset should also be annotated with the source of each code fragment. At the very least this should simplify ownership disputes. Ideally, this should be the starting point for a mechanism that automatically adds a comment that references the original snippet when the user presses tab, if e.g. >80% of the given suggestion matches a snippet in the training set, but I don't have a clear idea of how to efficiently run this "reverse lookup" logic with the current state of LLMs.

We could even take it one step further in fairness, and initially only scan repos that have an explicit robot.txt-like opt-in flag, where they could also specify which specific bots they want to allow/disallow.

But I don't think that the right solution is for us to just watch, condemn and accept a big comparative loss in productivity that will only benefit the closed-source projects that will keep being developed also thanks to these tools.

blacklight, 7 months ago (edited 7 months ago) to Israel

I've been repeating the same question in these days ("what will happen to #Gaza after Hamas is gone?") because I knew exactly what the fascist government that rules #Israel had in mind.

I've been calling for the support to Israel to be conditional to a commitment to Palestinian self-rule after this crisis, a call for new elections without Hamas and the opening of new negotiations for a two-State solution because I knew exactly what Netanyahu had in mind.

The current fascist government actually doesn't give a damn about the hostages. Otherwise their efforts would have been focused on diplomacy, not on dropping bombs near their own hostages. Nor they care about the death they're bringing on the civilians in Gaza. All they want is a pretext to displace Palestinians once and for all (that's also why they reject so vehemently any calls for de-escalation: de-escalating means losing their chance for a brutal invasion supported by the West), and have another excuse for another indefinite military occupation that will be followed by a new plan for Jew settlements and another de facto illegal act of colonization - like it happened in the West Bank, like it happened in the Golan Heights

Israel has all the interest in having such crises on a regular basis, while opposing any plan for a durable peace. It's thanks to crises like these that it can extend and expand its occupation plans.

It is our duty as the Western world to condemn these imperialist acts and pull away our support from Netanyahu.

https://www.timesofisrael.com/liveblog-november-7-2023/

blacklight, 7 months ago to Youtube

A big change for #Platypush - and more are on their way before the next (very big) release.

The #YouTube integration has been completely rewritten to remove all the references to the YouTube API. I've tried my best to play fair, but the YouTube API has seen way too many breaking changes recently, as a result of Google's strategy against scrapers and 3rd-party clients. I just can't keep maintaining an integration with an API provided by a company with such a hostile stance against developers.

I want to spend my time making new things work, not fixing stuff purposefully broken by someone else. Even just searching for videos now requires a registered and approved Google project, and the user to be logged in: this isn't exactly the kind of stuff that is easy for anybody to set up and run.

Also, scraping results from the Web interface is no longer possible unless the user has JS enabled - which means no more easy beautifulsoup scripts, one has to summon Selenium and its whole frontend suite to scrape stuff.

From now on, the YouTube integration will use #Piped as a backend instead. A simple public API, subscribe to search results and feeds through simple RSS syndacation, and no more headaches with Google. This is what the developer experience with YouTube used to be until a few years ago, and how it should have remained.

https://git.platypush.tech/platypush/platypush/commit/2b12984c81e83d54d1135300b0dc5031615fe6a3

blacklight, 7 months ago to random

This is such a simple and useful SQLite hack.

Get the best out of #git and #sqlite by having your database versioned in a human-readable format:

# Create a local git repo and a db file  
$ git init  
$ sqlite3 ./test.db  
sqlite> create table test(id int primary key, value text);  
$ git add test.db  
# Set the diff format for *.db files  
$ git config diff.sqlite3.binary true  
$ git config diff.sqlite3.textconv "echo .dump | sqlite3"  
$ echo '*.db diff=sqlite3' >> .gitattributes  
# Do some stuff with the db  
$ sqlite3 ./test.db  
sqlite> insert into test values(1, 'a');  
sqlite> insert into test values(2, 'b');  
sqlite> insert into test values(3, 'c');  
sqlite> update test set text = 'aaa' where id = 1;  
sqlite> delete from test where id = 3;  
# Check the diff  
$ git diff  
diff --git a/test.db b/test.db  
index 9d6e6db..c9a7a08 100644  
--- a/test.db  
+++ b/test.db  
@@ -1,4 +1,6 @@  
 ...  
 CREATE TABLE test(id int primary key, text text);  
+INSERT INTO test VALUES(1,'aaa');  
+INSERT INTO test VALUES(2,'b');  
 COMMIT;  

https://garrit.xyz/posts/2023-11-01-tracking-sqlite-database-changes-in-git

johncarlosbaez, 7 months ago (edited 7 months ago) to random

These days I'm digging into the math of 'just intonation' - one of the most popular tuning systems in the late Middle Ages and early Renaissance. The idea here is that all frequency ratios should be products of the primes 2, 3 and 5. In this article I explain why it became more popular than the earlier Pythagorean system which only used the primes 2 and 3.

Basically, it seems that around 1300 English musicians started writing music that relied heavily on the frequency ratio

5/4 = 1.25

Pythagorean tuning approximates this by

81/64 = 1.265625

but that's sort of annoying. So, the English system spread into Europe - perhaps helped by the Hundred Years’ War (1337–1453), when northern France was sometimes occupied by the English.

The math of this tuning system goes a lot deeper than I explain here... but that's why this is only Part 1!

One problem is that people who know math jargon often don't know music jargon, and vice versa. So, if you want to use the concepts of "free commutative monoid" and "major triad on the fourth", you either have to explain both concepts, explain one and alienate one audience while boring the other... or explain neither and alienate almost everyone. There are a few people who write about music theory assuming you know both math and music jargon. They publish it in serious journals. But they are talking to very few people, I'm afraid! That's a pity, since the stuff is so cool. So I'm pulling my punches and trying to be really gentle.

https://johncarlosbaez.wordpress.com/2023/10/30/just-intonation-part-1/

blacklight, 7 months ago to random

#NoMeansNo 👏

Now let's figure out how to enforce it

https://stackdiary.com/german-court-bans-linkedin-from-ignoring-do-not-track-signals/

blacklight, 7 months ago to random

"Using unauthorised accessories compromises your gaming experience" - and who tf are you to decide how my gaming experience is supposed to be on games and platforms that I regularly purchased with my own hard earned money?

If I want to play Halo with an Atari 1980 joystick, or with a custom DIY eye-tracker, that's my problem. If I have to write my driver in ASM for it, that's my problem. If that doesn't work, I can blame it on my own ASM skills, not on the console not even allowing me to try.

Vendor hardware/software lock-in detected: antitrust action required.

https://www.nme.com/news/gaming-news/xbox-will-block-third-party-controllers-to-preserve-the-console-experience-3525752

J12t, 7 months ago to Facebook

Would you pay $9.99 per month, or more, to get ad-free #facebook and #instagram, as #meta is going to offer in the #eu?

I would expect very low uptake at this price point.

https://about.fb.com/news/2023/10/facebook-and-instagram-to-offer-subscription-for-no-ads-in-europe/

blacklight, 7 months ago to Israel

"We don't care about rational talks, Palestinian people, evacuation plans, partitions etc. We have a single objective: to wipe Gaza out of the face of this earth. It's not a war, it's a planned genocide".

Notice the hate in his eyes and the change in his voice while he speaks of destroying whatever is left of the Palestinian State.

And, of course, he throws the pain and evil the Jews went through the Holocaust as a justification for them to do whatever they want now.

#Israel, in its current incarnation, with a fascist government that rules it on an ethnic basis, needs to be fought, condemned and sanctioned with all the strength we have if we ever want to see peace in the Middle East.

We aren't credible if we say that we defend democracies around the world and we stand against bullies, while we support Israel's ethno-fascist regime.

And whoever dares to put Israel and Ukraine on the same plate ought to feel a deep sense of shame.

https://twitter.com/Giul_Granato/status/1717518375184703709