albertcardona, 1 month ago

@computingnature

The scores are close, is there any advantage to the losing approach which would make it desirable in practise ?

Also, fun to see the adversarial Carsen-Marius dynamic ….

computingnature, 1 month ago

@albertcardona :) the challenge authors updated the test set since the challenge so we had to submit our masks and the winning team's masks to get the scores.

The winning Mediar team used the Cellpose flow approach (our code) and used a transformer instead of a CNN for prediction. On this dataset, it doesn't seem like the transformer improves performance. We also tested transformers on an even larger dataset and did not find a performance boost (blue and black curves) (from https://www.biorxiv.org/content/10.1101/2024.02.10.579780v2):

jonny, 1 month ago

@computingnature
Turns out domain specific models still rock. Love 2 see it ♥

fabrice13, 1 month ago

@jonny @computingnature and it's not only the domain thing, the Transformers hype has a sort of mystical aspect with the mathematical properties of the attention mechanism, it's been 4 years I think of constant press treating vision transformers as the only SOTA models, with ConvNets of comparable size and compute (if not lesser, i.e. better) achieving comparable results, anyway one can put it and look at it.
The fact we call the dot-product similarity "attention mechanism" has done wonders