The criticism about voter relevance is fair, but I wonder if this is still useful for a different purpose: training your own eye for landing page design.
When I've tried to improve at visual design, the hardest part was developing taste. Comparing options side-by-side and articulating why one is better is exactly how you build that skill. The aggregate rankings matter less than the act of judging.
Have you considered adding a "why did you choose this one?" prompt to collect qualitative reasoning alongside the votes?
Interesting idea, but it seems likely that the data this collects may not be very useful as a way to estimate the effectiveness of landing pages.
If I don't belong to one of the markets that the landing page is trying to sell products or services to, I shouldn't be able to vote -- if I vote, I'm just polluting the dataset of what their actual potential customers think of it.
If the two landing pages being compared aren't targeting the same market of potential customers [+], it's unclear what comparing them achieves.
It's unclear if voting on a landing page in this kind of artificial setting is predictive of something more material like how often a potential customer proceeds to sign up for a demo, or pay for something, or call sales.
[+] or given a lot of these are AI landing pages, I guess they're targeting investors.
Reminds me of how rankings for high schools that are supposed to be objective always faithfully reproduce the wealth differences between neighborhoods. This is even true in situations where clusters of moderately well compensated high-intellect workers (of a national lab or research Park or similar) consistently rank worse than the old money neighborhoods in the same cities.
The criticism about voter relevance is fair, but I wonder if this is still useful for a different purpose: training your own eye for landing page design.
When I've tried to improve at visual design, the hardest part was developing taste. Comparing options side-by-side and articulating why one is better is exactly how you build that skill. The aggregate rankings matter less than the act of judging.
Have you considered adding a "why did you choose this one?" prompt to collect qualitative reasoning alongside the votes?
Interesting idea, but it seems likely that the data this collects may not be very useful as a way to estimate the effectiveness of landing pages.
If I don't belong to one of the markets that the landing page is trying to sell products or services to, I shouldn't be able to vote -- if I vote, I'm just polluting the dataset of what their actual potential customers think of it.
If the two landing pages being compared aren't targeting the same market of potential customers [+], it's unclear what comparing them achieves.
It's unclear if voting on a landing page in this kind of artificial setting is predictive of something more material like how often a potential customer proceeds to sign up for a demo, or pay for something, or call sales.
[+] or given a lot of these are AI landing pages, I guess they're targeting investors.
I like the concept but am suspicious when I see a disproportionate amount of top ranked landed page sites, have ad banner stating they are YC funded.
Reminds me of how rankings for high schools that are supposed to be objective always faithfully reproduce the wealth differences between neighborhoods. This is even true in situations where clusters of moderately well compensated high-intellect workers (of a national lab or research Park or similar) consistently rank worse than the old money neighborhoods in the same cities.