Rating Reform [Yes. Another Request.]

[homes32]

Background:
Despite numerous attempts to "Fix" the current rating system it is still horribly broken. Average ratings for the entire site sit around 4.8 and players cry and get pissy over a rating of 3 (Which according to the CC Rules is Average - The definition of average being: "a level that is typical of a group, class, or series : a middle point between extremes" AKA - A normal player) for a game played like any normal player would expect. Its quite obvious from looking at ranks and player attitudes that the 5 star system is rarely used as it was intended and provides little indication of the players actual quality level. Being an old skool player from back in 2007 I understand why the old Feedback system was dropped and am not asking for its return. The tags greatly improve on the original incarnation of Ratings but they don't get much attention next to the stars. Remember, the Feedback system was not replaced for its simplicity (Good/Bad/Neutral), but for its open comment system that was becoming a chore for moderators to keep up with the flame fests.

Concise description:
Simplify the Rating system by dropping the 5 stars in favor of a return to the Positive/Negative approach keeping the tags as an indication of why the rating is +/-

Specifics/Details:
Simplicity is the key here. There is obviously ambiguity in how the 5 star approach should be used and it is crippling the rating system. Some use it as the site originally intended (and often get flamed for doing so), many give automatic 5 stars to everyone, and many don't even bother rating players anymore because they don't want to listen to the whining about giving someone 3-4 stars. Different approaches skew the rankings and there is simply too much disagreement on how to use the 5 star scale. Looking at current player ratings across the site the majority are already using it as a Positive (5 stars)/Negative (1 star) nothing in between (2-4 stars) so this really isn't as big a change to the system as one might imagine.

A positive/negative approach relieves the disagreements on what star ratings do/should stand for and should lead to more accurate player ratings.

Example:
Thumbs Up - I would play with this person again
Thumbs Down - I will avoid games with this player
Neutral - Doh.

Tags: Same as current tag system..

How this will benefit the site and/or other comments:

Strives to make player ratings more accurate and useful as there is no ambiguity as to what the rating should mean on a scale. You either had a good experience or you didn't.
Combines the current tag system with a +/- approach provides the user a way to define why the rating was positive or negative without regressing to the pitfalls of the old style feedback comments.
No increased workload for moderators. A-Holes are A-Holes and there will always be a few lurking about whether or not they give a "1 star" or a "thumbs down"


edit: corrected minor spelling and grammar

[sempaispellcheck]

I support this idea, and I find the OP's argument well-reasoned.

sempai

[Foxglove]

I like this a lot!

[BGtheBrain]

*****

[betiko]

I personally think that the ratings system is rather accurate the way it is.

5.0 mother theresa
4.9 queen of england
4.8 michelle obama
4.7 victoria beckham
4.6 britney spears
4.5 miley cyrus
4.4 sasha grey
4.3 melissa mccarthy
4.2 and below: Margaret Thatcher

[Fewnix]

I like the thumbs up thumbs down approach

Yes le'ts keep it simple. but a question Could thumbs up or thumbs down either cumulative or your own ratgnh, show up next to the player id?

Lets say I am not happy with Fred some game I played.its not a question of foe'ing him, just some unhappiness with style of play I give him a thumbs down,. Some time later I am thinking of joining a game and I see Fred is one of the player already signed up-next to him is a thumbs down. I Useful intel in deciding whether or not to join that game.

On the positive side I find a lot of Cc'ers I am happy to play with and I would give lots of thumbs up. Then when I am thinking of joinig a game I can lean to games where players already signed up have a thumbs up.
-----------------.
For a cumulative rating how about listing the total number of thumbs up , the total number of thumbs down and the nxt score-.e.g.Fred got 200 thumbs up, 50 thumbs down, net raring 150 thumb Some may think twice about joining that game, for others no problem..


.

[deathcomesrippin]

I personally think that the ratings system is rather accurate the way it is.

5.0 mother theresa
4.9 queen of england
4.8 michelle obama
4.7 victoria beckham
4.6 britney spears
4.5 miley cyrus
4.4 sasha grey
4.3 melissa mccarthy
4.2 and below: Margaret Thatcher

Why is Sasha Grey so low? That girl will do anything on camera.

I support the idea o fa Ratings revamp, I always hated this new Ratings system.
EDIT: I would make it a button right beside the player in game, just click the thumbs up or thumbs down while you are playing. No picking stars just a quick thumb.

[dakky21]

EDIT: I would make it a button right beside the player in game, just click the thumbs up or thumbs down while you are playing. No picking stars just a quick thumb.

quick thumb FTW !!! greatest idea so far !!!

[macbone]

Betiko's dead on the money. We might not use the rating system like it's intended, but there's still a clear difference between a 4.5 rated player and a 4.7. =)

[Metsfanmax]

The only proposed approach that has any chance to fix inflation of the system is automatic neutral ratings by default (as in the one sitting in Submitted).

[IcePack]

The only proposed approach that has any chance to fix inflation of the system is automatic neutral ratings by default (as in the one sitting in Submitted).

.....according to you.
-----------------------------------
I tend to like this idea, would the proposal to show total thumbs up and thumbs down (like a count 100 up 32 down) or is it some sort of % shown, or just strictly "this player overall is a thumbs up or overall a thumbs down" ?

[Metsfanmax]

The only proposed approach that has any chance to fix inflation of the system is automatic neutral ratings by default (as in the one sitting in Submitted).

.....according to you.

Do you have an example of any other proposed system that could fix it? You might not have been around during the earlier part of the site when we had feedback, but the tendency for people to give way more positive feedback than neutral or negative feedback still existed.

[betiko]

The only proposed approach that has any chance to fix inflation of the system is automatic neutral ratings by default (as in the one sitting in Submitted).

.....according to you.

Do you have an example of any other proposed system that could fix it? You might not have been around during the earlier part of the site when we had feedback, but the tendency for people to give way more positive feedback than neutral or negative feedback still existed.

What exactly do you want to fix? Are the ratings you see all around the site undeserved, when you know the real range (from 5.0 to 4.0).
People with time rate accordingly with this real scale.

The ratings system is perfect the way it is, if you touch anything you will ruin it.


Here is the only thing that should be done: a tag count. Put the top tags left on a given player.

[Metsfanmax]

The only proposed approach that has any chance to fix inflation of the system is automatic neutral ratings by default (as in the one sitting in Submitted).

.....according to you.

Do you have an example of any other proposed system that could fix it? You might not have been around during the earlier part of the site when we had feedback, but the tendency for people to give way more positive feedback than neutral or negative feedback still existed.

What exactly do you want to fix? Are the ratings you see all around the site undeserved, when you know the real range (from 5.0 to 4.0).
People with time rate accordingly with this real scale.

I have refused to rate on this scale for years now, because if I rate what the ratings actually demand (a 3 for average, most of the time) then people will think I'm a jerk.

It does achieve some of the desired purpose, in the sense that you can tell a 4.0 from a 5.0. But I see no evidence for the claim that people rated 4.9 are somehow consistently distinguishable from people rated 4.8, even though a huge number of people fall between these two places.

[betiko]

The only proposed approach that has any chance to fix inflation of the system is automatic neutral ratings by default (as in the one sitting in Submitted).

.....according to you.

Do you have an example of any other proposed system that could fix it? You might not have been around during the earlier part of the site when we had feedback, but the tendency for people to give way more positive feedback than neutral or negative feedback still existed.

What exactly do you want to fix? Are the ratings you see all around the site undeserved, when you know the real range (from 5.0 to 4.0).
People with time rate accordingly with this real scale.

I have refused to rate on this scale for years now, because if I rate what the ratings actually demand (a 3 for average, most of the time) then people will think I'm a jerk.

It does achieve some of the desired purpose, in the sense that you can tell a 4.0 from a 5.0. But I see no evidence for the claim that people rated 4.9 are somehow consistently distinguishable from people rated 4.8, even though a huge number of people fall between these two places.

there is a little but what it tells you, is that those players are "community approved". There is almost no chance a 4.9 with 100% attendance is going to screw up a game for fun.
When I rate, I give 5s (which would be your thumbs up) it the guy was nice/good opponent, I most often don't rate when the guy was normal but if I do, I give 5s).
Otherwise, if I give 3s for example is that I didn't like the guy for minor stuff in a game and I want to screw a bit his rate. I give 1s if the guy was a total jerk.

you just know how it goes: if you don't rate full 5s you are screwing the guy you are rating, so if you have nothing against him don't rate him if it's not to give full 5s.

[homes32]

Betiko's dead on the money. We might not use the rating system like it's intended, but there's still a clear difference between a 4.5 rated player and a 4.7. =)

Clear to who? By his own admission people don't catch onto the "current system" until they have played awhile. Usually as a result of being labeled a jerk or having enough people whine and complain that they were given a "bad" rating. You shouldn't need to "catch on" to any thing. A rating system should be quick, simple, and able to provide useful information without having to know the formula that differs from what the site rules actually say.

[betiko]

Betiko's dead on the money. We might not use the rating system like it's intended, but there's still a clear difference between a 4.5 rated player and a 4.7. =)

Clear to who? By his own admission people don't catch onto the "current system" until they have played awhile. Usually as a result of being labeled a jerk or having enough people whine and complain that they were given a "bad" rating. You shouldn't need to "catch on" to any thing. A rating system should be quick, simple, and able to provide useful information without having to know the formula that differs from what the site rules actually say.

When i was new to the site, i remember i used to rate to get a first medal and i saw that people had roughly rates between 5.0 and 4.0, so i kind of rated accordingly. Also, people can be tempted to give full 5s to get full 5s back. I don t see the problem, it means that both enjoyed the game together.
The only thing that we would need would be a short introduction in the rules with common etiquette.

[Lord Arioch]

Why not delete all ratings. Explain it better and put a non rating stop on new players until the 100th game? ... 3 should be the most common value on us all, shouldnt it?

[betiko]

Why not delete all ratings. Explain it better and put a non rating stop on new players until the 100th game? ... 3 should be the most common value on us all, shouldnt it?

again, the ratings make sense. the rates people have are well deserved. remove them and something that took years to give rates to people will be flushed down the toilets.

[stahrgazer]

When i was new to the site, i remember i used to rate to get a first medal and i saw that people had roughly rates between 5.0 and 4.0, so i kind of rated accordingly..

When I was new to the site, I saw that "3" meant "average" and if that's the case, most players should fall around 3, with some standard deviation.

Ratings don't fall into that, therefore your statement:

again, the ratings make sense...

is false.

To make sense, "average" should be around the middle of where the ratings fall.

[king achilles]

Just don't forget that there might come a time when someone gives you a low rating or a 'thumbs down' when you feel you don't deserve it. Having just two ratings to choose from means either it's a positive or a negative. When people get a negative rating, they will also likely take it to the extreme and might want that person banned immediately and say the ratings system is flawed.

[macbone]

As stated before, CC's rating system is more like Amazon's or Ebay's. At my job, we're all evaluated. 5 is excellent, 4 is good, 3 is average, and so forth. Most of us score something like a 4.6 out of 5. If we dip below 4.2 (which is still better than Good), we're actually under quite a bit of pressure to improve.

But you know, a thumbs up/thumbs down would probably result in the kind of store ratings you see in Amazon. Here's a page of people selling used copies of the Dark Knight: http://www.amazon.com/gp/offer-listing/ ... 1402971164

The store ratings are
88%
96%
98%
98%
100%
97%
89%
100%

If those were CC ratings, they'd be

4.4 - probably okay, but possibly problematic
4.8 - very good
4.9 - great
4.9 - great
5.0 - outstanding
4.8 - very good
4.8 - very good
4.4 - probably ok, but possibly problematic
5.0 - outstanding

The ratings are derived from different systems, but the results aren't that different from what you see on CC - most users are 4.7 and above. Of course, if an Amazon seller is crappy, they might find it hard to sell stuff. Perhaps that's why the 88% seller has the lowest price.

Interestingly, the real deviation comes when looking at products on Amazon, where people do tend to use the rating system more correctly. It's always amusing to me to see people rating things like The Old Man and the Sea and The Count of Monte Cristo with 1 star (they were probably forced to read the books for English class (= ). I guess we humans are more forgiving of people than products. =)

Would a thumbs up/thumbs down result in radically different results? Probably not over hundreds of ratings, although there will likely be a greater range for folks who've played less than 20 games or so.

[Kaskavel]

I personally find the ratings a bit too indifferent to me, hardly ever caring to even notice. There are many reasons for this.
1. I find the tab "fair play" too strange....What does a 3/5 rating in fair play mean?
2. The gameplay tab seems irrelevant. We do have a scoreboard after all
3. But the most important thing that I can definitely say, is that the ratings represent almost entirely the attitude tab. Most people will not put 1s in a cook just because he was playing badly, they will put 4s, maybe even 5s or no rating at all. But if the same opponent call the player "an idiot" or something, he will immediately take 3 1s on the spot in the majority of cases. This means that indeed....the scale 4.0-5.0 represents something...and the scale does not need to balance on 3.0, but...what it represents is irrelevant to the creator's intention. Basically, whenever I see a guy with 4.2, I know I am going to face a jerk. I even try some times out of curiosity to start very very polite conversation in chat to see if I am justified and it almost always turns out very badly.
I do not think that the rating system can be fixed. But the OPP suggestion seems a step towards a better situation.

[homes32]

Just don't forget that there might come a time when someone gives you a low rating or a 'thumbs down' when you feel you don't deserve it. Having just two ratings to choose from means either it's a positive or a negative. When people get a negative rating, they will also likely take it to the extreme and might want that person banned immediately and say the ratings system is flawed.

I honestly don't see much different with the current system. If you want examples go look at my wall or Ratings Left. Plenty of people leaving flames because I rated them 3's for Average (nothing against the players. they were truly just average games) and not all 5's. This isn't even the extreme where someone leaves a 1 star. Somebody will always be a dick and there's not much you can do about that with either system. The best you can do is make it useful for the decent players. If I see a guy that is consistently leaving negative ratings for everyone he plays seemingly for no reason he is probably going on my foes list. If I see a guy with a negative and a bad teammate tag next to it I may think twice before joining a team game with that player but would have no issues with a standard game. Rank/points don't always mean anything either. Maybe the guy is a brigadier general because hes awesome. or maybe its because he only plays 1v1 circus maximus with cooks...

One idea that may help he negative/thumbs down rating is to force the use of at least x number of tags to be used in order to justify and submit the rating. This would help the mods see the intention of the rater and with little trouble look at the game in question and see if the rating was justified.

@macbone
That's why I don't really like a percentage based approach. It seems to take away from the actual ratings themselves.

[EBConquer]

... a tag count. Put the top tags left on a given player.

This is a great idea regardless if anything changes. "Top tags at-a-glance" gets two thumbs up from me.

As with most people, when i receive a rating of 3 or 4 i do find myself a little bummed feeling that perhaps i could have done something better for the overall experience of the game even though on paper it's average or above. So w/ saying that, it does seem like something needs to change.

Maybe a rating of 1 is a thumbs down while a 5 is a thumbs up? Or a 1 is a while a 5 is a

Not sure, i'd have to read all the other rating reform threads.