>> Rating calculation method: ARL

[e_i_pi]

Concise description:

• The ratings system drawing a lot of criticism.

Specifics:

• People are complaining about abuse, ie - certain people giving out all 1s.
• People are complaining about lack of seriousness, ie - certain people giving out all 5s.
• The forums are getting clogged with this craziness, and I'm having trouble finding posts with worthwhile content.

This will improve the following aspects of the site:

• Suggestion:
  For arguments sake, I am going to consider that we only give players 1 rating out of 5, not the 4 in here. I think the 4 are good, but lets keep it simple, so the hoopleheads don't get confused.
  
  I suggest we calculate the rating you get using a reverse process that will stymy people who want to abuse the system.
  
  THE PROCESS:
  Everyone gets a number attached to them called 'Rating Left - average' (I'll call it ARL), which is an average of every rating they have left. People who give out lots of 5s will have a high ARL, people who give out lots of 1s will have a low ARL.
  
  Supposing CCBill has a 1v1 with CCJane, they both enjoy the game so much that they give each other 5-star.
• Before the game, CCBill had no ratings received, but had left a 5-star rating for CCIncognito in another game.
• Before the game, CCJane had no ratings received, but had left a 3-star rating for CCAverage in another game.
  
  They both have an average rating received of 5 stars, so both get 5 star ratings, BUT, we can calculate another average rating for them, based on ratings left, which may give more, or different, information ---
• CCBills ARL after the game is 5 (5+5/2)
• CCJanes ARL after the game is 4 (3+5/2)
• The NET rating left by CCBill is 0 (rating of 5 minus the ARL of 5)
• The NET rating left by CCJane is +1 (rating of 5 minus the ARL of 4)
• In the 'My Ratings' Section of CCBill, it has CCJanes left rating listed as 5 (+1).
• In the 'My Ratings' Section of CCJane, it has CCBills left rating listed as 5 (0)
  
  Then you can list average ratings as average rating received with the number in brackets next to it, or you can simply add the number after the brackets if you want for a different type of score. Personally, I think just using the number in the brackets by itself is the best. Also, capping the number at (-2, +2) will have its benefits. Heres why:
• People can leave all 5's but they're never going to be able to leave a 5 with (+num), so we won't get masses of high ranked players due to superhappyjoyjoy players. They can only leave ratings of (-2..0)
• People can leave all 1's but they're never going to be able to leave a 1 with (-num), so we won't have idiots killing peoples ratings all the time. They can only leave ratings of (0..+2)
• People who use the ratings system as it is intended, and submit 3's for average games, and 1-2s for poor play, 4-5s for good play, will have the benefit of being able to leave ratings of (-2..+2).
  
  All it takes is having one extra number attached to each account, which is average rating left. Now, in terms of the ratings database, this is gonna take updating and all that jazz, I can think of a few time and processor efficient ways of going about this, but for now I'll leave off, and wait for the following:
  
  Thoughts / ideas / evaluations / flames

[meathead]

Hmm, not a bad suggestion IMO. I think possibly it would be better to have 2 ratings though, the current one, and a note of what you have left other people, something like

3.4/4.1 for example, with the former being average ratings recieved and the latter ratings given.

[lackattack]

I was thinking of something along the same lines, and you beat me to the punch!

I think we should print ARL on each rating entry (perhaps between the stars and the username)

I don't think it's a good idea to mess with the raw data in listings, but I would like to change the formula for overall average to be based on ARL +-2 so that the number next to your username isn't inflated/deflated by high or low markers.

I could store ARL on player records as you suggest, and of course it could change each time you leave a rating. To keep things completely up-to-date I'd have to re-calculate the overalls of anyone you've left ratings each time your ARL changes. That's a lot of work on the server. Instead, I might just update the overall of the person receiving ratings. So your overall might be inaccurate due to the ARL effect but it will get corrected each time you receive ratings.

[antony.trupe]

I think this will encourage people(who actually think about the calculation) to leave 3's for the average so that their 1's and 5's have more weight when they do want to leave a non-average rating.

On that note, can you change all my 5's I've left to 3's?

[Frop]

If I understood this correctly we're going to unleash some complex mathemical thingamajig on completely arbitrary ratings? Stephen Hawking wouldn't touch this one with a 10 feet pole.

Maybe I'm too conservative, but I found the old description a lot simpler.

"Jack was a great teammate - we communicated well and coordinated our strategy."

"Jack and Jill never attacked each other and left their mutual borders undefended for the duration of the game, without announcing any alliance."

[LYR]

I'm just going to input the fact that some people give three stars for something where you couldn't have done any better.

I played an rt game with a player and only got three stars. I happen to know that I am a very punctual rt player. Could I have been any more punctual? No. Yet I still get three stars.

Again, on fair play. I don't understand have I could have gotten any fairer. The only way I can think of not playing fair is breaking treaties or using the clock tactic (in addition to the actual rules of the game), niether of which I did in this specific game (or do at all). I still only got three stars.

Solution: ?????

[Lindax]

I find the ARL suggestion very complicated! Why not simplify things a bit and use 3 options instead of 5? i.e.: poor, average, good?

[lackattack]

Why not simplify things a bit and use 3 options instead of 5? i.e.: poor, average, good?

We'd still face the same problem - some people would rate "average" by default and some would rate "good" by default!

[jako]

I'm just going to input the fact that some people give three stars for something where you couldn't have done any better.

I played an rt game with a player and only got three stars. I happen to know that I am a very punctual rt player. Could I have been any more punctual? No. Yet I still get three stars.

Again, on fair play. I don't understand have I could have gotten any fairer. The only way I can think of not playing fair is breaking treaties or using the clock tactic (in addition to the actual rules of the game), niether of which I did in this specific game (or do at all). I still only got three stars.

Solution: ?????

i find this makes quite a lot of sense. why constitutes someone being more fair or less fair or the fact that i took 10 hours to take my turn rather than say 2 hours? i have to work 8-10 hours a day, the fastest i can leave a rating is when im at home after work should i still get a 3 for taking 10 hours?

[e_i_pi]

If I understood this correctly we're going to unleash some complex mathemical thingamajig on completely arbitrary ratings? Stephen Hawking wouldn't touch this one with a 10 feet pole.

You consider this complex? It's a simple average applied to a range. It would be about 1 line of code, give or take 0 lines. Here, let me demonstrate my Super-Hawking intelligence:

RELATIVE VALUE LEFT = VALUE LEFT - AVERAGE(All VALUES LEFT)

Zomg, where did I put that 10 foot pole?!?! And FYI, this would fix the 'arbitrary' nature of the process, which you seem also to complain about.

Kj

[MajorRT]

Excellent idea , Lack . I think a single # would suffice, describing the player's average RELATIVE value over or under values left. Thus, the basic, run of the mill player should be 0.0 ; we could then see the best behaved players, at a max of 2.0 , and the worst at -2.0 ...

[gloryordeath]

So what your trying to do is make it so everyone is a three, why not just give 1 choice?

[e_i_pi]

So what your trying to do is make it so everyone is a three, why not just give 1 choice?

It won't make everyone a 3. The example demonstrates this.

[foregone]

I'm pretty sure this has been mentioned before, but (and correct me if I'm the only person who does this) I don't rate everyone in a game at all, only those who left me with a particularly good/bad experience. So, you'll generally notice 4s and 5s coming from me, or will rate someone low for whatever aspect I didn't like. This means I've left a couple of 1s for attendance where someone deadbeats out and generally I only rate them on attendance, if they deadbeat out without me being able to tell anything else.

How would ARL affect me if this were my standard methodology? (Sorry, no head for maths so working out the formula is tre difficult).

[e_i_pi]

I'm pretty sure this has been mentioned before, but (and correct me if I'm the only person who does this) I don't rate everyone in a game at all, only those who left me with a particularly good/bad experience. So, you'll generally notice 4s and 5s coming from me, or will rate someone low for whatever aspect I didn't like. This means I've left a couple of 1s for attendance where someone deadbeats out and generally I only rate them on attendance, if they deadbeat out without me being able to tell anything else.

How would ARL affect me if this were my standard methodology? (Sorry, no head for maths so working out the formula is tre difficult).

If you calculated based only on ratings left, and you left mostly 4s and 5s, probably you would only be leaving an ARL of -2 to +1 or so. You could remedy this by counting non-ratings as 3s for the sake of calculating ARL, which would balance things out. I know what you mean though, I don't leave Fair Play very often, and if someone deadbeats, they get a 1 and nothing else. Other than that its mostly 4s and 5s. If there was support for this method, I'd do dummy runs in Excel to test various scenarios and pass the results on to admin. There's definitely work-arounds to problem areas, what matters most (IMO) is whether the work-around is server-processor efficient, whether it fairly represents what the community wants, and whether it is an improvement on what currently exists. I'd just like to open a discourse on possible ways of adding value and enjoyment to the site.

Kj

[kewlboy4399]

I have to agree with Frop. I preferred the old rating system... Exponentially. Much easier to tell who has been left feedback for what. And when people were left negative feedback, you can read through their feedback and figure out for yourself if you find it warranted.

And, if you really don't care enough, just don't join a game with the person.

[PLAYER57832]

I like the basic idea ... that you include the ratings left along with ratings received. BUT, I don't think they should be combined or limited more.

Also, I think you need to have some indication of the number of players. Anyone will come across a few jerks. But if you come across a bunch ... well, maybe, just maybe someone else is the real jerk.

I don't think there is any real sense in worrying to much about the average/excellent distinction. Few people are going to decide to not play someone because they have a 3 vs 5. BUT, if they can see that they give mostly 1's ... this is probably someone who blames others and to be avoided, unless they have only played that one game.

[mightyredarmy]

I think I should have posted this here...

Problem 1 - too much confusion is caused by 5 stars in 4 categories

Problem 2 - we're not gonna get feedback back because of moderation 'problems'.

Easy solution

Award a plus star (equivalent of old positive feedback) for an enjoyable game.
Award a negative star (equivalent of old negative feedback) for a negative playing experience.
Award neutral if you just don't really care but just want to build up your rating medal

No commentary is added, but for a really special game use the wall

The total of pluses less minuses is your rating, now shown as a total figure.

So - if you were rated 83-15 under the old system, you now get a score of 68, with your old comments on your wall

4s and 5s awarded since the new system started get converted as pluses
3s get converted as neutral
1s and 2s get converted as negative.

Advantages:

1. Some of the experienced players who built up 500 or more positives get to have a rating which reflects that achievement so all that hard work doesn't need to be lost after all.

2. The odd negative received from a vindictive idiot will not have a major effect, but repeatedly stupid play and deadbeating will attract lots of negatives therefore leading to:

3. A consistently bad player will get a negative rating and be easy to spot (or they may have played lots of games without getting many pluses, again fairly easy to spot.)

4. It's simple.

Jod done.

[snifner]

To make the current system work there should be 3 choices.

Leave a 5 if you feel played well, ect.
Leave a 1 if they played like a fool or missed a lot of turns.
And leave nothing if you think they are average.

(then their number well be an average of the good and bad..)



UNLESS.. you want everyone to have 3's
Then i'll flood the system with them and nobody well be a 5.

It just seams stupid to give everyone a 3.

[CandaVespin]

Apparently I'm one of those evil people using the system as it was intended to be used and rate a lot of people as 3's. With that stated, I recently discovered that I was rated as a 1 because of this. When I approached the individual to discover why he felt I rated a 1 so that I could improve myself (note: I am not asking him to change his rating, I'm asking to understand his rating to improve myself) he told me flat out that he rated me a 1 because of how I rated other people as mostly 3's.

A little insight, I rate people a 3 in all categories except attendance unless they do something/say something to rate higher/lower. All people get rated a 5 for attendance if they miss no turns at all. If they miss another turn, I will lower their rating by 1 / turn missed (life happens, I understand that). If they deadbeat, it's an automatic 1.

My thoughts is that this is in line with how the ratings are described. I'm not sure why others feel that "5's" are the average or should be mostly rated as 5's. They leave no room for that truly exceptional person who exhibits exceptional behavior. I have left some individuals 5's. And usually those are people that I have had multiple encounters with that gave me a very good impression of their overall abilities. Or, I've rated them a 5 because they made that spectacular move right at the end of the game to pull the win. Or a daring attempt and nearly made it but still lost.

I wish I could say I had a good solution. I wish I knew what could be done to help people understand the rating system as it was intended. I hope that your idea Lack (which looks a little complicated to me, but I can deal) will help allay concerns others have. I wish I really knew how to make this better.

Personally, I like this new rating system and I think you did a great job with implementing it. I hope to see some great improvements to help alleviate some of those top/bottom box mentality many people seem to hold regarding this rating system. And at the end of the day. As long as I have a 3+ rating, I'm completely happy. (c:

[snifner]

So what your trying to do is make it so everyone is a three, why not just give 1 choice?

It won't make everyone a 3. The example demonstrates this.

Right this minute. I'll give you a 5 if you do something extraordinary and a 1 if you are an ass or miss turns. Besides that if I have nothing good or bad to say, I leave it blank so I don't cloud the system.

BUT,, from what I'm reading.. it sound like you want me to leave everyone 3's which makes a complete mess out of the averages.

If you dont have anything to say.. leaving a 3 is stupid.. this system makes no sense.

[CandaVespin]

Then i'll flood the system with them and nobody well be a 5.

Based on the way this rating system is written, the ONLY person that should be a "5" is that person who received 1 rating and it just happened to be a 5.

5's should be, based on how they are written, exceptional, Above average, way better than your normal player. Not the standard rating.

Strive to be better than a 3. Strive to do something to EARN those 5's. Don't expect them because that's the top box.

In addition, leaving 3's does NOT cloud the system. It makes perfect sense. It means that you felt they were a reasonable enough player to warrant an average rating. Leaving only 1's or only 5's just goes back to our "Positive/Negative" rating with no regard to levels of quality. You're talking about "Black and White" instead of a range of grays.

[snifner]

so to fix the system.

Leave a 5 if they do something great.
Leave a 1 if they miss turns or are lame.

Leave nothing if you have nothing to say.. this creates a true stat.

[CandaVespin]

so to fix the system.

Leave a 5 if they do something great.
Leave a 1 if they miss turns or are lame.

Leave nothing if you have nothing to say.. this creates a true stat.

How does it create a true statistic? It goes back to like I said, a 2 rating system. Why not just have a 1 / 2? Or 1 / -1? 0 / 1? True / False? Good / Bad? Black / White?

You're breaking down a scaled system into a Black/White system. Another thing. On averages, If you get 5 1s and 5 5's, it's the exact same rating as 10 3's.

[snifner]

If you get 5 1s and 5 5's, it's the exact same rating as 10 3's.

If you got 5 - 5's and 5 - 1's then your score would be about a 3 and it would be obvious you have an attitude problem or miss a lot of turns.

If you played ethically and never missed a turn your score would show it and you should ALWAYS be a high 4 or 5. IMHO everyone who plays ethically and takes their turns fast should be a 5. But there are assclowns on this site that could careless about fast turns or missed turns and will give people 1's and 3's just for giggles.

If you were a jerk in your games or missed a LOT of turns you would be a 1, 2 or 3 and it would be obvious you had issues. Leaving 3's just help hide all the 1's and 5's. Any score under 5 should show that the person has a problem in one or more area.

I mean seriously if you and everyone else mindlessly just give everyone 3's, then someone giving them a 1 or 5 wouldn't really reflect anything.

Examples:
1. Person plays well and never misses a turn. After 10 to 20 games he has 20 - 5's. His score reflects as 5.
2. Person plays well and occasionally misses a turn. After 10 to 20 games he has 15 - 5's and 5 - 1's. His score reflects 4.
3. Person plays like an ass and misses a bunch of turns. After 10 to 20 games he has 5 - 5's and 15 - 1's. His score reflects 2.
Anything below a 3.5 should start to set off alarms about the player your about to join a game with.

Now add 20 to 40 3's in each of those and all it does is clouds it all up a little.

I think we should leave 1's, 2's, 4's & 5's if you dont have anything good or bad to say which is 80% of the time leave it blank.