Ratings Committees Tips/Tools

 

ORGANIZING A RATINGS COMMITTEE

 
 
Abstract:
This paper will outline the purpose and basic steps in organizing a ratings committee at the local, state or regional level.
 
 
Author:  
Chris Thomas, USAPA Ratings Chair
 
 
Background
In mid-2013, Chris Thomas, organized a Regional Ratings Committee composed of eight experienced tournament directors (TD’s) from all the states in the West Region that hold sanctioned tournaments.
 
 
Purpose:  
Purpose of this committee was to have a more systematic methodology for establishing/changing players’ skill levels.  Over the past several years, TD’s and players have pointed out three problem areas with rating players:

  • Subjectivity – Questions and complaints regarding who rated the player as well as relying on a rating given by one or two individuals.
  • Fairness – Players believed they were rated unfairly when compared to other players; or didn’t understand what their rating was based on.
  • Communication – TD’s and players not informed in a timely manner regarding a player’s new skill rating.  In several cases, neither the player nor the TD had advance knowledge of registered tournament player’s rating change.

 
 
Solution:         
Although there is no perfect ratings system, ratings committees can go a long way in addressing the three problem areas listed above.  In terms of subjectivity and fairness, an experienced committee of several members, who either have observed the individual player or have played with/against them, is more objective in analyzing a player’s skill than just one or two people.  Communication guidelines established and executed by the committee chair provide mandatory email communication with a player, in advance of actually changing their skill rating in the USAPA ratings file.
 
 
Example:
In mid-2013, after several tournaments in the West Region, the Regional Ratings Committee reviewed 45 players for possible rating change.  Committee members, via emails, provided detailed data and reasons on why a player should be re-rated.   Of these 45, the committee decided on changing the rating of 43 players.  An email was sent to each player informing them of this proposed change, why they were being re-rated, and that this change would be effective the next USAPA rating date of 9/1.  Of the 43, only 1 player filed an appeal.  In the same email, players were also given the option of accepting the new rating immediately.  Players were informed that they have a responsibility of informing TD’s, of a tournament that they are registered for, of their new rating.
 
 
Committee Basics:

  • Rating committees can be local, state or regional in nature.
  • Committee members should be experienced players and/or tournament directors.
  • Committee members also provide a boarder understanding of players’ skills if the committee members have played or directed tournaments outside of their local area, state or region.
  • Anyone, including non-committee members, can nominate a player for a rating change.
  • Nominations must include player’s basic info, current rating (if available) and why the nominator has submitted this nomination.
  • Consensus decision making should be strived for, but a simple majority vote on a player’s rating will suffice.
  • If there is considerable pro/con discussion regarding a player, it is suggested that the committee make no recommendation for a change, but instead place that player on a “watch list.”
  • Committees should keep in mind the current USAPA guideline of changing a player’s rating is limited to four times a year: 1/1, 4/1, 6/1, 9/1.  Discussion/review of players, subsequent recommendations, and communicating decisions to the players must be done with enough advance notice prior to these dates.  That way, players, their double partners and TD’s won’t be blindsided by a last minute rating change.
  • There are and have been exceptions to changing a player’s rating in between these dates.  Committees should have flexibility, based on the circumstances of the individual player’s results and skill level, to change a rating at other times of the year.
  • Committees should examine three factors for a rating change: 1) USAPA skill definitions as observed by one or more committee member; 2) performance results, both age and skill based tournaments, which may include the level of competition; and 3) potential of the player to compete at the next skill level, not necessarily if they can win a medal at the next level.
  • Players’ results should include more than just one tournament.  There can be unusual exceptions to this guideline, but committee members should be wary of proposing a change based on one tournament.

 
 
Guidelines:
As mentioned previously, ratings committees can be a method to provide for a more systematic review of player skill ratings.  These committees should not conflict with established USAPA ratings documentation, skill rating definitions and appeal process.  Ratings committees should operate within USAPA guidelines.  Committee chairs should inform USAPA ratings chairman of the establishment of their committee.  Here is a link for a sample spread sheet (Excel file) for committees to use in their evaluation process.

USAPA Affiliate Partners

fdb4754712f821b07cfcb8ab127ef852##################