The QWK essentially measures how much the first trait scores differ from the second trait scores for a given trait. QWK has a high of 1 and a low of -1. The closer QWK is to 1 the closer the first and second scores are to each other and the lower it is indicates greater disagreement between the scores.
This QWK value is generated by looking through all the trait scores between the first and second scores and comparing them. This comparision consists of generating an expected value that is made up by multiplying the number of scores for the first and second trait score then dividing them by the total number of scores. Next, the value d is generated which represents the diffrence between the first and second trait score value. D is calulated by squaring the diffrence beween the two trait scores and dividing it by the number of trait scores on the rubric minus one and also squaring this value. The numerator is generated by multply d by the number of trait scores in the current cross section of the first and second trait scores. The demoninator is generated by multiping d by the number of expected scores. Finally, when the numerator and denominator have been factored in for all trait score combinations, The QWK is generated by 1 minus the numberator/demoninator
numerator = 0
demoninator = 0
total_scores = Sum of all the cells in the trait score matrix
num_rating = Number of trait scores on the rubric
for each fst score trait score (the rows):
for each snd score trait score (the columns):
fst_scores = Sum of all cells for the trait score row on the score matrix
snd_scores = Sum of all cells for the trait score collumn on the score matrix
expected = (fst_scores * snd_scores) / total_scores
d = (diff between fst_score_trait_score and snd_score_trait_score)² / (num_rating - 1)²
numerator = numerator + (d * cell value at meeting of fst_trait_score and snd_trait_score) / total_scores
demoninator = demoninator + (d * expected) / total_scores
QWK = 1 - numerator / demoninator