Difference between revisions of "Quality score formula"

Revision as of 10:13, 9 June 2017

How Quality Score is calculated?

Quality score reflects the number of mistakes made per 1000 words of translated text.

The formula is: Q=E/V*1000, where

Q – Quality score

E – Number of mistake points scored for the selected piece of text

V – The volume of text in which the mistakes were found.

E.g. the text contains 500 words and contains 12 mistake points, the quality score will be 12/500*1000=24

How the number of translated words is selected?

Before starting the evaluation, the reviewer selects the number of words to evaluate:

File:Evaluation report-configure evaluation process.png

The system will randomly find the piece of text with corrected segments containing the indicated number of words. And the size of this text may differ depending on the percentage of corrections. E.g. when you select 300 words to evaluate, they may be found in a piece of text containing 500, 1000, or even 10000 translated words. You’ll just get your 300 words with corrections to classify mistakes, and the system will remember how many total words are contained in the selected piece. You’ll be able to see these numbers on evaluation page:

File:Corrected text.png

And the system will use the total number of words as V in the formula above.

Why is it made that way?

We came to this through several stages of evolution.

First, the system was selecting just the beginning of the text, and we found out the translators started to translate first 1000 words better then the rest of the text. So we decided to select the random part of the text. But then the system sometimes selected the pieces containing no corrections while skipping heavily corrected parts. So we changed the logic, and now the system returns the required number of corrections to the evaluator, but remembers how much text it took to find these corrections.