Difference between revisions of "Quality score formula"
| Line 7: | Line 7: | ||
The formula is: Q=E/V*1000, where | The formula is: Q=E/V*1000, where | ||
| − | *Q | + | *Q - quality score |
| − | *E | + | *E - number of mistake points scored for the selected piece of text |
| − | *V | + | *V - the volume of text in which the mistakes were found. |
E.g. the text contains 500 words and contains 12 mistake points, the quality score will be 12/500*1000=24 | E.g. the text contains 500 words and contains 12 mistake points, the quality score will be 12/500*1000=24 | ||
| Line 22: | Line 22: | ||
[[File:CR Configure evaluation process.jpg|border|450px]] | [[File:CR Configure evaluation process.jpg|border|450px]] | ||
| − | The system will randomly find the piece of text with corrected segments containing the indicated number of words. And the size of this text may differ depending on the percentage of corrections. E.g. when you select 300 words to evaluate, they may be found in a piece of text containing 500, 1000, or even 10000 translated words. You’ll just get your 300 words with corrections to classify mistakes, and the system will remember how many total words are contained in the selected piece. You’ll be able to see these numbers on evaluation page: | + | The system will randomly find the piece of text with corrected segments containing the indicated number of words. And the size of this text may differ depending on the percentage of corrections. E.g. when you select 300 words to evaluate, they may be found in a piece of text containing 500, 1000, or even 10000 translated words. You’ll just get your 300 words with corrections to classify mistakes, and the system will remember how many total words are contained in the selected piece. You’ll be able to see these numbers on the evaluation page: |
[[File: CR Evaluation details.jpg|border|250px]] | [[File: CR Evaluation details.jpg|border|250px]] | ||
| Line 32: | Line 32: | ||
We came to this through several stages of evolution. | We came to this through several stages of evolution. | ||
| − | First, the system was selecting just the beginning of the text, and we found out the translators started to translate first 1000 words better | + | First, the system was selecting just the beginning of the text, and we found out the translators started to translate first 1000 words better than the rest of the text. So we decided to select the random part of the text. But then the system sometimes selected the pieces containing no corrections while skipping heavily corrected parts. So we changed the logic, and now the system returns the required number of corrections to the evaluator, but remembers how much text it took to find these corrections. |
Revision as of 10:50, 8 December 2017
How Quality Score is calculated?
Quality score reflects the number of mistakes made per 1000 words of translated text.
The formula is: Q=E/V*1000, where
- Q - quality score
- E - number of mistake points scored for the selected piece of text
- V - the volume of text in which the mistakes were found.
E.g. the text contains 500 words and contains 12 mistake points, the quality score will be 12/500*1000=24
How the number of translated words is selected?
Before starting the evaluation, the reviewer selects the number of words to evaluate:
The system will randomly find the piece of text with corrected segments containing the indicated number of words. And the size of this text may differ depending on the percentage of corrections. E.g. when you select 300 words to evaluate, they may be found in a piece of text containing 500, 1000, or even 10000 translated words. You’ll just get your 300 words with corrections to classify mistakes, and the system will remember how many total words are contained in the selected piece. You’ll be able to see these numbers on the evaluation page:
And the system will use the total number of words as V in the formula above.
Why is it made that way?
We came to this through several stages of evolution.
First, the system was selecting just the beginning of the text, and we found out the translators started to translate first 1000 words better than the rest of the text. So we decided to select the random part of the text. But then the system sometimes selected the pieces containing no corrections while skipping heavily corrected parts. So we changed the logic, and now the system returns the required number of corrections to the evaluator, but remembers how much text it took to find these corrections.
