Evaluator user manual

The page is under creation

Introduction

TQAuditor is the system that evaluates and monitors translation quality and allows you to:

• compare unedited translation made by a translator with edited version received from an editor;

• generate a report about editor’s corrections;

• classify each correction by mistake type and severity, thus allowing to get the translation quality evaluation score. By default, a maximum score is set to 100 points;

• ensure anonymous communication between a translator and an editor regarding corrections and mistakes classification;

• automate a process of maintaining the evaluation project;

How to accept invitation

1. The translation agency you cooperate with adds your account into the system. You receive an invitation email.

2. Accept the invitation by clicking the link in the received letter:

3. The following window appears:

Choose your own username and password, confirm the password, accept the Privacy policy and Terms of service (to read them, click the corresponding links highlighted in blue) and press "Submit".

4. Your account will be created.

Get started with an evaluator account

There are several main menus that you need to focus on:

• Quick compare — here you can compare two versions of translated files in the system.

• Projects — here you can find the list of projects.

• Reports — here you can find your reports.

• System — here you can find the Quality standard of the company you cooperate with.

Profile settings

On the My Profile page you can update your personal information and change your password.

To do so, go to the My profile menu:

Quick compare

You can compare two versions of translated files in the system. Press the "Quick compare" button to do it:

Click "Supported bilingual file types" to see the file formats TQA work with.

Press the "Upload selected files" button:

The comparison report will be created, where you can see the editor’s amendments highlighted with color, and different filters to simplify your work.

Projects

On the "Projects" page, you can find the list of projects that you have been assigned as an evaluator.

Click on the project's ID to open a project:

Project details

Once you've been assigned the project, you can see general information, project manager, and their email:

Upload files

First, you have to upload files. Press the "Upload files" button to do it:

Press the "Choose files" button to select two versions of the translated file.

Note: the evaluator is the only user who can upload files.

Create comparison report

Once you have uploaded files you can create the comparison report. Press the "Create comparison report" button to do it:

When the comparison report is generated, you will see time limits details, the comparison of the original and amended translation, markup display settings, and comparison details:

For more information, please check the "Comparison report" page.

Start evaluation

Once files are uploaded and the comparison report is created, you can start an evaluation by selecting whether an automatic or manual word count.

The automatic word count is used for fully reviewed files and the manual word count is used for partially reviewed files. You can read more on the differences between the two types of word count and how to use them on this page.

Evaluation with automatic word count

To start the evaluation with the automatic word count, press the "Start evaluation (automatic word count)" button:

You can also adjust evaluation settings:

• Skip locked units — hide "frozen" units (for example, a client wants some important parts of the translated text to stay unchanged. Besides, extra units slow down the editor's work).

• Skip repetitions — the system will hide repeated segments (only one of them will be displayed in this case).

• Skip segments with match — fuzzy match percentage. The program will hide segments with matches greater than or equal to a number you specified.

• Evaluation sample word count limit — the number of words in edited segments chosen for evaluation.

Press the "Start evaluation" button once settings are adjusted:

Quality evaluation

Now you have to classify every correction by type and severity. Click the "Add mistake" button to do it:

Add the information about the mistake and click "Submit":

You can edit, delete mistake/comment or add another mistake by clicking the corresponding buttons:

When the classification of mistakes is done, click "Complete evaluation", write an evaluation summary, and press the "Complete" button.

The system will send the quality assessment report to the translator.

Please note that you may export the evaluation report with mistakes classification.

Back to the table of contents.

Reevaluation and arbitration requests

If the translator has requested a reevaluation, you will receive a corresponding email notification:

Click on the project ID, and it will bring you to the "Project details" page. Next, press the "Evaluation report" button and start reevaluation.

Note: in order to do reevaluation you have to reply to all the comments left by the translator.

Once done, press "Complete evaluation" -> "Complete".

Arbitration request

If there is still no agreement regarding the severity of mistakes, the translator can request arbitration. Then the stage of the project will change:

You'll also receive an email notification when the project is completed by the arbiter. Press the project ID to see the changes:

After you receive the email with translation quality evaluation, follow the project ID link in your email:

And do the following actions:

1. View the Comparison report. Look through all the corrections made by the evaluator:

2. Go to the Evaluation report:

3. Look through classification of each mistake.

4. If you agree with classification of all the mistakes, press "Complete project":

The project and its evaluation score are finalized at this stage. If you do not agree with classification of some mistakes, do the following actions.

5. Press "Add Comment" in the box of the mistake that you do not agree with and enter it by clicking "Submit":

6. When you have entered all the comments, send the project for reevaluation by pressing "Request reevaluation":

7. The project will be sent to the evaluator, who will review your comments.

If they are convincing, the evaluator will change the mistake severity in your favor.

You will receive the reevaluated project. You are able to send this project for reevaluation one more time.

8. If you have not reached agreement with the evaluator, you can send the project to the arbiter by pressing "Request arbitration" (it appears instead of "Request reevaluation"):

9. The arbiter will provide a final score that cannot be disputed.

Reports

To check your translator reports, go to Reports ⇾ Translator report

Here you can find different charts and diagrams:

• Score dynamics:

• Monthly average score dynamics:

• Managed by:

• Projects by specialization:

• Mistakes by type:

• Mistakes by severity:

Quality standard

To check the quality standard of the company you cooperate with, go to Settings ⇾ Quality standard

Here you can find:

• Score limit and Mistake severities:

• Mistake types:

• Mistake type score weight coefficients per specialization:

• Quality levels

Filters

There are filters that can help you to display the data you need.

Project filters

You may order projects by particular criteria: click the title of any column and all the projects will line up (the arrow button appears).

Note: The column headers which enable this sorting feature are highlighted in blue.

Also, you may find here additional filters:

• Is overdue — the system will display the overdue projects only.

• Stage — tick one of the cells, and the system will display only projects at a particular stage.

• Project code — the system will display projects with the specified project code.

• Translation job code — the system will display the projects with a particular translation job ID, entered by the manager.

• Review job code — the system will display the projects with a particular review job ID, entered by the manager (differs from translation job code).

• Project name — the system will display the projects with the specified name.

• Specialization — the system will display the projects with a particular translation specialization.

• Service — the system will display the projects with a particular translation service.

• Evaluation count — the system will display projects with the specified evaluation count.

• Manager — the system will display the projects with a particular supervisor.

• Creation date — the system will display the projects with particular creation dates.

Report filter

Here you can specify the project creation date range: