Dashboard
Enter the dashboard password
Human evaluation · live totals
How the study is going
Mean ratings by pipeline
No judgments yet.
Which version raters preferred
One pick per judgment. Ties are kept rather than forced.
Slot bias check
The order is shuffled per sentence, so a lopsided split here means raters favour a position, not a pipeline.
Coverage of the live sentences
The plan is three judgments per sentence; the queue serves the thinnest first.
The 60-sentence sample
Sentences and pipeline outputs
| Id | Doc | State | Dutch original | Judged | P1 clear | P2 clear | P1 detail | P2 detail | Preferred |
|---|
Nothing matches that filter.
Who has rated
Raters
| Nickname | Dutch | English | Judgments | Sittings | First seen | Last seen | Id |
|---|
No one has started a sitting yet.
Sentences in, answers out
Import & export
Import a CSV
Rows are matched on item_id: an existing sentence is updated, a new id is added. A column the file leaves out keeps the value already stored, so a file of pipeline outputs alone is enough.
Columns that are understood:
- item_id (required)
- complex_nl · simple_nl
- output_p1 — translate → simplify (also p1, translate_simplify)
- output_p2 — simplify → translate (also p2, simplify_translate)
- row_id · doc_id · enabled
Export
Both files are UTF-8 CSV. The judgments file is one row per completed comparison, with each rating already attributed to the pipeline that produced it.
Judgment columns: judgment_id, created_at, participant_id, nickname, dutch_level, english_level, session_id, item_id, doc_id, a_pipeline, p1_understand, p1_meaning, p2_understand, p2_meaning, preferred, preferred_side, elapsed_ms.
Running the study