Text2Pose Human Evaluation

This repository contains data and scripts for reproducing the human evaluation of the paper:

@inproceedings{Jiang2025PoseEvaluation,
  title={Meaningful Pose-Based Sign Language Evaluation},
  author={Zifan Jiang, Colin Leong, Amit Moryossef, Oliver Cory, Maksym Ivashechkin, Neha Tarigopula, Biao Zhang, Anne Göhring, Annette Rios, Rico Sennrich, Sarah Ebling},
  booktitle={Conference on Machine Translation},
  year={2025},
  url={https://github.com/sign-language-processing/pose-evaluation}
}

We use the Signsuisse dataset released in the WMT-SLT 23 campaign and follow the practices in:

https://github.com/WMT-SLT/wmt-slt23.

See also the guidelines to set up Appraise:

https://github.com/AppraiseDev/Appraise/blob/main/INSTALL.md

and EASIER interim evaluation as an example:

https://github.com/bricksdont/easier-interim-eval

Raw pose data of the system outputs (CC BY 4.0)

https://drive.google.com/file/d/1nQwm0F50RjcmU6RgHUA9dBmk31AdioMU/view?usp=sharing https://drive.google.com/file/d/1y0PN6UQlO41cJY8-Iv0W0MSbbvWCaxQs/view?usp=sharing