We have enough benchmarking experience for:

• MT quality comparison
• Summarization comparison
• speed tests
• Quality of Vector search (semantic search)
• the benchmarks can be automatically re-run if a new model appears to the mix
• the benchmark test suite can automatically produce graphs
• The benchmarking test suite also includes a dataset for benchmarking
• publish final work somewhere