Group Logo

The Benchmarking Task Team is working to develop benchmarking toolkits for variant calls. The initial focus of the team has been developing standardized definitions of performance metrics, as well as tools to calculate and stratify these performance metrics for human germline small variants, including SNPs, small indels, and complex variants. Our work includes developing a sophisticated tool to compare variant call files, including handling of different representations of complex variants (presentation). The team has also been developing standardized definitions for performance metrics and standardized ways to stratify performance into different categories (link), including some standard bed files of genome regions for stratification (e.g., GC content, low complexity and repetitive regions, segmental duplications, and functional regions (link). This work is closely aligned with the Genome in a Bottle Consortium, which has been developing high-confidence variant call sets for benchmarking variant calls in human genomes.
Authors of 3 variant comparison tools that handle complex variant comparisons are involved in the Team:

Future work may include developing benchmarking toolkits for structural variants, somatic mutations, RNA-sequencing, and variant annotations.




To review Team membership, please visit

How We Work

The Global Alliance produces high-impact deliverables to enable data sharing and catalyzes key projects to demonstrate value.

Get Involved!

Get started by becoming an Individual Member and receive updates about ongoing work.