Steven Brenner
UC Berkeley
Title : Findings from the third Critical Assessment of Genome Interpretation, CAGI 2013, a community experiment to evaluate phenotype prediction.
Abstract :

The Critical Assessment of Genome Interpretation (CAGI, 'k¨¡-j¨¥) is a community experiment to objectively assess computational methods for predicting the phenotypic impacts of genomic variation. In the experiment, participants are provided genetic variants and make predictions of resulting phenotype. These predictions are evaluated against experimental characterizations by independent assessors. A long-term goal for CAGI is to improve the accuracy of phenotype and disease predictions in clinical settings.

The third CAGI experiment (concluded in July 2013) consisted of ten diverse challenges.CAGI deliberately extends challenges from previous years, with the continuity allowing measurement of progress. For example, in the second CAGI, in a challenge to predict Crohn¡¯s disease from exomes, one group was able to identify 80% of affected individuals before the first false positive healthy person.In the third CAGI experiment, this challenge used an improved dataset, and several groups performed remarkably well, with one group achieving a ROC AUC of 0.94.The experiment also revealed important population structure to Crohn¡¯s disease in Germany.

For three years, CAGI has posed a challenge with Personal Genome Project (PGP) genome data.This year, two groups were able to successfully map a significant number of complete genomes to their corresponding trait profiles submitted by PGP participants. In the expanded challenge to predict benign versus deleterious variants in DNA double-strand break repair MRN genes¡ªRad50 (from last year), Mre11, and Nbs1¡ªas determined by those that appear in a breast cancer case versus healthy control, predictions show how methods differ sharply in their effectiveness even amongst proteins in the same complex.

A new challenge this year was to use exomes from families with lipid metabolism disorders.In the case of hypoalphalipoproteinemia (HA), a company made predictions which showed how understanding the problem structure and employing an extensive knowledgebase led to remarkably good results.Another related challenge revealed a twist wherein real-world data differed sharply from theoretical models.

The other challenges were to predict which variants of BRCA1 and BRCA2 are associated with increased risk of breast cancer; to predict how variants in p53 gene exons affect mRNA splicing; to predict how well variants of a p16 tumor suppressor protein inhibit cell proliferation; and to identify potential causative SNPs in disease-associated loci.

Overall, CAGI revealed that the phenotype prediction methods embody a rich and diverse representation of biological knowledge, and they are able to make predictions that are highly statistically significant.However, we also found the accuracy of prediction on the phenotypic impact of any specific variant was unsatisfactory and of questionable clinical utility.The most effective predictions came from methods honed to the precise challenge, including the specific genes of interest as well as the problem context.Prediction methods are clearly growing in sophistication, yet there are extensive opportunities for further progress.

Complete information about CAGI may be found at

Biography :

Professor Steven Brenner is currently a full Professor at the Department of Plant and Microbial Biology at the University of California Berkeley. While working at MRC, the Laboratory of Molecular Biology in Cambridge, UK, Dr. Brenner was one of the authors for the paper “SCOP – A Structural Classification of Proteins Database for the Investigation of Sequences and Structures”. According to ISI Science Citation Index, so far the 1996 JMB paper has been cited more than 4000 times! His other highly impact paper, the 2004 Genome Research paper has so far been cited more than 2,300 times. In 2010 he was awarded the Overton Prize from the International Society for Computational Biology. Professor Brenner serves in several international scientific advisory boards, including Human Genome Variation Society and Publications Committee, International Society for Computational Biology. He is also the Founding Editor of “Public Library of Science Computational Biology” and “Editorial Board Member of “Journal of Structural and Functional Genomics”. Professor Brenner’s publication has been cited more than 16,000 times according to ISI Science Citation Index (Web of Science) with h-factor of 43.

Copyright © 2007-2013. All rights reserved.
Microsoft Internet Explorer is recommended if the page is not displayed properly !