sampl.eyesopen.com

Welcome to the SAMPL website. Please continue reading for a more detailed description of what SAMPL is, the current challenges in SAMPL3, and a look back at previous SAMPLs. If you decide to participate in SAMPL3, please get started by creating a new account using clicking the link on the right side of the page, below the login fields.

The SAMPL experiment

SAMPL is an attempt at prospectively testing protein and ligand modeling. Ideally this would consist of experiments conceived to distinguish between competing ideas or methods and perhaps over time we shall get there. For now, this is an analysis of methods applied to data not seen by participants, a 'blind' assessment. We make no claims against the limitations of such an attempt, believing it more important to light a candle than curse the darkness. And blind tests do help us avoid the tendency to bias our theories and approaches to known answers. They also provide a more realistic "real world" setting for methods.

With SAMPL we intend to avoid "who won, who lost". On such a small sample size, no such statistically sound pronouncement could be made anyway. Rather, we see this as an opportunity for groups to test their methods, learn from the experience and share lessons learned.

SAMPL3

For SAMPL3 we intend to return to protein-ligand interactions as well as continue our work with vacuum-water transfer energies. In addition, for the first time for SAMPL we intend to have host-guest systems through a collaboration with Mike Gilson of UCSD, Lyle Isaacs of the University of Maryland, and Adam Urbach of Trinity University. Host-guest systems consist of molecular ‘containers’ that can bind slightly smaller ligands and, as such, lie conveniently half way in complexity between simple solvation energy calculations and full protein-ligand interactions. We are very excited to be expanding SAMPL in this direction and may yet further extend the set of systems in the near future. Our protein-ligand systems are to be provided from the DINGO dataset developed by Tom Peat, Janet Newman (CSIRO, Melbourne), Kim Branson (Vertex), and others (http://www.ncbi.nlm.nih.gov/pubmed/19822883). This blinded fragment-screening study will provide both crystallographic and affinity data for challenges focused on virtual screening, pose prediction, and affinity prediction. And as usual, we shall have a set of cunningly culled small molecule transfer energies from Peter Guthrie (U. Western Ontario), this time focusing on halogenated molecules.

The date of the full workshop is set for August 1st and 2nd, 2011 at Stanford, hosted by Vijay Pande, our long time collaborator and contributor to SAMPL. The deadline for data submission is Thursday, June 30, 2011. More details will be available soon but feel free to contact us with your questions and interest.

We will again follow an 'opt-in' policy for any publication, i.e. the results from any participants can be used but attribution is by permission. We hope this may encourage some who, perhaps correctly, feel they have more to lose than gain. Simply put, participants can choose to remain anonymous. Details can be found in the sign-up agreement.

Previous SAMPLs

At CUP8, in 2007, we had a small taste of blinded prospective evaluation data sets on vacuum-water transfer and protein-ligand binding. We were fortunate to have Vijay Pande, Peter Guthrie and Anthony Nicholls discuss transfer energies and Hideaki Fujitani, Scott Brown and Marti Head discuss protein-ligand binding predictions. That was enough to demonstrate the value of this exercise. In addition, the work on solvation prediction appeared in J. Med. Chem. in 2008 (Nicholls et. al, JMC 2008 51(4):769-779 http://pubs.acs.org/doi/abs/10.1021/jm070549%2B).

SAMPL1, held in winter 2007-2008 consisted of two sets of protein-ligand binding data, generously provided by Abbott Labs and Vertex Pharmaceuticals, and sixty three vacuum-water transfer energies, courtesy of Peter Guthrie at the University of Ontario, and CCG. The two sets of protein-ligand data each have between twenty and sixty active compounds, each with a protein-ligand crystal structure. The protein-ligand assessment consisted of three parts: virtual screening, pose prediction and affinity prediction. The vacuum-water transfer energies are provided by Peter Guthrie. SAMPL1 had over 50 participants who submitted over 200 data samples. Approximately half the participants were from industry and half were from academic or government labs. The results of SAMPL1 were discussed March 19th at CUP9, including presentations from Vijay Pande of Stanford, Marti Head of GSK, Enrico Purisima of BRI-NRCC and Dave Mobley of UCSF. A reprise of the results was also presented at the GRC, ACS meetings in the US and the ICCS meeting in Europe. Finally, Peter Guthrie has organized a recently-published special issue of JPC that focuses on the vacuum-water transfer results from SAMPL1. (http://pubs.acs.org/toc/jpcbfk/113/14)

SAMPL2, which took place during the spring of 2009, consisted of transfer energy and tautomer ratio predictions. Once again, Peter Guthrie organized the transfer energy data, and Peter Taylor, formerly from Astra-Zeneca, gathered the tautomer ratio data. Over 65 submissions were made from almost 20 different individuals or groups, including several participants who had taken part in previous SAMPLs. A new aspect of SAMPL2 was a dedicated workshop in Montreal, which took place in June, where all participants were encouraged to convene and discuss our results and performance as a field as well as focus in on individual cases of interest. As with previous years, SAMPL2 resulted in a special issue of JCAMD with several publications by participants covering both transfer energies and tautomer ratios. (http://www.springerlink.com/content/0920-654x/24/4/)

SAMPL in the Future

We hope SAMPL continues to be useful and regular event. There are many possible variants we would like to try in future assessments, such as providing partial information, e.g. a subset of actives or binding modes, more typically of an industrial project setting. Additional physical properties, such as tautomer ratios, pKas, thermodynamic data could be sought. Of course, these efforts rely on the availability of appropriate, blinded data, which is often difficult to obtain. We strongly believe a blinded assessment component to our field will help us judge progress and hope you both agree and can contribute.

Kind Regards,
Matt Geballe, Geoff Skillman & Anthony Nicholls
OpenEye Scientific Software, Inc.