In this collection of reviews from top scientists in academia and industry, we hope to present the reader with more than simply a survey of the current state of the field in high throughput screening (HTS) and related technologies. Such reviews, while useful for a brief period of time, are quickly surpassed by changes in the field. This may be due to the discovery and development of new systems adapted from nature (e.g. gene editing techniques), which provide researchers with powerful new tools that make previously impossible studies relatively routine. Changes may also reflect simple incremental improvements or more robust commercialization of existing technologies that make them more readily available to researchers in a plug and play format (e.g. new fluorophores or luciferase enzyme systems).

Why, then, assemble such a survey of HTS methods? By examining the state of the art in relation to how we arrived here and considering what areas still require improvement, we hope to also encourage our readers to consider the philosophy underlying technological change. How and why are improvements in technology made? Surely they reflect an underlying demand from researchers to be able to generate results more quickly and answer questions more efficiently or more robustly.

Small-molecule activity screening, considered as a specific discipline practiced initially within the pharmaceutical industry, and now more widely available to academics and biotechnology firms, could be considered to have started in the mid-1900s. This involved the routine parallel testing of natural product extracts and purified compounds, as well as synthetically available compounds from related industries, in cellular phenotypic assays such as microbial viability systems or whole animals.¹  It became more robust and standardized with the shift towards target-based enzymatic assays, large scale combinatorial synthetic compound collections, and robotic automation in the 1990s, driven in part by genomic studies drastically increasing the number of putative therapeutic targets.²  This new approach required significant capital investment, taking it out of reach of all but the largest companies. However, in the 2000s, through the efforts of government funding agencies as well as the establishment of contract research organizations offering HTS as a service, a democratization occurred in the field of small-molecule discovery. This change led both to more routine application of screening to early biological targets as well as to the development of a larger variety of assays for measuring biology through alternative and ideally more relevant methods.

Consider, then, the shifts over the years in the approach to small-molecule discovery. Why apply automation, previously used in areas such as industrial manufacturing, to increasing the number of measurements made? What demands did this increase in throughput have on the related biology, assays, and number and type of compounds required to feed the system? What is the most effective use of small-molecule discovery in academia? Surely these questions are all affected by larger societal and technological changes, including the information revolution and political considerations that affect funding decisions. They also relate to the technologies themselves—each technology is designed to improve some shortcoming in the existing capabilities, but in turn can lead to its own problems.

Figure 1 shows the evolution of methods for addressing four key components of HTS: chemical perturbagens, bioassays, data analysis and management, and organizational infrastructure. For example, with the increase in throughput of cell free enzymatic and cell based reporter assays, more compounds were required. These were accessed through new methods in high throughput chemical synthesis, which greatly increased the number of compounds available but did not always consider the optimal chemical diversity necessary for a range of biological targets.³  Analysis of the desired properties of compounds changed,⁴  with many pharmaceutical companies paring back their collections by 30% or more from their peaks.⁵  Alternative approaches, such as encoded libraries and diversity oriented synthesis, changed the types of libraries available. Lower throughput but arguably more biologically relevant assays, such as high content imaging and activity profiling, changed the nature of the information available.²  In turn, this led to new requirements, as phenotypic assays required systematic methods for identification of cellular targets. Overall, of course, these different approaches have different requirements for capital and operational investment, and all scientific research remains a trade-off between available funding and hoped for return in the understanding of basic biology and the impact on improvements in human health.

Given these historical and ongoing changes, no one would argue that we are at the ideal state of HTS discovery now. This book is a snapshot of the state of the field in 2016, as well as a historical survey of how the methods presented arrived at their current state of the art. Our contributors also highlight weaknesses and potential solutions to further improve the field. This, then, is our hope and challenge for our readers: we seek to provide a means of understanding how and why we have arrived where we are, through the above-mentioned combination of technological and societal changes. With that understanding, we hope to illuminate the way forward—what changes are necessary, what impact will they have, how can they be implemented practically and what future challenges will those changes in turn bring about?

We cannot predict or understand in a single book written at a single time all of the technologies yet to come. However, by providing the reader with a means of understanding how and why technological change occurs, as illustrated by the evolution and refinement of HTS methods, we hope that future capabilities can more readily be anticipated, developed, applied and further improved, to the benefit of researchers and society.

Joshua Bittker and Nathan Ross

Cambridge, MA, USA