Combination therapies have shown to be more effective than single drugs in multiple diseases such as HIV and tuberculosis (Tan et al., 2012; Yilancioglu and Cokol, 2019). Synergistic combinations can improve both potency and efficacy, either achieving stronger therapeutic effect and/or decreasing the dosage thereby reducing side-effects. In the times of the current public health crisis, finding a successful combination of approved molecules have an additional benefit over designing a de-novo molecule: time to clinical adoption. Approved drugs are typically commercially available and have well studied safety profiles. Taken in aggregate, these considerations motivate us to explore combination therapies for SARS-CoV-2 antivirals.
Since exploring the space of combinations via high-throughput screening is prohibitively expensive as it involves combinatorial search, in-silico screening based on machine learning is an appealing alternative. In fact, a number of such methods have been reported in the literature Preuer et al. (2018); Xia et al. (2018). These techniques have been shown effective when the model was provided with large amounts of training data capturing synergy of various combinations. Unfortunately, this requirement prevents us from utilizing these techniques for many diseases where such data is not available. Therefore, it is crucial to reduce data dependence to make combination algorithm applicable in multiple therapeutic contexts.
In this paper, we present a novel algorithm for finding combinations that achieves this goal. Our main hypothesis is that by explicitly modeling interaction between compounds and the biological targets, we can significantly decrease dependence on combination training data. The proposed model has two components. The first component models drug-target interactions (DTI) predicting which targets are inhibited by a compound. This component is trained on individual compounds since DTI information is readily available for multiple targets across multiple diseases. Our second component focuses on modeling target-disease association. Explicitly incorporating this information enables the model to capture the relation between targets that results in synergistic activity.
We further expand this basic architecture to refine our biological modeling. The first such expansion concerns target identification. Since our knowledge of disease relevant targets is often incomplete which may negatively impact model’s predictive power, we augment the target set with additional latent targets learned from single compound and combination data. Our second extension focuses on multi-disease training. Since screening data for combinations is typically sparse, we exploit additional combination data from related diseases for better generalization.
We develop our model using single agent and drug combinations of multiple diseases. The model incorporates known biological targets relevant to SARS-CoV-2 (Gordon et al., 2020), as well as the corresponding drug-target activity data collected from ChEMBL (Gaulton et al., 2017). Our model is evaluated on a SARS-CoV-2 combination screen from Bobrowski et al. (2020) and achieves 0.777 AUC in terms of drug synergy prediction. Moreover, we demonstrate that incorporating known COVID targets and multi-disease training yields 50% relative increase in model accuracy. Finally, we show biological interpretation of the latent targets learned by the model.
2 Related Work
Existing approach on drug synergy prediction can be roughly divided into two categories:
have trained a deep neural network on a large-scale oncology screenO’Neil et al. (2016) (23K training examples) to predict anti-cancer drug synergy. Xia et al. (2018) and Sidorov et al. (2019) have trained deep neural networks to predict anti-cancer drug synergy on a larger dataset compiled by NCI-ALMANAC Holbeck et al. (2017), which contains around 300K training examples across 40 different cell lines.
Biological networks: Another category of drug synergy models builds on biological networks, assuming that drugs with complementary mechanism of actions are more likely to be synergistic. In particular, Cheng et al. (2019); Zhou et al. (2020) proposed to model synergy using distance measures on drug-target interaction and protein-protein interaction networks.
The major challenge of supervised approaches is the lack of combination data. For many diseases such as SARS-CoV-2 and tuberculosis, the size of drug combination data is very limited (less than 200) Bobrowski et al. (2020); Yilancioglu and Cokol (2019)
. Deep learning methods are prone to over-fitting in this low-resource scenario. Moreover, the number of possible pair-wise combinations grows quadratically with the number of drugs. In fact, the current largest combination screen for cancerHolbeck et al. (2017) only covers around 100 different drugs. This significantly limits the ability of trained models to generalize to new drugs outside of the training set. On the other hand, while network-driven methods have a wider coverage over the chemical space, they cannot make predictions on new compounds outside of the biological network (i.e., no edge between this compound and the targets) since the model is not parametric.
We propose a new method that combines the merit of both approaches while addressing their limitations. As drug interaction is often characterized by their biological targets, we train our model to predict both drug-target interaction and drug synergy. This enables us to make predictions on new compounds even if their drug-target interaction is unknown. This also addresses the data scarcity challenge since there are large amounts of drug-target interaction publicly available.
In this section, we describe our model architecture for drug combinations. A drug combination is called synergistic if its antiviral effect is greater than the sum of the individual effects. Synergy arises from various types of drug interaction. For example, two drugs can be synergistic when they interact with different sets of biological targets or pathways. Indeed, most of the anti-HIV drug combinations, such as Dolutegravir and Lamivudine, are drugs with different mechanisms of actions (i.e., interacting with different biological targets). To account for this inductive bias, it is crucial to model the interaction between drugs and biological targets in our model architecture.
Motivated by these observations, we propose to decompose our model into two parts: a drug-target interaction (DTI) module and a target-disease association module . The DTI module predicts the biological targets activated by a given compound. The target-disease association module decides the importance of a biological target to the disease. The vocabulary of biological targets are determined based on its relevance to the disease treatment given by expert knowledge. To introduce our method, we first describe how these two modules are used to predict antiviral activity of single compounds and then extend it to drug combinations.
3.1 Single Compound Model
Each drug is represented as a graph , whose nodes represent atoms and edges represent bonds. Each node/edge is labeled with its atom/bond type. To predict the antiviral effect of a single compound , our model needs to accomplish two tasks: 1) it has to identify the interaction between and a list of biological targets ; 2) decide the relevance of each target to the disease.
into a continuous vector through directed message passing operationsYang et al. (2019), which associate hidden vectors with each node and updates these vectors by passing messages over edges . The output of is a vector representing the biological targets activated by drug :
is a sigmoid function andis a two-layer feed-forward network. Each element
represents the probability of drugbeing active to target . Each target is associated with a drug-target interaction dataset , where if a drug is interacts with target . We will train this module on the DTI dataset of all biological targets in our vocabulary.
Target-disease Association We parametrize the target-disease association module as a simple linear layer due to its interpretability. As shown in Figure 1, our model predicts its antiviral activity of a single drug as:
3.2 Drug Combination Model
Synergy are often quantified under Bliss independence assumption Bliss (1939): suppose the individual antiviral effect of drugs are . The expected effect of combination is given as . The drug pair is synergistic if its observed effect . Following this definition, we introduce a new Bliss layer to predict the synergistic effect of a drug combination . Given two drugs and their predicted DTI vectors , the Bliss layer computes the DTI vector as
where stands for element-wise multiplication. With this aggregation function, a drug combination will benefit most from complementary targets. If only one drug is active to target (e.g., ), the combination is still active to (). In other words, the set of active targets for is the union of active targets of the two drugs.
As shown in Figure 2, for a drug combination , our model predicts its antiviral activity as:
Following Bliss independence model, we predict the synergy score of a combination as , where . Intuitively, a combination is more likely to be synergistic if they have complementary targets with high target-disease association score.
3.3 Training Loss
Let be a cross-entropy loss. Our training loss consists of three components:
DTI Loss: The drug-target interaction loss enforces the predicted DTI vector to be biologically accurate. The corresponding training data is the union of DTI datasets . For each drug and its DTI label , the DTI module is trained to minimize:
Single Compound Loss: Our second training data contains molecules and their labeled antiviral activity (active/inactive). Both modules are trained to minimize
Combination Loss: The third training data contains drug combinations and their labeled synergy. means is synergistic and additive or antagonistic. We train both modules to minimize
We further introduce two extensions to the basic architecture described above that lead to notable performance improvement in practice:
Latent Targets In order to learn synergy, it is important to incorporate all the relevant biological targets into our model. However, this is challenging for two reasons: First, many biological targets do not have drug-target interaction data and thus cannot be incorporated in our model. Second, the current biological understanding of a disease may be incomplete. For instance, Riva et al. (2020) reported around 50 new biological targets that contributes to anti-SARS-CoV-2 activity, but they are not reported in the previous work by Gordon et al. (2020).
To this end, we propose to create additional latent targets that can be learned indirectly from single compound and combination data. Specifically, we allow the dimension of to be greater than the total number of considered targets in . The first entries in corresponds to the real biological targets and the other entries are latent targets. As we will show in the experiments, it is possible for us to identify new biological targets related to given diseases.
Multi-Disease Training Another challenge is the scarcity of drug combination data. To our best knowledge, there are only around 180 drug combinations with labeled synergy against SARS-CoV-2. To address the low-resource challenge, we utilize drug synergy data from other viral diseases such as HIV. In particular, we augment our model with additional HIV biological targets as well as HIV single-drug and drug combination data. The HIV related datasets will be elaborated in Section 4.
The DTI module now outputs a DTI vector for each disease . is shared across two diseases and trained to learn drug-target interaction for all diseases. Since each disease operates on different targets, we create a target-disease association module for each disease. Let be the losses for each disease computed from its own dataset. Our multi-disease training loss becomes
SARS-CoV-2 Data For SARS-CoV-2 infection, we consider three types of biological targets in our target vocabulary :
Viral proteases: Replication of SARS-CoV-2 virus requires the processing of two polyproteins by two virally encoded proteases: chymotrypsin-like protease (3CLpro) and papain-like protease (PLpro). Inhibitors that block either protease could inhibit viral replication. We have compiled 3CLpro enzymatic activity (NCATS) (2020c) and PLpro inhibition Davis-Gardner (2020) data made public by NCATS and ReframeDB.
Viral entry proteins: SARS-CoV-2 cell entry depends on angiotensin converting enzyme 2 (ACE2) Hoffmann et al. (2020). Inhibiting ACE2 enzyme or the interaction between SARS-CoV-2 and ACE2 could block viral entry. To this end, we utilize ACE2 enzymatic activity (NCATS) (2020a) and Spike-ACE2 protein-protein interaction (NCATS) (2020e) from NCATS.
Host proteins: Gordon et al. (2020) identified 335 human proteins physically associated with SARS-CoV-2 viral proteins. Inhibitors for these proteins may also hinder viral replication. Among these proteins, we selected 31 proteins that have sufficient amount of drug-target interaction data in the ChEMBL database (i.e., both positive and negative interactions).
The above drug-target interaction data contains around 20K compounds in total. Our training data for SARS-CoV-2 utilizes another two assays:
Single-agent Activity: We use the NCATS CPE assay in VeroE6 cells ((NCATS), 2020d), which contains around 10K compounds and 320 hits with EC50 M.
HIV Data The training data for HIV comes from the following assays:
Drug-target Interaction: Existing anti-HIV drugs mainly target viral proteins (HIV-1 protease, integrase and reverse transcriptase) or host proteins involved in viral entry (CCR5, CXCR4 and CD4). We compiled DTI data for these six targets from ChEMBL.
Single-agent Activity: NCI conducted an anti-HIV assay (NCI) (2004) with 35K compounds, among which 309 compounds are active (EC50 M).
Drug Combination: Tan et al. (2012) conducted high-throughput screen for HIV drug combinations. The dataset contains 114 two-drug combinations.
Evaluation Protocol Since our goal is to predict synergy against SARS-CoV-2, our validation and test set only consist of SARS-CoV-2 combinations. All the drug-target interaction, single-drug activity and HIV data are used for training only. Our validation set contains 20 combinations from Riva et al. (2020) and test set contains 72 combinations from Bobrowski et al. (2020). The training set contains 90 SARS-CoV-2 combinations from (NCATS) (2020b), where we remove combinations that appear in both the training and test set.
Hyperparameters For DTI module
, we adopt default hyperparameters fromYang et al. (2019), with hidden dimension 300 and three message passing iterations. We set the dimension of DTI vector , with 42 real biological targets (SARS-CoV-2 and HIV) and 58 latent targets, so that the number of real and latent targets are roughly equal. We set for our final model.
To show the effectiveness of different components, we compare with the following baselines:
A GCN trained only on SARS-CoV-2 single-agent and combination data ().
+DTI: A GCN trained only on SARS-CoV-2 single-agent, combination as well as drug-target interaction data ().
+MultiD: A GCN trained on both SARS-CoV-2 and HIV data (single-agent + combination), but without drug-target interaction data ().
+All,-latent: A GCN trained on both SARS-CoV-2 and HIV data (single-agent + combination + drug-target interaction), but the latent targets are removed ().
+All: A GCN trained on both SARS-CoV-2 and HIV data (single-agent + combination + drug-target interaction), but the latent targets are removed ().
Our results are shown in Figure 3. As expected, the GCN baseline performs poorly, with AUC. Adding drug-target interaction data (+DTI) improves the AUC to . Adding HIV data (+MultiD) improves the AUC to . Our final model, trained with both HIV and drug-target interaction (+All), achieves the best AUC of . This validates the advantage of adding drug-target interaction data and multi-disease training. Note that if we remove the latent targets (+All,-latent), the performance decreases to . This also shows the importance of using latent targets to complement missing biological information.
4.2 Latent Target Interpretation
While the latent targets are trained to represent biological targets, we found that they are predictive of certain SARS-CoV-2 biological targets that are not provided during training. Therefore, we propose to interpret their biological meaning by the following procedure:
Suppose we have a list of real SARS-CoV-2 targets that are not provided to the model during training (i.e., ). Let be the drug-target interaction dataset for .
Compute DTI vectors for each compound . For each latent target , compute the AUROC between DTI labels and predicted DTI scores .
Match latent target to target if the AUROC is higher than threshold .
The target set contains 34 biological targets reported in Riva et al. (2020) that are correlated with SARS-CoV-2 antiviral activity and have drug-target interaction data in the ChEMBL database. These targets are not provided to the model during training. As shown in Figure 3, among the 58 latent targets, 27 of them are matched to the antiviral targets in Riva et al. (2020) using the above procedure. This shows that our model is capable of capturing additional SARS-CoV-2 targets through single-agent and combination data.
In this paper, we present a new algorithm for predicting SARS-CoV-2 drug synergy. The model consists of two components: a drug-target interaction module and target-disease association module. Our method requires much less combination data since we explicitly model the drug-target interaction in our model architecture. The proposed method is general and applicable to many other diseases.
- ACE2 enzymatic activity. Note: https://opendata.ncats.nih.gov/covid19/assay?aid=6 Cited by: 2nd item.
- In vitro sars-cov-2 evaluation of drug combinations for exploring chemical biology and potential therapeutic use. Note: https://tripod.nih.gov/matrix-client/?project=2983 Cited by: 2nd item, §4.
- SARS-cov-2 3cl protease enzymatic activity. Note: https://opendata.ncats.nih.gov/covid19/assay?aid=9 Cited by: 1st item.
- SARS-cov-2 cytopathic effect (cpe) screening. Note: https://opendata.ncats.nih.gov/covid19/assay?aid=14 Cited by: 1st item.
- SARS-cov-2 spike-ace2 protein-protein interaction (alphalisa). Note: https://opendata.ncats.nih.gov/covid19/assay?aid=1 Cited by: 2nd item.
- AIDS antiviral screen data. Note: https://wiki.nci.nih.gov/display/NCIDTPdata/AIDS+Antiviral+Screen+Data Cited by: 2nd item.
- The toxicity of poisons applied jointly 1. Annals of applied biology 26 (3), pp. 585–615. Cited by: §3.2.
- Discovery of synergistic and antagonistic drug combinations against sars-cov-2 in vitro. BioRxiv. Cited by: §1, §2, 2nd item, §4.
- Network-based prediction of drug combinations. Nature communications 10 (1), pp. 1–11. Cited by: 2nd item.
- SARS-cov-2 plpro inhibitor assay. Note: https://reframedb.org/assays/A00461 Cited by: 1st item.
- Convolutional networks on graphs for learning molecular fingerprints. In Advances in neural information processing systems, pp. 2224–2232. Cited by: §3.1.
- Identification of inhibitors of sars-cov-2 in-vitro cellular toxicity in human (caco-2) cells using a large scale drug repurposing collection.
- The chembl database in 2017. Nucleic acids research 45 (D1), pp. D945–D954. Cited by: §1.
- Neural message passing for quantum chemistry. arXiv preprint arXiv:1704.01212. Cited by: §3.1.
- A sars-cov-2 protein interaction map reveals targets for drug repurposing. Nature, pp. 1–13. Cited by: §1, §3.4, 3rd item.
- Network medicine framework for identifying drug repurposing opportunities for covid-19. arXiv preprint arXiv:2004.07229.
- SARS-cov-2 cell entry depends on ace2 and tmprss2 and is blocked by a clinically proven protease inhibitor. Cell 181 (2), pp. 271–280. Cited by: 2nd item.
- The national cancer institute almanac: a comprehensive screening resource for the detection of anticancer drug pairs with enhanced therapeutic activity. Cancer research 77 (13), pp. 3564–3576. Cited by: 1st item, §2.
- Adaptive invariance for molecule property prediction. arXiv preprint arXiv:2005.03004.
- Domain extrapolation via regret minimization. arXiv preprint arXiv:2006.03908.
- Die quantitativen probleme der pharmakologie. Ergebnisse der Physiologie 27 (1), pp. 47–187.
- An unbiased oncology compound screen to identify novel combination strategies. Molecular cancer therapeutics 15 (6), pp. 1155–1162. Cited by: 1st item.
- DeepSynergy: predicting anti-cancer drug synergy with deep learning. Bioinformatics 34 (9), pp. 1538–1546. Cited by: §1, 1st item.
- Discovery of sars-cov-2 antiviral drugs through large-scale compound repurposing. Nature, pp. 1–11. Cited by: §3.4, Figure 3, 2nd item, §4.2, §4.
- Predicting synergism of cancer drug combinations using nci-almanac data. Frontiers in chemistry 7, pp. 509. Cited by: 1st item.
- Systematic identification of synergistic drug pairs targeting hiv. Nature biotechnology 30 (11), pp. 1125–1130. Cited by: §1, 3rd item.
- Predicting tumor cell line response to drug pairs with deep learning. BMC bioinformatics 19 (18), pp. 71–79. Cited by: §1, 1st item.
- Analyzing learned molecular representations for property prediction. Journal of chemical information and modeling 59 (8), pp. 3370–3388. Cited by: §3.1, §4.
- Design of high-order antibiotic combinations against m. tuberculosis by ranking and exclusion. Scientific reports 9 (1), pp. 1–11. Cited by: §1, §2.
- Network-based drug repurposing for novel coronavirus 2019-ncov/sars-cov-2. Cell discovery 6 (1), pp. 1–18. Cited by: 2nd item.