- About this Journal ·
- Abstracting and Indexing ·
- Aims and Scope ·
- Annual Issues ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents
BioMed Research International
Volume 2013 (2013), Article ID 726239, 11 pages
Immune Monitoring in Cancer Vaccine Clinical Trials: Critical Issues of Functional Flow Cytometry-Based Assays
Department of Hematology, Oncology and Molecular Medicine, Istituto Superiore di Sanità, 00161 Rome, Italy
Received 20 June 2013; Accepted 19 August 2013
Academic Editor: Jelena Urosevic
Copyright © 2013 Iole Macchia et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
The development of immune monitoring assays is essential to determine the immune responses against tumor-specific antigens (TSAs) and tumor-associated antigens (TAAs) and their possible correlation with clinical outcome in cancer patients receiving immunotherapies. Despite the wide range of techniques used, to date these assays have not shown consistent results among clinical trials and failed to define surrogate markers of clinical efficacy to antitumor vaccines. Multiparameter flow cytometry- (FCM-) based assays combining different phenotypic and functional markers have been developed in the past decade for informative and longitudinal analysis of polyfunctional T-cells. These technologies were designed to address the complexity and functional heterogeneity of cancer biology and cellular immunity and to define biomarkers predicting clinical response to anticancer treatment. So far, there is still a lack of standardization of some of these immunological tests. The aim of this review is to overview the latest technologies for immune monitoring and to highlight critical steps involved in some of the FCM-based cellular immune assays. In particular, our laboratory is focused on melanoma vaccine research and thus our main goal was the validation of a functional multiparameter test (FMT) combining different functional and lineage markers to be applied in clinical trials involving patients with melanoma.
The primary objective of immune monitoring in cancer vaccine clinical trials is to find a correlation between the efficient induction of tumor-specific T-cell responses and clinical efficacy, due to the importance of the host immune system in controlling tumor progression. However, although in several cancer vaccine trials there is indication of increased frequency of tumor-specific T-cells, no validated biomarkers exist for cancer immunotherapy as yet .
One reason for the lack of correlation between the immune parameters measured and objective clinical response might be the complexity of the immune responses required for a successful tumor eradication which cannot be dissected through the most frequently used T-cell assays.
Antitumor cell-mediated immunity is a key biomarker for most vaccines and immunotherapies and involves the activity of specialized cells including antigen specific cytotoxic T lymphocytes (CTLs)  and CD4+ helper T lymphocytes  as well as components of innate immunity such as macrophages, dendritic cells (DC), natural killer cells (NK), and granulocytes.
In addition, since the success of immune response against cancer is due to the balance between the effector and the suppressive compartments, immunological monitoring should also include analysis of immune suppressive cells (ISCs) such as regulatory T-cells (TREGs), myeloid-derived suppressor cells (MDSCs), and tumor-associated macrophages (TAMs) [4–6]. These cells play an important role in cancer progression and responses to immunotherapy. However, due to their tremendous phenotypic and functional heterogeneity, their usefulness as biomarkers of outcome or response to therapy has to await for further development of monitoring assays that better reflect their biologic significance in cancer.
In addition, a recent report  indicates that regulatory B cells (BREGs), a small subset of suppressor cells, may have profound effects on the development of T-cell responses, further complicating the interpretation of antitumor immune suppression in disease.
Local and systemic antitumor immune responses can show markedly different patterns and many functional responses could be missed when only peripheral blood lymphocytes (PBLs) and not tumor infiltrating lymphocytes (TILs) are evaluated; therefore, more emphasis should be put on immune monitoring also at the effector site by taking a biopsy of a metastatic lesion . To this end, the concept of “Immunoscore,” initially described for colorectal cancer patients, has been recently introduced as an essential prognostic and potentially predictive tool to classify cancers, beside the traditional tumor staging classification (AJCC/UICC-TNM) [9–13]. This parameter might facilitate clinical decision making including rational stratification of patient treatment. Usually the immunoscore approach refers to the analysis of the location, density, and functional orientation of different immune cell populations infiltrating the tumor, including macrophages, DC, mast cells, NK cells, naïve and memory lymphocytes, B cells, and T lymphocytes (which include various subsets of T-cell: TH1, TH2, TH17, regulatory T-cells (TREGS), T follicular helper cells (TFH), and cytotoxic T-cells). Such “immune contexture” annotated in a large collections of human tumors has allowed the identification of components that are beneficial for patients and those that are deleterious . For instance, in a study by Pagès et al. , high densities of T-cells (CD3+), cytotoxic T-cells (CD8+), and of memory T-cells (CD45RO+) were clearly associated with a longer disease-free survival (DFS) (after surgical resection of the primary tumour) and/or overall survival (OS).
However, the analysis of tumor microenvironment is not always feasible and the only samples available are those obtained from peripheral blood. Furthermore, sometimes peripheral responses should be of some relevance and could integrate and increase information given by the tumor microenvironment. For this reason, we believe that, in addition to scoring T-cells at tumor sites, the frequency and functions of T-cells circulating in the peripheral blood of cancer patients should be examined as potential biomarkers, by means of validated and standardized immune assays.
Concerning the quality of T-cell response, several papers showed that the multifunctionality of effector cells is an important factor to predict the immunological protection . In particular, it has been demonstrated that the functional profile of HIV-specific CD8+ T-cells in progressors is limited compared to that of nonprogressors, which consistently maintain highly functional CD8+ T-cells and that the frequency and proportion of the HIV-specific T-cell response with highest functionality inversely correlates with viral load in the progressors . In addition, other reports indicated that vaccine-induced multifunctional CD4+ and CD8+ T-cells produce greater amounts of IFN-γ than cells that secrete IFN-γ alone .
In the setting of cancer immunotherapy, the induction of polyfunctional NY-ESO-1-specific T-cell responses, following anti-CTLA-4 treatment of metastatic melanoma patients, has been recently shown to enhance T-cell responses and to induce durable clinical responses . Further, a recent paper  demonstrated that the triple combination of IFN-γ, IL-2, and TNF-α represents a Th1 pattern of polycytokine secretion with greater antigen sensitivity and superior tumor recognition.
In order to get new insights in exploitation of vaccine-induced polyfunctional T-cells, standardization and validation of multiparameter assays are required.
In this review, we will overview the current technology used for immune monitoring during cancer immunotherapy in melanoma patients, focusing on a polychromatic FCM-based approach for ex vivo detection of tumor-antigen specific T lymphocytes producing multiple functional molecules simultaneously. To this aim, we will provide few experimental examples to discuss critical process steps encountered during validation of an FMT developed in our laboratory, consisting of a six-color panel for assessment of polyfunctionality of tumor-specific CD8+ T-cells in cryopreserved human peripheral blood mononuclear cells (PBMCs).
2. Overview of Immunoassays
The objective of any immune monitoring study is to collect interpretable, reliable, and reproducible data for the detection, quantification, and characterization of immune responses directed at specific antigens.
The principal techniques utilized for immune monitoring are reviewed in .
Measuring cytokine production and profile represents an integral part of immune monitoring during immunotherapeutic treatments . First-generation immune-monitoring techniques included proliferation and cytotoxicity assays following short-term in vitro expansion; more recently, tetramer and Elispot (second-generation assays) allowed to assess directly ex vivo the frequency of vaccine specific T-cells and their ability to produce cytokines, usually IFN-γ . However, this type of analyses is limited by the lack of information about the functional state of the cells. In the setting of cancer patients, where tumor escape mechanisms may induce T-cell anergy by altering lymphocyte signalling and effector functions , the need for third-generation assays aimed at evaluating the functional properties of rare cell populations of vaccine-induced T-cells with a multiparameter approach is becoming increasingly evident. To this aim, the development of polychromatic flow cytometry for immune monitoring has significantly contributed to progress in the field of human immunology.
In this paper, we will focus on some of the most widely used FCM-based assays for measurement of antigen-specific T-cells. In particular, intracellular cytokine staining (ICS) represents one of the main FCM-based assays and it has been previously validated by Horton et al. and De Rosa et al. [24, 34].
The CD107 mobilization assay measures the exposure of CD107 (LAMP: lysosomal associated membrane protein) a and b, present in the membrane of cytotoxic granules of CTLs, onto the cell surface as a result of degranulation and it can be used as an alternative to 51Cr release assay. In fact, a good correlation has been demonstrated between degranulation and cytotoxic activity of tumor-specific CD8+ T-cell clones and CD8+ T-cells, as measured in an FCM-based killing assay [35, 36]. Further, CD107-expressing CD8+ T-cells are shown to mediate cytolytic activity in an antigen-specific manner. Soluble major histocompatibility class I tetramers are a widely utilized tool for the direct ex vivo detection, characterization, and isolation of antigen-specific T-cells in a variety of clinical settings such as infectious, autoimmune, or neoplastic diseases [37–41].
To provide a more complete assessment of the functionality of CD8+ T-cells expressing cognate T-cell receptors (TCR), measurement of CD107a and b expression can be combined with MHC-class I tetramer labeling and ICS .
In order to insure reproducibility and worldwide comparisons for conclusive longitudinal monitoring in multicenter studies, standardized operating procedure (SOPs), as well as standardized reagents and analysis protocols, need to be used .
Advances in multiparameter flow cytometric technologies and reagent applications for characterization and functional analysis of cells modulating the immune network have been recently reviewed in .
Researchers from Europe and the United States have started a project called Minimal Information About T-cell Assays (MIATA)  to standardize and harmonize commonly used assays such as the enzyme linked immunosorbent spot assay (Elispot)  and major histocompatibility complex tetramer assays . The establishment of universally accepted guidelines for performing and presenting immunological assays in scientific publications will create a framework that will allow the comparison of immune responses across clinical trials. Many groups performed also optimization and harmonization of intracellular cytokine assays [49–52].
3. Key Issues Involved in FCM-Based Assays and Development of FMT
Flow cytometry is a powerful and versatile technique, ideal for phenotyping, enumerating, and assessing the function of rare and precisely defined cell subsets at the single cell level. Functions assessed by flow cytometry include cytokine/chemokine production, CD107 expression, multimer analysis, natural cytotoxicity, antibody dependent cell cytotoxicity (ADCC), and proliferation. Critical steps for immune monitoring by flow cytometry which may affect yield, viability, and immunologic function of cells, include shipping blood variables such as temperature  and time delay of drawn blood processing, freezing/thawing conditions [17, 19, 51], type of anticoagulant used for blood collection, and type of density-gradient centrifugation used for the isolation of peripheral blood mononuclear cells. Other variables facing multicolor assay depend on antibodies and fluorochromes used, fixation and permeabilization reagents, instrument setup, data acquisition and analysis, reporting of results, internal quality control, external quality assessment, and flow sorting  (Table 1).
The importance of ex vivo analysis versus in vitro analysis has been addressed by .
The principal challenge for FCM-based assays for immune monitoring in cancer clinical trials is often due to the need of detecting rare subsets of cells avoiding the spurious positive events. This goal can be achieved by using a multiparameter approach in order to minimize the false positive and negative events by gating and subgating the cells of interest which express multiple markers simultaneously. Other critical issues are represented by in vitro T-cell culture methods when immune responses are analyzed in expanded T-cell cultures instead of ex vivo. To this aim, optimization of a cell culture method for analysis of polyfunctional T-cells has been previously dissected .
By setting up a procedure to assess polyfunctionality of tumor-associated antigen- (TAA-)specific cells in clinical trials, we observed a reduction of cell number at the end of the experiment, probably due to a loss of cells at different steps of cell processing (unpublished data). These observations led us to initiate a set of controlled in vitro studies to investigate the impact of different reagents and methods on recovery, viability, and immunological function of cells.
Our final goal was the optimization and validation of a reliable method, which we named FMT, for assessment of antigen specificity and effector functions against the melanocyte differentiation antigen Melan-A/MART-1. This protocol, adapted from , was based on a six-color panel combining CD8, MHC-tetramer, CD107a, and intracellular cytokine staining for three soluble factors with distinct properties (CD107, TNF-α, and IL-2), in response to the peptide Melan-A/MART-1 or other stimuli.
In previous experiments, based on previous reports indicating that different fixation/permeabilization buffers may affect the results of intracellular cytokine detection , we performed the FMT for CD107, TNF-α, and IL-2, after stimulation with Staphylococcus Enterotoxin B (SEB) and PHA and we compared two distinct standardized commercial lysing/permeabilization buffers: the Lysing/Perm solutions and the Intrasure kit (both purchased from BD Biosciences) (Figure 1).
Overall, we found that the fixation/permeabilization with the Intrasure kit resulted in a stronger response for all the parameters analyzed and that stimulation with SEB yielded the higher percentages of CD8+ T-cells producing one, two, or three factors (Figure 1). Based on these results, we decided to use in our next experiments the Intrasure permeabilization kit and SEB as positive control.
Next, we investigated the impact of DNAse, which is usually used to digest extracellular DNA and reduce cellular clumping, on cell recovery and viability as well as its effect on cell function (Table 2).
Compared to previous reports  facing this issue, we try to keep the DNAse during the all steps of the FMT procedure, from thawing to culturing, even during washing steps.
Our results indicated that using DNAse after thawing PBMCs samples and during the entire procedure increased the absolute number and percentages of CD8+/MART-1+ cells as shown in Table 2 that summarizes the effect of DNAse on cell recovery at the end of FMT, by enumerating TAA-specific (Melan-A/MART-1) CD8+ T-cells in the presence or not of DNAse (DN25- SIGMA).
Soluble tetrameric MHC/peptide complexes have opened the possibility to directly identify and monitor antigen-specific CD8+ T-cells at the tumor site and in blood . Multiparameter monitoring of antigen-specific T-cell responses that combines ex vivo tetramer staining with various phenotyping and functional assays provides a novel approach to assess the functional potential of tumor-specific T lymphocytes and may also facilitate the optimization of vaccination protocols.
Dextramers are multimers based on a dextran backbone bearing multiple fluorescein and streptavidin moieties, used for the analysis of relatively low frequency antigen-specific T-cells in peripheral blood. The functionality and optimization of dextramers have been previously demonstrated on human CD8+ T-cell clones with four independent antigen specificities .
Staining of a CD8+ line from a healthy donor with either MART-1-specific tetramers or pentamers or dextramers shows that dextramers produce a stronger signal against Melan-A antigen and a lower background signal than their tetramer and pentamer counterparts (Figure 2). Thus, dextramers could become the reagents of choice as the antigen-specific T-cell labeling transitions from basic research to clinical application.
Finally, validation of the FMT for analysis of the functionality of T-cells directly ex vivo was performed on a melanoma patient with discrete percentage of CD8+ MART-1+ specific T-cells.
On our side, our FMT experiments were acquired on a BD-Canto instrument by DIVA software. We chose to analyze them by a classical approach, using a standard software dedicated to flow cytometry analysis, Flow Jo (Treestar, MA, USA) and generating graphical representation by Excel (Microsoft, WA, USA) elaboration. Gating strategy might have a potential impact on the analysis of antigen-specific polyfunctional T-cell responses. In our setting, our population of interest was defined meeting the criteria of a lymphocyte morphology, based on forward- and side-scatter parameters, singlet morphology, based on forward height scatter and forward area scatter, positivity of surface antigen expression (tetramer and CD8). Expression of 4 parameters, CD107a and the intracellular cytokines, IL-2, IFN-γ, and TNF-α, was simultaneously investigated by a 6-colors staining to assess polyfunctionality of the gated population. This sequential gating strategy is shown in Figure 3(a), along with some bidimensional plots showing some of the possible representation of parameters under study.
The possible combinations of positivity/negativity of these 4 parameters generate a large number of variables (30 for each sample). An effective way to give a graphic representation of such a lot of variables is to use histograms and pie chart (Figure 3(b)). Pie charts give a quick shot of the proportion of responding cells producing one or more functions without specifying which is the particular function . We drew these graphics by elaborating FlowJo results export using a standard Excel (Microsoft, WA, USA) worksheet, adapting the Simplified Presentation of Incredibly Complex Evaluations software’s approach (SPICE, Version 2.9, Mario Roederer, Vaccine Research Center, NIAID, NIH), one of the most largely utilized free Apple Mac-based data mining software [54, 58, 59].
4. Tools and Software for Analysis of Flow Cytometric Data
Traditional methods to analyze flow cytometric data involve gating of populations in one- or two-dimensional displays and manually selecting populations of interest. However, such methods are highly subjective and time consuming. Particularly, with the advent of multiple lasers flow cytometry analyzer, it is possible to have up to 18 colors of fluorescence detection simultaneously in the same sample. This leads to an enormous amount of variables, due to the all possible combinations of each parameter acquired. So that, critical is the analysis approach: bioinformatics will be surely the way to manage this kind of data in the next future. Looking back to the classical way to analyze FCS files, by manual, sequential gating, in the past years an enormous number of dedicated software has been developed by industries and academies (most of these last being freeware) .
Just to cite the most common among them, BD-DIVA, Miltenyi-MACSQuantify, Millipore-GuavaSuite, (acquisition and analysis commercial software, being interfaces of flow cytometer), FlowJo (one of the most common analyser software for flow cytometry), BeckmanCoulter-Kaluza, Weasel (commercial analysis software), and WinMDI, (free academic analysis software); each of them is endowed with peculiar tools and utility.
A detailed list of cytometry software and educational materials in cytometry is provided by the “original cytometry software catalog,” developed and managed by Dr. Eric Martz and by the Purdue University Cytometry Laboratories (http://www.cyto.purdue.edu/flowcyt/software/Catalog.htm).
Reviewing the new computational approach of analysis, often based on automated gating and high level of statistical analysis and representation output, several groups have developed different strategies.
Among them, open source tools like Bioconductor flowFlowJo, able to extracting information from a FlowJo workspace and deliver the data into (one of the most common statistical processor) in the flowCore paradigm, have been developed to allow the management of high throughput data .
Probability binning algorithm extensively described by Roederer et al. in  is today a powerful tool employed, that is, to define subpopulation of cells, by means of-high resolution phenotype .
One unique approach, an algorithm called SPADE, utilizes downsampling, clustering, minimum spanning tree, and upsampling algorithms to generate two-dimensional branched visualizations [64, 65]. The branched tree structure incorporates information from all measurements in the data, partially addressing scalability issues. However, SPADE has many of the same subjective inputs as conventional clustering algorithms (e.g., number of clusters) and also may have issues of reproducibility and generation of nonbiological branches.
Similar to the SPADE software, the Euroflow Consortium software called INFINICYT uses nearest-neighbor analysis to associate the data around the center of the mass of cells. Adopting Euclidean distance analysis, this software associates a normal profile for a cell type (through phenotyping of multiple normal samples) to identify and characterize an abnormal profile (http://www.infinicyt.com/). Developed as a diagnostic tool, this approach is limited by the relative frequency of the cell subset of interest and restriction of the parameter chosen to determine the normal profile that was used to create the database.
An additional way to look at the data is using the probability state modeling (PSM) method and the visualization tools in GemStone (http://www.vsh.com/products/gemstone/) software for the analysis of multidimensional flow cytometry data. A probability state model is a set of generalized functions, one for each correlated measurement, where the common cumulative probability axis can be a surrogate for time or cellular progression. By exploiting the unique characteristics of functions, PSM can model any number of correlated measurements and present one comprehensive yet understandable view of the results. In summary, these various software packages work to reduce the complexity into a relatively small set of model parameters that are amenable to group statistics and comparisons.
A model-based analysis based on statistical mixture models has been recently reported by , for cell subtype identification in flow cytometry.
Multivariate analysis of flow cytometric data using decision trees is interestingly described in , where, in order to examine whether the production of a certain cytokine is depended on other cytokines, datasets from intracellular staining for six cytokines with complex patterns of co-expression were analyzed.
A number of these approaches involve some variation of clustering analysis, which can have important limitations. Other approaches have been developed in addition to clustering, including principal components analysis (PCA) and Bayesian inference. These approaches have been evaluated through the FlowCAP initiative (http://flowcap.flowsite.org/).
To standardize the universe of Flow Data, MIFlowCyt (minimal information about a flow cytometry) experiment standard has been approved by the International Society for Advancement of Cytometry for the reporting of any flow cytometry results (http://flowcyt.sourceforge.net/miflowcyt/).
Our comment on computational approach, using of command line languages, algorithm design is that it is a potent and precious tool but quite far from the mean expertise of a flow cytometer user, which should have a specific training and/or collaborating with a bioinformatician. Maybe the future will reserve us more user friendly interfaces dedicated to computational analysis of flow data.
5. Consortia and Useful Links for Harmonization and Standardization of FCM-Based Assays
Many immune monitoring Consortia have been recently created worldwide to help facilitate and harmonize immune monitoring approaches in the cancer immunotherapy field and establish rigorous quality control standards for serial monitoring of immunologic functions.
Among them, the Cancer Immunotherapy Consortium (CIC) of the Cancer Research Institute (CRI) in USA and the Association for Cancer Immunotherapy (CIMT) in Europe supported the web-based reporting framework on “Minimal Information about T-Cell Assays” MIATA , a project aimed at generating recommendations on the minimum information required to allow an objective and thorough interpretation of published results from immunological T-cell assays. (http://www.miataproject.org/). As mentioned before, up to date, this framework has completed proficiency panels and published harmonization guidelines for the top immune assays (Elispot, peptide-multimer assays, intracellular cytokine staining, and Luminex) [48, 68].
In addition, several links might help researcher to find their response to common questions regarding technical issues about FCM based-assays. Just to cite some of them, the Maecker lab weblog (http://www.miataproject.org/) provides research and training materials for flow cytometry and immune monitoring; the (http://cytobank.org/facselect/) might help to assist with optimization of staining conditions; the ICH Q2(R1) document (http://www.ich.org/products/guidelines/quality/article/quality-guidelines.html) is a guideline for validation of analytical procedures. In addition, a number of companies, such as the IST (http://immunositetechnologies.com/services/automation/immune-monitoring-automation.html) offer their expertise to design, develop, validate, and run polychromatic flow assays.
The development of new vaccines and immunotherapeutic strategies against cancer requires the sophisticated assessment of immune parameters (biomarkers) to reliably measure antitumor immune responses.
A wide range of advanced monitoring assays is currently used to determine phenotypical and functional characteristics of antitumor T-cells in cancer immunotherapy trials, including T-cell proliferation, cytokine profile, CTL assays, CTL-associated molecules (CD107, perforin, granzyme B, and CD154), and MHC-multimer analysis. However, these assays still fail to establish the possible correlation between immune response and clinical outcome. The lack of this correlation might reflect the methodological limitations of immunologic assays or the postvaccination absence of antitumor responses sufficiently robust to induce disease-free or overall survival.
Multiparameter flow cytometry expands platform for assessing functional profiles and patterns of immune responses. In particular, the use of polychromatic flow cytometry is likely to assume a role in defining the correlates of protection for vaccine efficacy as well as in monitoring immunotherapies in diseases such as HIV and cancer. In fact, it allows simultaneous detection of various parameters such as enumeration at the single cell level of different T-cell subsets (naïve, effector, central memory, effector memory), as well as cells belonging to the innate and myeloid compartment and providing information about the breadth and the quality of the induced immune response. The recent literature relates simultaneous expression of multiple functions (polyfunctionality) to immunity, since measurement of IFN-γ alone underestimates the total response .
Nevertheless, despite advances in the development of immune monitoring assays during the past decade, further advances are needed to implement throughput and standardization of such assays according to Good Laboratory Practice guidelines, such as those recently formulated based on recommendations from the iSBTc-SITC/FDA/NCI Workshop on Immunotherapy Biomarkers .
The goal of the present paper was to provide further insights into the development of standardized FCM-based immune assays. In particular, we revised the most common and critical issues of FCM-based technologies used for immune monitoring and evaluated the applicability of a six-color flow cytometric assay, previously developed in our laboratory, for immune monitoring in the setting of melanoma studies. This assay simultaneously measures effector cell degranulation and cytokine production by Melan-A/MART-1 specific CD8+ T-cells. We were able to define some of the crucial aspects regarding sample processing and evaluated various staining and gating strategies. Concerning flow data analysis, we might conclude that different approaches of analysis and visual representations should be performed in order to obtain a complete picture of results about polyfunctionality of tumor specific T-cells.
Shown experiments were carried out according to the principles set out in the Declaration of Helsinki in 1964 and all subsequent revisions. Blood donors received and signed appropriate informed consent.
The authors would like to thank Dr. Federica Moschella for her valuable comments and scientific contribution.
- T. L. Whiteside, “Immune responses to cancer: are they potential biomarkers of prognosis?” Frontiers in Oncology, vol. 3, article 107, 2013.
- M. Vergati, C. Intrivici, N.-Y. Huen, J. Schlom, and K. Y. Tsang, “Strategies for cancer vaccine development,” Journal of Biomedicine and Biotechnology, vol. 2010, Article ID 596432, 13 pages, 2010.
- M. J. Dobrzanski, “Expanding roles for CD4 T-cells and their subpopulations in tumor immunity and therapy,” Frontiers in Oncology, vol. 3, article 63, 2013.
- M. Vergati, J. Schlom, and K. Y. Tsang, “The consequence of immune suppressive cells in the use of therapeutic cancer vaccines and their importance in immune monitoring,” Journal of Biomedicine and Biotechnology, vol. 2011, Article ID 182413, 8 pages, 2011.
- B. M. Olson and D. G. McNeel, “Monitoring regulatory immune responses in tumor immunotherapy clinical trials,” Frontiers in Oncology, vol. 3, article 109, 2013.
- I. Marigo, L. Dolcetti, P. Serafini, P. Zanovello, and V. Bronte, “Tumor-induced tolerance and immune suppression by myeloid derived suppressor cells,” Immunological Reviews, vol. 222, no. 1, pp. 162–179, 2008.
- A. Biragyn and C. Lee-Chang, “A new paradigm for an old story: the role of regulatory B cells in cancer,” Frontiers in Immunology, vol. 3, article 206, 2012.
- E. P. M. Tjin, D. Konijnenberg, G. Krebbers et al., “T-cell immune function in tumor, skin, and peripheral blood of advanced stage melanoma patients: implications for immunotherapy,” Clinical Cancer Research, vol. 17, no. 17, pp. 5736–5747, 2011.
- J. Galon, F. Pagès, F. M. Marincola et al., “The immune score as a new possible approach for the classification of cancer,” Journal of Translational Medicine, vol. 10, article 1, 2012.
- J. Galon, F. Pag ès, F. M. Marincola, et al., “Cancer classification using the Immunoscore: a worldwide task force,” Journal of Translational Medicine, vol. 10, article 205, 2012.
- L. H. Sobin and C. C. Compton, “TNM seventh edition: what's new, what's changed: communication from the International Union Against Cancer and the American Joint Committee on Cancer,” Cancer, vol. 116, no. 22, pp. 5336–5339, 2010.
- C. Wittekind, C. C. Compton, F. L. Greene, and L. H. Sobin, “TNM residual tumor classification revisited,” Cancer, vol. 94, no. 9, pp. 2511–2516, 2002.
- P. A. Ascierto, M. Capone, W. J. Urba, et al., “The additional facet of immunoscore: immunoprofiling as a possible predictive tool for cancer treatment,” Journal of Translational Medicine, vol. 11, article 54, 2013.
- W. H. Fridman, F. Pagès, C. Sautès-Fridman, and J. Galon, “The immune contexture in human tumours: impact on clinical outcome,” Nature Reviews Cancer, vol. 12, no. 4, pp. 298–306, 2012.
- F. Pagès, A. Kirilovsky, B. Mlecnik, et al., “In situ cytotoxic and memory T cells predict outcome in patients with early-stage colorectal cancer,” Journal of Clinical Oncology, vol. 27, no. 35, pp. 5944–5951, 2009.
- W. C. Olson, M. E. Smolkin, E. M. Farris et al., “Shipping blood to a central laboratory in multicenter clinical trials: effect of ambient temperature on specimen temperature, and effects of temperature on mononuclear cell yield, viability and immunologic function,” Journal of Translational Medicine, vol. 9, article 26, 2011.
- M. L. Disis, C. Dela Rosa, V. Goodell et al., “Maximizing the retention of antigen specific lymphocyte function after cryopreservation,” Journal of Immunological Methods, vol. 308, no. 1-2, pp. 13–18, 2006.
- H. T. Maecker, J. Moon, S. Bhatia et al., “Impact of cryopreservation on tetramer, cytokine flow cytometry, and ELISPOT,” BMC Immunology, vol. 6, article 17, 2005.
- A. Weinberg, L.-Y. Song, C. Wilkening et al., “Optimization and limitations of use of cryopreserved peripheral blood mononuclear cells for functional and phenotypic T-cell characterization,” Clinical and Vaccine Immunology, vol. 16, no. 8, pp. 1176–1186, 2009.
- B. E. McLaughlin, N. Baumgarth, M. Bigos et al., “Nine-color flow cytometry for accurate measurement of T cell subsets and cytokine responses. Part I: panel design by an empiric approach,” Cytometry A, vol. 73, no. 5, pp. 400–410, 2008.
- L. Papagno, J. R. Almeida, E. Nemes, B. Autran, and V. Appay, “Cell permeabilization for the assessment of T lymphocyte polyfunctional capacity,” Journal of Immunological Methods, vol. 328, no. 1-2, pp. 182–188, 2007.
- H. T. Maecker and J. Trotter, “Flow cytometry controls, instrument setup, and the determination of positivity,” Cytometry A, vol. 69, no. 9, pp. 1037–1042, 2006.
- T. Kalina, J. Flores-Montero, V. H. van der Velden, et al., “Euroflow standardization of flow cytometer instrument settings and immunophenotyping protocols,” Leukemia, vol. 26, no. 9, pp. 1986–2010, 2012.
- S. C. De Rosa, D. K. Carter, and M. J. McElrath, “OMIP-014: validated multifunctional characterization of antigen-specific human T cells by intracellular cytokine staining,” Cytometry A, vol. 81, no. 12, pp. 1019–1021, 2012.
- G. Makedonas and M. R. Betts, “Polyfunctional analysis of human t cell responses: importance in vaccine immunogenicity and natural infection,” Springer Seminars in Immunopathology, vol. 28, no. 3, pp. 209–219, 2006.
- M. R. Betts, M. C. Nason, S. M. West et al., “HIV nonprogressors preferentially maintain highly functional HIV-specific CD8+ T cells,” Blood, vol. 107, no. 12, pp. 4781–4789, 2006.
- M. L. Precopio, M. R. Betts, J. Parrino et al., “Immunization with vaccinia virus induces polyfunctional and phenotypically distinctive CD8+ T cell responses,” Journal of Experimental Medicine, vol. 204, no. 6, pp. 1405–1416, 2007.
- J. Yuan, S. Gnjatic, H. Li et al., “CTLA-4 blockade enhances polyfunctional NY-ESO-1 specific T cell responses in metastatic melanoma patients with clinical benefit,” Proceedings of the National Academy of Sciences of the United States of America, vol. 105, no. 51, pp. 20410–20415, 2008.
- S. Wilde, D. Sommermeyer, M. Leisegang, et al., “Human antitumor CD8+ T cells producing Th1 polycytokines show superior antigen sensitivity and tumor recognition,” Journal of Immunology, vol. 189, no. 2, pp. 598–605, 2012.
- S. C. de Rosa, “Vaccine applications of flow cytometry,” Methods, vol. 57, no. 3, pp. 383–391, 2012.
- T. M. Clay, A. C. Hobeika, P. J. Mosca, H. K. Lyerly, and M. A. Morse, “Assays for monitoring cellular immune responses to active immunotherapy of cancer,” Clinical Cancer Research, vol. 7, no. 5, pp. 1127–1135, 2001.
- U. Keilholz, P. Martus, and C. Scheibenbogen, “Immune monitoring of T-cell responses in cancer vaccine development,” Clinical Cancer Research, vol. 12, no. 7, part 2, pp. 2346s–2352s, 2006.
- N. Imai, H. Ikeda, I. Tawara, and H. Shiku, “Tumor progression inhibits the induction of multifunctionality in adoptively transferred tumor-specific CD8+ T cells,” European Journal of Immunology, vol. 39, no. 1, pp. 241–253, 2009.
- H. Horton, E. P. Thomas, J. A. Stucky et al., “Optimization and validation of an 8-color intracellular cytokine staining (ICS) assay to quantify antigen-specific T cells induced by vaccination,” Journal of Immunological Methods, vol. 323, no. 1, pp. 39–54, 2007.
- M. R. Betts, J. M. Brenchley, D. A. Price et al., “Sensitive and viable identification of antigen-specific CD8+ T cells by a flow cytometric assay for degranulation,” Journal of Immunological Methods, vol. 281, no. 1-2, pp. 65–78, 2003.
- V. Rubio, T. B. Stuge, N. Singh et al., “Ex vivo identification, isolation and analysis of tumor-cytolytic T cells,” Nature Medicine, vol. 9, no. 11, pp. 1377–1382, 2003.
- J. D. Altman, P. A. H. Moss, P. J. R. Goulder et al., “Phenotypic analysis of antigen-specific T lymphocytes,” Science, vol. 274, no. 5284, pp. 94–96, 1996.
- C. Yee and P. Greenberg, “Modulating T-cell immunity to tumours: new strategies for monitoring T-cell responses,” Nature Reviews Cancer, vol. 2, no. 6, pp. 409–419, 2002.
- P. Klenerman, V. Cerundolo, and P. R. Dunbar, “Tracking T cells with tetramers: new tales from new tools,” Nature Reviews Immunology, vol. 2, no. 4, pp. 263–272, 2002.
- M. J. Pittet, D. E. Speiser, D. Valmori et al., “Ex vivo analysis of tumor antigen specific CD8+ T cell responses using MHC/peptide tetramers in cancer patients,” International Immunopharmacology, vol. 1, no. 7, pp. 1235–1247, 2001.
- P. Guillaume, D. Dojcinovic, and I. F. Luescher, “Soluble MHC-peptide complexes: tools for the monitoring of T cell responses in clinical trials and basic research,” Cancer Immunity, vol. 9, article 7, 2009.
- S. K. Singh, B. Tummers, T. N. Schumacher, et al., “The development of standard samples with a defined number of antigen-specific T cells to harmonize T cell assays: a proof-of-principle study,” Cancer Immunology, Immunotherapy, vol. 62, no. 3, pp. 489–501, 2013.
- B. A. Fox, D. J. Schendel, L. H. Butterfield, et al., “Defining the critical hurdles in cancer immunotherapy,” Journal of Translational Medicine, vol. 9, article 214, 2011.
- S. H. van der Burg, M. Kalos, C. Gouttefangeas, et al., “Harmonization of immune biomarker assays for clinical studies,” Science translational medicine, vol. 3, no. 108, Article ID 108ps44, 2011.
- E. A. O'Donnell, D. N. Ernst, and R. Hingorani, “Multiparameter flow cytometry: advances in high resolution analysis,” Immune Network, vol. 13, no. 2, pp. 43–54, 2013.
- S. Janetzki, C. M. Britten, M. Kalos et al., ““MIATA”-minimal information about T cell assays,” Immunity, vol. 31, no. 4, pp. 527–528, 2009.
- Z. Moodie, L. Price, S. Janetzki, and C. M. Britten, “Response determination criteria for ELISPOT: toward a standard that can be applied across laboratories,” Methods in Molecular Biology, vol. 792, pp. 185–196, 2012.
- C. M. Britten, S. Janetzki, L. Ben-Porat et al., “Harmonization guidelines for HLA-peptide multimer assays derived from results of a large scale international proficiency panel of the cancer vaccine consortium,” Cancer Immunology, Immunotherapy, vol. 58, no. 10, pp. 1701–1713, 2009.
- M. J. Welters, C. Gouttefangeas, T. H. Ramwadhdoebe, et al., “Harmonization of the intracellular cytokine staining assay,” Cancer Immunology, Immunotherapy, vol. 61, no. 7, pp. 967–978, 2012.
- H. T. Maecker and J. P. McCoy, “Corrigendum: a model for harmonizing flow cytometry in clinical trials,” Nature Immunology, vol. 12, no. 271, pp. 975–978, 2011.
- H. T. Maecker, A. Rinfret, P. D'Souza, et al., “Standardization of cytokine flow cytometry assays,” BMC Immunology, vol. 6, article 13, 2005.
- L. Nomura, V. C. Maino, and H. T. Maecker, “Standardization and optimization of multiparameter intracellular cytokine staining,” Cytometry A, vol. 73, no. 11, pp. 984–991, 2008.
- A. Pierzchalski, A. Mittag, and A. Tárnok, “Introduction A: recent advances in cytometry instrumentation, probes, and methods—review,” Methods in Cell Biology, vol. 102, pp. 1–21, 2011.
- P. Baumgaertner, C. Jandus, J.-P. Rivals et al., “Vaccination-induced functional competence of circulating human tumor-specific CD8 T-cells,” International Journal of Cancer, vol. 130, no. 11, pp. 2607–2617, 2012.
- Y. Lin, H. F. Gallardo, G. Y. Ku et al., “Optimization and validation of a robust human T-cell culture method for monitoring phenotypic and polyfunctional antigen-specific CD4 and CD8 T-cell responses,” Cytotherapy, vol. 11, no. 7, pp. 912–922, 2009.
- L. Lamoreaux, M. Roederer, and R. Koup, “Intracellular cytokine optimization and standard operating procedure,” Nature Protocols, vol. 1, no. 3, pp. 1507–1516, 2006.
- P. Batard, D. A. Peterson, E. Devêvre et al., “Dextramers: new generation of fluorescent MHC class I/peptide multimers for visualization of antigen-specific CD8+ T cells,” Journal of Immunological Methods, vol. 310, no. 1-2, pp. 136–148, 2006.
- M. Roederer, J. L. Nozzi, and M. C. Nason, “SPICE: exploration and analysis of post-cytometric complex multivariate datasets,” Cytometry A, vol. 79, no. 2, pp. 167–174, 2011.
- M. G. Duvall, M. L. Precopio, D. A. Ambrozak, et al., “Polyfunctional T cell responses are a hallmark of HIV-2 infection,” European Journal of Immunology, vol. 38, no. 2, pp. 350–363, 2008.
- S. C. Bendall, G. P. Nolan, M. Roederersend, and P. K. Chattopadhyay, “A deep profiler's guide to cytometry,” Trends in Immunology, vol. 33, no. 7, pp. 323–332, 2012.
- F. Hahne, N. LeMeur, R. R. Brinkman et al., “flowcore: a bioconductor package for high throughput flow cytometry,” BMC Bioinformatics, vol. 10, article 106, 2009.
- M. Roederer, W. Moore, A. Treister, R. R. Hardy, and L. A. Herzenberg, “Probability binning comparison: a metric for quantitating multivariate distribution differences,” Cytometry, vol. 45, no. 1, pp. 47–55, 2001.
- T. Kalina, J. Stuchlý, A. Janda et al., “Profiling of polychromatic flow cytometry data on B-cells reveals patients' clusters in common variable immunodeficiency,” Cytometry A, vol. 75, no. 11, pp. 902–909, 2009.
- P. Qiu, “Inferring phenotypic properties from single-cell characteristics,” PLoS One, vol. 7, no. 5, Article ID e37038, 2012.
- P. Qiu, E. F. Simonds, S. C. Bendall et al., “Extracting a cellular hierarchy from high-dimensional cytometry data with SPADE,” Nature Biotechnology, vol. 29, no. 10, pp. 886–891, 2011.
- J. Frelinger, J. Ottinger, C. Gouttefangeas, and C. Chan, “Modeling flow cytometry data for cancer vaccine immune monitoring,” Cancer Immunology, Immunotherapy, vol. 59, no. 9, pp. 1435–1441, 2010.
- S. Simon, R. Guthke, T. Kamradt, and O. Frey, “Multivariate analysis of flow cytometric data using decision trees,” Frontiers in Microbiology, vol. 3, article 114, 2012.
- S. Janetzki, K. S. Panageas, L. Ben-Porat et al., “Results and harmonization guidelines from two large-scale international elispot proficiency panels conducted by the cancer vaccine consortium (CVC/SVI),” Cancer Immunology, Immunotherapy, vol. 57, no. 3, pp. 303–315, 2008.
- S. C. De Rosa, F. X. Lu, J. Yu et al., “Vaccination in humans generates broad T cell cytokine responses,” Journal of Immunology, vol. 173, no. 9, pp. 5372–5380, 2004.
- L. H. Butterfield, A. K. Palucka, C. M. Britten et al., “Recommendations from the iSBTc-SITC/FDA/NCI workshop on immunotherapy biomarkers,” Clinical Cancer Research, vol. 17, no. 10, pp. 3064–3076, 2011.