Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

A sequence-oriented comparison of gene expression measurements across different hybridization-based technologies

Abstract

Over the last decade, gene expression microarrays have had a profound impact on biomedical research. The diversity of platforms and analytical methods available to researchers have made the comparison of data from multiple platforms challenging. In this study, we describe a framework for comparisons across platforms and laboratories. We have attempted to include nearly all the available commercial and 'in-house' platforms. Using probe sequences matched at the exon level improved consistency of measurements across the different microarray platforms compared to annotation-based matches. Generally, consistency was good for highly expressed genes, and variable for genes with lower expression values as confirmed by quantitative real-time (QRT)-PCR. Concordance of measurements was higher between laboratories on the same platform than across platforms. We demonstrate that, after stringent preprocessing, commercial arrays were more consistent than in-house arrays, and by most measures, one-dye platforms were more consistent than two-dye platforms.

This is a preview of subscription content, access via your institution

Access options

Buy this article

Purchase on Springer Link

Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Cross-platform agreement of probes matched within one exon.
Figure 2: Cross-platform PCA plot.
Figure 3: Scatter plot of QRT-PCR versus all microarrays.

Similar content being viewed by others

Accession codes

Accessions

Gene Expression Omnibus

References

  1. Edgar, R., Domrachev, M. & Lash, A.E. Gene Expression Omnibus: NCBI gene expression and hybridization array data repository. Nucleic Acids Res. 30, 207–210 (2002).

    Article  CAS  Google Scholar 

  2. Brazma, A. et al. ArrayExpress–a public repository for microarray gene expression data at the EBI. Nucleic Acids Res. 31, 68–71 (2003).

    Article  CAS  Google Scholar 

  3. Ali-Seyed, M. et al. Cross-platform expression profiling demonstrates that SV40 small tumor antigen activates Notch, Hedgehog, and Wnt signaling in human cells. BMC Cancer 6, 54–68 (2006).

    Article  Google Scholar 

  4. Bammler, T. et al. Standardizing global gene expression analysis between laboratories and across platforms. Nat. Methods 2, 351–356 (2005).

    Article  Google Scholar 

  5. Barczak, A. et al. Spotted long oligonucleotide arrays for human gene expression analysis. Genome Res. 13, 1775–1785 (2003).

    Article  CAS  Google Scholar 

  6. Barnes, M., Freudenberg, J., Thompson, S., Aronow, B. & Pavlidis, P. Experimental comparison and cross-validation of the Affymetrix and Illumina gene expression analysis platforms. Nucleic Acids Res. 33, 5914–5923 (2005).

    Article  CAS  Google Scholar 

  7. Irizarry, R.A. et al. Multiple-laboratory comparison of microarray platforms. Nat. Methods 2, 345–350 (2005).

    Article  CAS  Google Scholar 

  8. Kothapalli, R., Yoder, S.J., Mane, S. & Loughran, T.P., Jr. Microarray results: how accurate are they? BMC Bioinformatics 3, 22–32 (2002).

    Article  Google Scholar 

  9. Kuo, W.P., Jenssen, T.K., Butte, A.J., Ohno-Machado, L. & Kohane, I.S. Analysis of matched mRNA measurements from two different microarray technologies. Bioinformatics 18, 405–412 (2002).

    Article  CAS  Google Scholar 

  10. Larkin, J.E., Frank, B.C., Gavras, H., Sultana, R. & Quackenbush, J. Independence and reproducibility across microarray platforms. Nat. Methods 2, 337–344 (2005).

    Article  CAS  Google Scholar 

  11. Lee, J.K. et al. Comparing cDNA and oligonucleotide array data: concordance of gene expression across platforms for the NCI-60 cancer cells. Genome Biol. 4, R82–94 (2003).

    Article  Google Scholar 

  12. Li, J., Pankratz, M. & Johnson, J.A. Differential gene expression patterns revealed by oligonucleotide versus long cDNA arrays. Toxicol. Sci. 69, 383–390 (2002).

    Article  CAS  Google Scholar 

  13. Mecham, B.H. et al. Sequence-matched probes produce increased cross-platform consistency and more reproducible biological results in microarray-based gene expression measurements. Nucleic Acids Res. 32, e74–82 (2004).

    Article  Google Scholar 

  14. Park, P.J. et al. Current issues for DNA microarrays: platform comparison, double linear amplification, and universal RNA reference. J. Biotechnol. 112, 225–245 (2004).

    Article  CAS  Google Scholar 

  15. Parrish, M.L. et al. A microarray platform comparison for neuroscience applications. J. Neurosci. Methods 132, 57–68 (2004).

    Article  CAS  Google Scholar 

  16. Petersen, D. et al. Three microarray platforms: an analysis of their concordance in profiling gene expression. BMC Genomics 6, 63–77 (2005).

    Article  Google Scholar 

  17. Pylatuik, J.D. & Fobert, P.R. Comparison of transcript profiling on Arabidopsis microarray platform technologies. Plant Mol. Biol. 58, 609–624 (2005).

    Article  CAS  Google Scholar 

  18. Rogojina, A.T., Orr, W.E., Song, B.K. & Geisert, E.E., Jr. Comparing the use of Affymetrix to spotted oligonucleotide microarrays using two retinal pigment epithelium cell lines. Mol. Vis. 9, 482–496 (2003).

    CAS  PubMed  PubMed Central  Google Scholar 

  19. Schlingemann, J. et al. Patient-based cross-platform comparison of oligonucleotide microarray expression profiles. Lab. Invest. 85, 1024–1039 (2005).

    Article  CAS  Google Scholar 

  20. Shi, L. et al. Cross-platform comparability of microarray technology: intra-platform consistency and appropriate data analysis procedures are essential. BMC Bioinformatics 6 suppl. Suppl. 2, S12–S26 (2005).

    Article  Google Scholar 

  21. Tan, P.K. et al. Evaluation of gene expression measurements from commercial microarray platforms. Nucleic Acids Res. 31, 5676–5684 (2003).

    Article  CAS  Google Scholar 

  22. Shippy, R. et al. Performance evaluation of commercial short-oligonucleotide microarrays and the impact of noise in making cross-platform correlations. BMC Genomics 5, 61–76 (2004).

    Article  Google Scholar 

  23. Walker, S.J., Wang, Y., Grant, K.A., Chan, F. & Hellmann, G.M. Long versus short oligonucleotide microarrays for the study of gene expression in nonhuman primates. J. Neurosci. Methods 152, 179–189 (2005).

    Article  Google Scholar 

  24. Wang, H., He, X., Band, M., Wilson, C. & Liu, L. A study of inter-lab and inter-platform agreement of DNA microarray data. BMC Genomics 6, 71–80 (2005).

    Article  Google Scholar 

  25. Wang, H.Y. et al. Assessing unmodified 70-mer oligonucleotide probe performance on glass-slide microarrays. Genome Biol. 4, R5–R18 (2003).

    Article  Google Scholar 

  26. Warnat, P., Eils, R. & Brors, B. Cross-platform analysis of cancer microarray data improves gene expression based classification of phenotypes. BMC Bioinformatics 6, 265–280 (2005).

    Article  Google Scholar 

  27. Woo, Y. et al. A comparison of cDNA, oligonucleotide, and Affymetrix GeneChip gene expression microarray platforms. J. Biomol. Tech. 15, 276–284 (2004).

    PubMed  PubMed Central  Google Scholar 

  28. Yauk, C.L., Berndt, M.L., Williams, A. & Douglas, G.R. Comprehensive comparison of six microarray technologies. Nucleic Acids Res. 32, e124–e131 (2004).

    Article  Google Scholar 

  29. Yuen, T., Wurmbach, E., Pfeffer, R.L., Ebersole, B.J. & Sealfon, S.C. Accuracy and calibration of commercial oligonucleotide and custom cDNA microarrays. Nucleic Acids Res. 30, e48–e57 (2002).

    Article  Google Scholar 

  30. Zhu, B., Ping, G., Shinohara, Y., Zhang, Y. & Baba, Y. Comparison of gene expression measurements from cDNA and 60-mer oligonucleotide microarrays. Genomics 85, 657–665 (2005).

    Article  CAS  Google Scholar 

  31. Sherlock, G. Of fish and chips. Nat. Methods 2, 329–330 (2005).

    Article  CAS  Google Scholar 

  32. Lee, M.L., Kuo, F.C., Whitmore, G.A. & Sklar, J. Importance of replication in microarray gene expression studies: statistical methods and evidence from repetitive cDNA hybridizations. Proc. Natl. Acad. Sci. USA 97, 9834–9839 (2000).

    Article  CAS  Google Scholar 

  33. Mecham, B.H. et al. Increased measurement accuracy for sequence-verified microarray probes. Physiol. Genomics 18, 308–315 (2004).

    Article  CAS  Google Scholar 

  34. Carter, S.L., Eklund, A.C., Mecham, B.H., Kohane, I.S. & Szallasi, Z. Redefinition of Affymetrix probe sets by sequence overlap with cDNA microarray probes reduces cross-platform inconsistencies in cancer-associated gene expression measurements. BMC Bioinformatics 6, 107–122 (2005).

    Article  Google Scholar 

  35. Blackshaw, S., Fraioli, R.E., Furukawa, T. & Cepko, C.L. Comprehensive analysis of photoreceptor gene expression and the identification of candidate retinal disease genes. Cell 107, 579–589 (2001).

    Article  CAS  Google Scholar 

  36. Blackshaw, S. et al. Genomic analysis of mouse retinal development. PLoS Biol. 2, E247–E268 (2004).

    Article  Google Scholar 

  37. Velculescu, V.E., Zhang, L., Vogelstein, B. & Kinzler, K.W. Serial analysis of gene expression. Science 270, 484–487 (1995).

    Article  CAS  Google Scholar 

  38. Brenner, S. et al. Gene expression analysis by massively parallel signature sequencing (MPSS) on microbead arrays. Nat. Biotechnol. 18, 630–634 (2000).

    Article  CAS  Google Scholar 

  39. Pounds, S. & Cheng, C. Statistical development and evaluation of microarray gene expression data filters. J. Comput. Biol. 12, 482–495 (2005).

    Article  CAS  Google Scholar 

  40. Chu, T.M., Deng, S., Wolfinger, R., Paules, R.S. & Hamadeh, H.K. Cross-site comparison of gene expression data reveals high similarity. Environ. Health Perspect. 112, 449–455 (2004).

    Article  CAS  Google Scholar 

  41. Qin, L.X. et al. Evaluation of methods for oligonucleotide array data via quantitative real-time PCR. BMC Bioinformatics 7, 23 (2006).

    Article  Google Scholar 

  42. Roth, M.E. et al. Expression profiling using a hexamer-based universal microarray. Nat. Biotechnol. 22, 418–426 (2004).

    Article  CAS  Google Scholar 

  43. Gunderson, K.L. et al. Decoding randomly ordered DNA arrays. Genome Res. 14, 870–877 (2004).

    Article  CAS  Google Scholar 

  44. Workman, C. et al. A new non-linear normalization method for reducing variability in DNA microarray experiments. Genome Biol 3, research0048 (2002).

  45. Berger, J.A. et al. Optimized LOWESS normalization parameter selection for DNA microarray data. BMC Bioinformatics 5, 194–207 (2004).

    Article  Google Scholar 

  46. Bolstad, B.M., Irizarry, R.A., Astrand, M. & Speed, T.P. A comparison of normalization methods for high density oligonucleotide array data based on variance and bias. Bioinformatics 19, 185–193 (2003).

    Article  CAS  Google Scholar 

  47. Bussey, K.J. et al. MatchMiner: a tool for batch navigation among gene and gene product identifiers. Genome Biol. 4, R27–34 (2003).

    Article  Google Scholar 

  48. Kent, W.J. BLAT–the BLAST-like alignment tool. Genome Res. 12, 656–664 (2002).

    Article  CAS  Google Scholar 

  49. Liu, G. et al. NetAffx: Affymetrix probesets and annotations. Nucleic Acids Res. 31, 82–86 (2003).

    Article  CAS  Google Scholar 

  50. Gentleman, R.C. et al. Bioconductor: open software development for computational biology and bioinformatics. Genome Biol. 5, R80–R96 (2004).

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank vendors, Applied Biosystems, GE Healthcare, and Mergen for providing and running microarrays as part of this large-scale evaluation. In addition, we would like to thank Applied Biosystems for running TaqMan assays and Exiqon for supplying us with the ProbeLibrary kit as well as Roche Diagnostics for allowing us to use their 480 LightCycler. We thank Robert A. Greenes for reviewing the manuscript. W.P.K. was supported by the National Institutes of Health (NIH) EY014466 grant and by the Bioinformatics Division of the Harvard Center for Neurodegeneration and Repair. C.L.C. was supported by the Howard Hughes Medical Institute. F.L. and E.H. were supported by the functional genomics program (FUGE) in the Research council of Norway. G.M.C. was supported by NIH-NHGRI-CEGS. M.W.F., B.S. and G.F.S. were supported by Programs for Genomic Applications grants HL66678 and HL72358. R.B. was supported by NIH grants HL072370 and ES011387.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Winston Patrick Kuo.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Fig. 1

Assessment of cross-platform data agreement using CAT plots. (PDF 760 kb)

Supplementary Fig. 2

Assessment of cross-laboratory data agreement using PCA. (PDF 471 kb)

Supplementary Table 1

Evaluation of data consistency within platforms. (PDF 106 kb)

Supplementary Table 2

Evaluation of data consistency across platforms. (PDF 145 kb)

Supplementary Table 3

QRT-PCR primer sequences and validation results. (PDF 154 kb)

Supplementary Data 1

Study design and experimental protocols of microarray platforms and QRT-PCR technologies. (PDF 150 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kuo, W., Liu, F., Trimarchi, J. et al. A sequence-oriented comparison of gene expression measurements across different hybridization-based technologies. Nat Biotechnol 24, 832–840 (2006). https://doi.org/10.1038/nbt1217

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nbt1217

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing