• OmicSHIELD
  • Welcome
  • I Preamble
  • 1 Introduction
    • 1.1 Materials to read beforehand
    • 1.2 What are “resources”: A very simple explanation without any technicalities
    • 1.3 Capabilities of OmicSHIELD
    • 1.4 Opal servers
  • 2 Omic data analysis: types of implemented analyses
  • 3 Datasets description
    • 3.1 CINECA project
    • 3.2 Data organization in the Opal server
    • 3.3 TCGA Liver project
    • 3.4 Epigenetic data from the GSE66351 project
    • 3.5 HELIX project
      • 3.5.1 Data organization in the Opal server
  • 4 Exploration of Genotype data
    • 4.1 Connection to the Opal server
    • 4.2 Assign the VCF resources
    • 4.3 Assign the phenotypes
    • 4.4 Merge VCF (genotype) and phenotype information
    • 4.5 Basic information of the Genotype data
    • 4.6 Basic statistics of the genotype data
    • 4.7 Principal component analysis
  • II Workflows
  • 5 Genome wide association study (GWAS) analysis
    • 5.1 Single cohort
      • 5.1.1 Connection to the Opal server
      • 5.1.2 Assign the VCF resources
      • 5.1.3 Assign the phenotypes
      • 5.1.4 Merge VCF (genotype) and phenotype information
      • 5.1.5 Performing the GWAS
    • 5.2 Multi cohorts
      • 5.2.1 Connection to the Opal server
      • 5.2.2 Assign the VCF resources
      • 5.2.3 Assign the phenotypes
      • 5.2.4 Merge VCF (genotype) and phenotype information
      • 5.2.5 Performing a meta - GWAS
      • 5.2.6 GWAS Plots
      • 5.2.7 Meta-level quality control
      • 5.2.8 Meta-analysis
      • 5.2.9 Ultra fast pooled GWAS analysis
    • 5.3 Post-omic analysis
      • 5.3.1 Enrichment analysis
  • 6 Polygenic risk scores
    • 6.1 Definition
    • 6.2 Study configuration
    • 6.3 Source of polygenic scores
    • 6.4 Connection to the Opal server
    • 6.5 Assign the VCF resources
    • 6.6 Calculate the PRS
      • 6.6.1 PGS Catalog Example
      • 6.6.2 Custom prs_table example
    • 6.7 Study of the PRS results
  • 7 PLINK
    • 7.1 Connection to the Opal server
    • 7.2 Build the PLINK call
    • 7.3 Call the analysis
  • 8 SNPTEST
    • 8.1 Connection to the Opal server
    • 8.2 Build the SNPTEST call
    • 8.3 Call the analysis
  • 9 Differential gene expression (DGE) analysis
    • 9.1 Connection to the Opal server
    • 9.2 Assign the RSE resource
    • 9.3 Inspect the RSE
    • 9.4 Pre-processing for RNAseq data
    • 9.5 DGE analysis
    • 9.6 Surrogate variable analysis
  • 10 Epigenome-wide association analysis (EWAS)
    • 10.1 Initial steps for all use cases
      • 10.1.1 Connection to the Opal server
      • 10.1.2 Assign the ExpressionSet resource
      • 10.1.3 Inspect the ExpressionSet
    • 10.2 Single CpG pooled analysis
    • 10.3 Multiple CpGs pooled analysis
    • 10.4 Full genome meta-analysis
    • 10.5 Adjusting for surrogate variables
  • III Showcase on a real dataset
  • 11 Transcriptomic analysis in the HELIX cohort.
    • 11.1 Getting started.
    • 11.2 Data formatting and manipulation in DataSHIELD.
    • 11.3 Differential gene expression analysis.
    • 11.4 Surrogate variable analysis.
    • 11.5 Enrichment analysis of functional annotations.
  • 12 Epigenetic analysis in the HELIX cohort.
    • 12.1 Getting started.
    • 12.2 Data formatting and manipulation in DataSHIELD.
    • 12.3 Full genome meta-analysis.
    • 12.4 Adjusting for surrogate variables.
    • 12.5 Enrichment analysis of functional annotations.
  • IV Technical details
  • 13 Fast GWAS
  • 14 PCA
  • 15 Differential privacy
  • 16 Internal structure of the PRS functionality
    • 16.1 Client
    • 16.2 Server resolver
    • 16.3 Server analysis
  • 17 Compression of GDS files vs. performance
  • 18 Recommended R server specs
    • 18.1 Limma + voom
    • 18.2 GWAS + PCA
    • 18.3 General remarks
  • 19 Disclosure filter configurations
  • V Appendix
  • 20 Session info
  • 21 Contributors
  • VI References
  • 22 References
  • Published with bookdown

Privacy protected federated omic data analysis in multi-center studies with DataSHIELD

18 Recommended R server specs

We conducted some stress tests to assess the maximum allocated RAM and HDD space consumed by the analysis portraited on this bookdown.

18.1 Limma + voom

For epigenomics and differential gene expression analysis we portrayed the use of limma + voom. The disk space consumed by the ExpressionSets used is dependent on the array dimensions and the number of individuals, and so is the RAM allocation; this yields the following table:

Array Individuals Storage [GB] RAM [GB]
EPIC 450K 100 0,34 3
EPIC 450K 500 1,7 12
EPIC 450K 1000 3,4 24
EPIC 450K 2000 6,8 48
EPIC 850K 100 0,64 8
EPIC 850K 500 3,2 36
EPIC 850K 1000 6,4 72
EPIC 850K 2000 12,8 144

This table can be summarized on the following graphs:

Disk space requirements for EPIC 450K array versus number of individuals (10 meta-variables)

Figure 18.1: Disk space requirements for EPIC 450K array versus number of individuals (10 meta-variables)

Disk space requirements for EPIC 850K array versus number of individuals (10 meta-variables)

Figure 18.2: Disk space requirements for EPIC 850K array versus number of individuals (10 meta-variables)

Maximum allocated RAM for EPIC 450K array versus number of individuals (1 adjustin covariable)

Figure 18.3: Maximum allocated RAM for EPIC 450K array versus number of individuals (1 adjustin covariable)

Maximum allocated RAM for EPIC 850K array versus number of individuals (1 adjustin covariable)

Figure 18.4: Maximum allocated RAM for EPIC 850K array versus number of individuals (1 adjustin covariable)

We can clearly see a lineal correlation to estimate the hardware requirements.

18.2 GWAS + PCA

Genomic data is accepted in OmicSHIELD as gds files, which can be compressed using different algorithms, therefore it makes no sense to extract any relationship between individuals / number of variants and file size. Please refer to the appropiate section for information about that.

Also, the methodologies used (both pooled and meta-analysis) are not as RAM dependent as Limma analysis. This has to do with the nature of genomic data, which is typically extremely large; to analyze this kind of data we do not load it entirely on the RAM, the algorithms perform clever techniques that read portions of the data at a time, and the size of that portion can be configured by the user (see the argument snpBlock), therefore the researchers can tune the function calls if RAM limits become an issue.

18.3 General remarks

On top of this information some details must be noted:

  • Disk space (storage) is cheap and easy to upgrade, RAM is expensive. Therefore storage requirements should be easy to fulfill and RAM requirements may require a steeper investment.
  • The server has to perform the analysis AND host the Opal/Armadillo + R server. For reference, our deployment, which contains Apache + Opal + 4 x R servers, allocates 5 GB of RAM memory.
  • Results are approximate, it is advisable to apply a security coefficient (especially on the RAM) of about 1.5 to be safe.
  • Imputed GWAS data can be quite heavy on disk space, however it is not that heavy on RAM usage (https://www.well.ox.ac.uk/~gav/snptest/#other_options chunk flag).
  • Methylation data in contrast is more RAM intensive when being analyzed.
  • There is no queuing system implemented at the moment, so if multiple people are conducting analyses at the same time, the performance of the system will be compromised.
  • If possible, get more RAM and storage than the bare minimum, as it is expected that omics data will increase in longitudinal cohort studies during next years.

Regarding the discussion of dimensioning the servers for number of cores; this does not depend on the size and complexity of the analysis to be carried out, but some functionalities benefit from a multi-core architecture to speed up the calculations. However, it is not a constraint, as everything will run perfectly on one core. The discussion to take place is about the number of concurrent users of the servers. As a rule of thumb, we recommend having one core per user, that way things will not collapse when all users perform analyses at the same time. However, this has financial consequences. A good starting point would be to work with 8 cores minimum, but this is for the partners to decide.