All benchmarks

Dimensionality Reduction for Visualisation

Reduction of high-dimensional datasets to 2D for visualization & interpretation

11 methods
3 control methods
4 datasets
10 metrics
1 release

Dimensionality reduction is one of the key challenges in single-cell data representation. Routine single-cell RNA sequencing (scRNA-seq) experiments measure cells in roughly 20,000-30,000 dimensions (i.e., features - mostly gene transcripts but also other functional elements encoded in mRNA such as lncRNAs). Since its inception, scRNA-seq experiments have been growing in terms of the number of cells measured. Originally, cutting-edge SmartSeq experiments would yield a few hundred cells, at best. Now, it is not uncommon to see experiments that yield over 100,000 cells or even > 1 million cells.

Each feature in a dataset functions as a single dimension. While each of the ~30,000 dimensions measured in each cell contribute to an underlying data structure, the overall structure of the data is challenging to display in few dimensions due to data sparsity and the "curse of dimensionality" (distances in high dimensional data don’t distinguish data points well). Thus, we need to find a way to dimensionally reduce the data for visualization and interpretation.

contributors
summary figure

Leaderboard

Methods ranked by scaled overall mean. Each cell encodes a score from 0 to 1 by size and intensity.

QC: Normalisation Visualisation 10 plots

Per metric: points placed by control-anchored scaled score (x); dashed lines mark scaled 0 and 1 (worst/best control); the lower axis shows the raw score. Points beyond [-0.2, 1.2] are clamped to the edge as triangles. Hover a dot or line to highlight it and read details.

methodcontrol
  • co-KNN AUChigher better
    True FeaturesTrue FeaturesdensMAP (logCP10k)densMAP (logCP10k)t-SNE (logCP10k)t-SNE (logCP10k)PyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k)NeuralEE (CPU) (logCP…NeuralEE (CPU) (logCP10k, 1kHVG)densMAP PCA (logCP10k)densMAP PCA (logCP10k)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k)UMAP (logCP10k)UMAP (logCP10k)UMAP PCA (logCP10k)UMAP PCA (logCP10k)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k, 1kHVG)NeuralEE (CPU) (Defau…NeuralEE (CPU) (Default)t-SNE (logCP10k, 1kHV…t-SNE (logCP10k, 1kHVG)PHATE (gamma=0)PHATE (gamma=0)PHATE (logCP10k, 1kHV…PHATE (logCP10k, 1kHVG)PCA (logCP10k)PCA (logCP10k)PHATE (logCP10k)PHATE (logCP10k)UMAP PCA (logCP10k, 1…UMAP PCA (logCP10k, 1kHVG)Diffusion mapsDiffusion mapsdensMAP PCA (logCP10k…densMAP PCA (logCP10k, 1kHVG)PyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k, 1kHVG)PCA (logCP10k, 1kHVG)PCA (logCP10k, 1kHVG)UMAP (logCP10k, 1kHVG)UMAP (logCP10k, 1kHVG)densMAP (logCP10k, 1k…densMAP (logCP10k, 1kHVG)PHATE (default)PHATE (default)Spectral FeaturesSpectral FeaturesRandom FeaturesRandom Features0.50.6250.750.875100.250.50.751rawscaled
  • co-KNN sizehigher better
    True FeaturesTrue Featurest-SNE (logCP10k)t-SNE (logCP10k)densMAP PCA (logCP10k)densMAP PCA (logCP10k)densMAP (logCP10k)densMAP (logCP10k)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k)UMAP PCA (logCP10k)UMAP PCA (logCP10k)UMAP (logCP10k)UMAP (logCP10k)t-SNE (logCP10k, 1kHV…t-SNE (logCP10k, 1kHVG)NeuralEE (CPU) (Defau…NeuralEE (CPU) (Default)densMAP PCA (logCP10k…densMAP PCA (logCP10k, 1kHVG)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k, 1kHVG)PHATE (logCP10k)PHATE (logCP10k)NeuralEE (CPU) (logCP…NeuralEE (CPU) (logCP10k, 1kHVG)densMAP (logCP10k, 1k…densMAP (logCP10k, 1kHVG)UMAP PCA (logCP10k, 1…UMAP PCA (logCP10k, 1kHVG)UMAP (logCP10k, 1kHVG)UMAP (logCP10k, 1kHVG)PCA (logCP10k, 1kHVG)PCA (logCP10k, 1kHVG)PHATE (gamma=0)PHATE (gamma=0)PHATE (default)PHATE (default)PHATE (logCP10k, 1kHV…PHATE (logCP10k, 1kHVG)PCA (logCP10k)PCA (logCP10k)PyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k)PyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k, 1kHVG)Diffusion mapsDiffusion mapsSpectral FeaturesSpectral FeaturesRandom FeaturesRandom Features2.5e-30.2520.5010.751100.250.50.751rawscaled
  • continuityhigher better
    True FeaturesTrue FeaturesdensMAP (logCP10k)densMAP (logCP10k)UMAP (logCP10k)UMAP (logCP10k)densMAP PCA (logCP10k)densMAP PCA (logCP10k)PHATE (logCP10k)PHATE (logCP10k)t-SNE (logCP10k)t-SNE (logCP10k)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k)NeuralEE (CPU) (logCP…NeuralEE (CPU) (logCP10k, 1kHVG)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k, 1kHVG)t-SNE (logCP10k, 1kHV…t-SNE (logCP10k, 1kHVG)UMAP PCA (logCP10k, 1…UMAP PCA (logCP10k, 1kHVG)UMAP (logCP10k, 1kHVG)UMAP (logCP10k, 1kHVG)densMAP (logCP10k, 1k…densMAP (logCP10k, 1kHVG)PHATE (gamma=0)PHATE (gamma=0)PHATE (logCP10k, 1kHV…PHATE (logCP10k, 1kHVG)NeuralEE (CPU) (Defau…NeuralEE (CPU) (Default)UMAP PCA (logCP10k)UMAP PCA (logCP10k)PHATE (default)PHATE (default)densMAP PCA (logCP10k…densMAP PCA (logCP10k, 1kHVG)PCA (logCP10k)PCA (logCP10k)PCA (logCP10k, 1kHVG)PCA (logCP10k, 1kHVG)PyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k)PyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k, 1kHVG)Random FeaturesRandom FeaturesSpectral FeaturesSpectral FeaturesDiffusion mapsDiffusion maps0.1310.3480.5650.783100.250.50.751rawscaled
  • Density preservationhigher better
    True FeaturesTrue FeaturesdensMAP (logCP10k)densMAP (logCP10k)densMAP PCA (logCP10k)densMAP PCA (logCP10k)densMAP (logCP10k, 1k…densMAP (logCP10k, 1kHVG)densMAP PCA (logCP10k…densMAP PCA (logCP10k, 1kHVG)PyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k)PyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k, 1kHVG)PCA (logCP10k, 1kHVG)PCA (logCP10k, 1kHVG)NeuralEE (CPU) (logCP…NeuralEE (CPU) (logCP10k, 1kHVG)t-SNE (logCP10k)t-SNE (logCP10k)UMAP (logCP10k)UMAP (logCP10k)Spectral FeaturesSpectral Featurest-SNE (logCP10k, 1kHV…t-SNE (logCP10k, 1kHVG)UMAP PCA (logCP10k)UMAP PCA (logCP10k)PHATE (default)PHATE (default)PHATE (gamma=0)PHATE (gamma=0)PCA (logCP10k)PCA (logCP10k)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k, 1kHVG)UMAP PCA (logCP10k, 1…UMAP PCA (logCP10k, 1kHVG)Random FeaturesRandom FeaturesUMAP (logCP10k, 1kHVG)UMAP (logCP10k, 1kHVG)PHATE (logCP10k, 1kHV…PHATE (logCP10k, 1kHVG)Diffusion mapsDiffusion mapsNeuralEE (CPU) (Defau…NeuralEE (CPU) (Default)PHATE (logCP10k)PHATE (logCP10k)-0.0810.1890.4590.73100.250.50.751rawscaled
  • Distance correlationhigher better
    True FeaturesTrue FeaturesPyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k)densMAP (logCP10k)densMAP (logCP10k)PCA (logCP10k)PCA (logCP10k)UMAP (logCP10k)UMAP (logCP10k)t-SNE (logCP10k)t-SNE (logCP10k)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k)NeuralEE (CPU) (logCP…NeuralEE (CPU) (logCP10k, 1kHVG)UMAP PCA (logCP10k)UMAP PCA (logCP10k)Diffusion mapsDiffusion mapsNeuralEE (CPU) (Defau…NeuralEE (CPU) (Default)densMAP PCA (logCP10k)densMAP PCA (logCP10k)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k, 1kHVG)PHATE (logCP10k, 1kHV…PHATE (logCP10k, 1kHVG)PyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k, 1kHVG)UMAP PCA (logCP10k, 1…UMAP PCA (logCP10k, 1kHVG)PHATE (gamma=0)PHATE (gamma=0)PHATE (logCP10k)PHATE (logCP10k)densMAP PCA (logCP10k…densMAP PCA (logCP10k, 1kHVG)t-SNE (logCP10k, 1kHV…t-SNE (logCP10k, 1kHVG)UMAP (logCP10k, 1kHVG)UMAP (logCP10k, 1kHVG)densMAP (logCP10k, 1k…densMAP (logCP10k, 1kHVG)PHATE (default)PHATE (default)PCA (logCP10k, 1kHVG)PCA (logCP10k, 1kHVG)Spectral FeaturesSpectral FeaturesRandom FeaturesRandom Features-0.0560.20.4560.7110.96700.250.50.751rawscaled
  • Distance correlation (spectral)higher better
    Spectral FeaturesSpectral FeaturesPyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k, 1kHVG)NeuralEE (CPU) (logCP…NeuralEE (CPU) (logCP10k, 1kHVG)t-SNE (logCP10k, 1kHV…t-SNE (logCP10k, 1kHVG)PCA (logCP10k, 1kHVG)PCA (logCP10k, 1kHVG)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k, 1kHVG)PHATE (logCP10k, 1kHV…PHATE (logCP10k, 1kHVG)UMAP PCA (logCP10k, 1…UMAP PCA (logCP10k, 1kHVG)densMAP PCA (logCP10k…densMAP PCA (logCP10k, 1kHVG)True FeaturesTrue FeaturesDiffusion mapsDiffusion mapsPyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k)Random FeaturesRandom Featurest-SNE (logCP10k)t-SNE (logCP10k)UMAP (logCP10k, 1kHVG)UMAP (logCP10k, 1kHVG)NeuralEE (CPU) (Defau…NeuralEE (CPU) (Default)densMAP (logCP10k, 1k…densMAP (logCP10k, 1kHVG)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k)densMAP PCA (logCP10k)densMAP PCA (logCP10k)densMAP (logCP10k)densMAP (logCP10k)PHATE (gamma=0)PHATE (gamma=0)UMAP (logCP10k)UMAP (logCP10k)UMAP PCA (logCP10k)UMAP PCA (logCP10k)PHATE (logCP10k)PHATE (logCP10k)PHATE (default)PHATE (default)PCA (logCP10k)PCA (logCP10k)-0.1280.1540.4360.718100.250.50.751rawscaled
  • global propertyhigher better
    True FeaturesTrue FeaturesdensMAP (logCP10k)densMAP (logCP10k)PyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k)UMAP (logCP10k)UMAP (logCP10k)NeuralEE (CPU) (logCP…NeuralEE (CPU) (logCP10k, 1kHVG)t-SNE (logCP10k)t-SNE (logCP10k)UMAP PCA (logCP10k)UMAP PCA (logCP10k)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k, 1kHVG)densMAP PCA (logCP10k)densMAP PCA (logCP10k)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k)PyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k, 1kHVG)PHATE (logCP10k, 1kHV…PHATE (logCP10k, 1kHVG)NeuralEE (CPU) (Defau…NeuralEE (CPU) (Default)Diffusion mapsDiffusion mapst-SNE (logCP10k, 1kHV…t-SNE (logCP10k, 1kHVG)PCA (logCP10k)PCA (logCP10k)PCA (logCP10k, 1kHVG)PCA (logCP10k, 1kHVG)PHATE (gamma=0)PHATE (gamma=0)PHATE (logCP10k)PHATE (logCP10k)UMAP PCA (logCP10k, 1…UMAP PCA (logCP10k, 1kHVG)UMAP (logCP10k, 1kHVG)UMAP (logCP10k, 1kHVG)densMAP PCA (logCP10k…densMAP PCA (logCP10k, 1kHVG)densMAP (logCP10k, 1k…densMAP (logCP10k, 1kHVG)PHATE (default)PHATE (default)Random FeaturesRandom FeaturesSpectral FeaturesSpectral Features0.490.6180.7460.8741.00200.250.50.751rawscaled
  • local continuity meta criterionhigher better
    True FeaturesTrue Featurest-SNE (logCP10k)t-SNE (logCP10k)densMAP PCA (logCP10k)densMAP PCA (logCP10k)densMAP (logCP10k)densMAP (logCP10k)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k)UMAP PCA (logCP10k)UMAP PCA (logCP10k)UMAP (logCP10k)UMAP (logCP10k)t-SNE (logCP10k, 1kHV…t-SNE (logCP10k, 1kHVG)NeuralEE (CPU) (Defau…NeuralEE (CPU) (Default)densMAP PCA (logCP10k…densMAP PCA (logCP10k, 1kHVG)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k, 1kHVG)PHATE (logCP10k)PHATE (logCP10k)NeuralEE (CPU) (logCP…NeuralEE (CPU) (logCP10k, 1kHVG)densMAP (logCP10k, 1k…densMAP (logCP10k, 1kHVG)UMAP PCA (logCP10k, 1…UMAP PCA (logCP10k, 1kHVG)UMAP (logCP10k, 1kHVG)UMAP (logCP10k, 1kHVG)PCA (logCP10k, 1kHVG)PCA (logCP10k, 1kHVG)PHATE (gamma=0)PHATE (gamma=0)PHATE (default)PHATE (default)PHATE (logCP10k, 1kHV…PHATE (logCP10k, 1kHVG)PCA (logCP10k)PCA (logCP10k)PyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k)PyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k, 1kHVG)Diffusion mapsDiffusion mapsSpectral FeaturesSpectral FeaturesRandom FeaturesRandom Features-3.4e-30.2460.4950.7450.99400.250.50.751rawscaled
  • local propertyhigher better
    True FeaturesTrue FeaturesdensMAP PCA (logCP10k)densMAP PCA (logCP10k)UMAP PCA (logCP10k)UMAP PCA (logCP10k)densMAP (logCP10k)densMAP (logCP10k)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k)t-SNE (logCP10k)t-SNE (logCP10k)UMAP (logCP10k)UMAP (logCP10k)PHATE (logCP10k)PHATE (logCP10k)PHATE (logCP10k, 1kHV…PHATE (logCP10k, 1kHVG)densMAP PCA (logCP10k…densMAP PCA (logCP10k, 1kHVG)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k, 1kHVG)t-SNE (logCP10k, 1kHV…t-SNE (logCP10k, 1kHVG)UMAP PCA (logCP10k, 1…UMAP PCA (logCP10k, 1kHVG)NeuralEE (CPU) (logCP…NeuralEE (CPU) (logCP10k, 1kHVG)NeuralEE (CPU) (Defau…NeuralEE (CPU) (Default)PHATE (gamma=0)PHATE (gamma=0)PCA (logCP10k, 1kHVG)PCA (logCP10k, 1kHVG)densMAP (logCP10k, 1k…densMAP (logCP10k, 1kHVG)PCA (logCP10k)PCA (logCP10k)PHATE (default)PHATE (default)UMAP (logCP10k, 1kHVG)UMAP (logCP10k, 1kHVG)Diffusion mapsDiffusion mapsPyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k, 1kHVG)PyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k)Spectral FeaturesSpectral FeaturesRandom FeaturesRandom Features2.8e-30.2530.5020.7521.00200.250.50.751rawscaled
  • trustworthinesshigher better
    True FeaturesTrue Featurest-SNE (logCP10k)t-SNE (logCP10k)UMAP PCA (logCP10k)UMAP PCA (logCP10k)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k)densMAP PCA (logCP10k)densMAP PCA (logCP10k)PHATE (gamma=0)PHATE (gamma=0)t-SNE (logCP10k, 1kHV…t-SNE (logCP10k, 1kHVG)PHATE (default)PHATE (default)PHATE (logCP10k)PHATE (logCP10k)UMAP PCA (logCP10k, 1…UMAP PCA (logCP10k, 1kHVG)PyMDE Preserve Neighb…PyMDE Preserve Neighbors (logCP10k, 1kHVG)densMAP PCA (logCP10k…densMAP PCA (logCP10k, 1kHVG)UMAP (logCP10k, 1kHVG)UMAP (logCP10k, 1kHVG)densMAP (logCP10k, 1k…densMAP (logCP10k, 1kHVG)PHATE (logCP10k, 1kHV…PHATE (logCP10k, 1kHVG)NeuralEE (CPU) (Defau…NeuralEE (CPU) (Default)densMAP (logCP10k)densMAP (logCP10k)UMAP (logCP10k)UMAP (logCP10k)NeuralEE (CPU) (logCP…NeuralEE (CPU) (logCP10k, 1kHVG)PCA (logCP10k)PCA (logCP10k)Diffusion mapsDiffusion mapsPyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k)PCA (logCP10k, 1kHVG)PCA (logCP10k, 1kHVG)PyMDE Preserve Distan…PyMDE Preserve Distances (logCP10k, 1kHVG)Spectral FeaturesSpectral FeaturesRandom FeaturesRandom Features0.4990.6250.750.875100.250.50.751rawscaled
QC: Indicator table 1 error32 warnings

Automated checks on the benchmark run and its results: missing values, score scaling, metric ranges and similar. Errors are high-severity issues that usually need a maintainer's attention; warnings are lower-severity signals. Findings that are expected for this task are listed separately as silenced.

1 high-severity issue need review. 553 of 586 checks passed.

  • error Raw results Dataset 'zebrafish_labs' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction dataset id: zebrafish_labs Percentage missing: 60%

Show 32 warnings
  • warning Raw results Metric 'continuity' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction Metric id: continuity Percentage missing: 25%

  • warning Raw results Metric 'lcmc' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction Metric id: lcmc Percentage missing: 25%

  • warning Raw results Metric 'qglobal' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction Metric id: qglobal Percentage missing: 25%

  • warning Raw results Metric 'qlocal' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction Metric id: qlocal Percentage missing: 25%

  • warning Raw results Metric 'qnn' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction Metric id: qnn Percentage missing: 25%

  • warning Raw results Metric 'qnn_auc' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction Metric id: qnn_auc Percentage missing: 25%

  • warning Raw results Method 'densmap_logCP10k' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: densmap_logCP10k Percentage missing: 15%

  • warning Raw results Method 'densmap_logCP10k_1kHVG' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: densmap_logCP10k_1kHVG Percentage missing: 15%

  • warning Raw results Method 'densmap_pca_logCP10k' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: densmap_pca_logCP10k Percentage missing: 15%

  • warning Raw results Method 'densmap_pca_logCP10k_1kHVG' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: densmap_pca_logCP10k_1kHVG Percentage missing: 15%

  • warning Raw results Method 'diffusion_map' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: diffusion_map Percentage missing: 15%

  • warning Raw results Method 'neuralee_default' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: neuralee_default Percentage missing: 15%

  • warning Raw results Method 'neuralee_logCP10k_1kHVG' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: neuralee_logCP10k_1kHVG Percentage missing: 15%

  • warning Raw results Method 'pca_logCP10k' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: pca_logCP10k Percentage missing: 15%

  • warning Raw results Method 'pca_logCP10k_1kHVG' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: pca_logCP10k_1kHVG Percentage missing: 15%

  • warning Raw results Method 'phate_default' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: phate_default Percentage missing: 15%

  • warning Raw results Method 'phate_logCP10k' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: phate_logCP10k Percentage missing: 15%

  • warning Raw results Method 'phate_logCP10k_1kHVG' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: phate_logCP10k_1kHVG Percentage missing: 15%

  • warning Raw results Method 'phate_sqrt' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: phate_sqrt Percentage missing: 15%

  • warning Raw results Method 'pymde_distances_log_cp10k' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: pymde_distances_log_cp10k Percentage missing: 15%

  • warning Raw results Method 'pymde_distances_log_cp10k_hvg' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: pymde_distances_log_cp10k_hvg Percentage missing: 15%

  • warning Raw results Method 'pymde_neighbors_log_cp10k' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: pymde_neighbors_log_cp10k Percentage missing: 15%

  • warning Raw results Method 'pymde_neighbors_log_cp10k_hvg' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: pymde_neighbors_log_cp10k_hvg Percentage missing: 15%

  • warning Raw results Method 'random_features' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: random_features Percentage missing: 15%

  • warning Raw results Method 'spectral_features' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: spectral_features Percentage missing: 15%

  • warning Raw results Method 'true_features' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: true_features Percentage missing: 15%

  • warning Raw results Method 'tsne_logCP10k' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: tsne_logCP10k Percentage missing: 15%

  • warning Raw results Method 'tsne_logCP10k_1kHVG' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: tsne_logCP10k_1kHVG Percentage missing: 15%

  • warning Raw results Method 'umap_logCP10k' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: umap_logCP10k Percentage missing: 15%

  • warning Raw results Method 'umap_logCP10k_1kHVG' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: umap_logCP10k_1kHVG Percentage missing: 15%

  • warning Raw results Method 'umap_pca_logCP10k' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: umap_pca_logCP10k Percentage missing: 15%

  • warning Raw results Method 'umap_pca_logCP10k_1kHVG' %missing

    Percentage of missing results should be less than 10%. Task id: dimensionality_reduction method id: umap_pca_logCP10k_1kHVG Percentage missing: 15%

Method info 11

densMAP is a modification of UMAP that adds an extra cost term in order to preserve information about the relative local density of the data. It is performed on the same inputs as UMAP.

parameter sets tested logCP10klogCP10k, 1kHVG

densMAP is a modification of UMAP that adds an extra cost term in order to preserve information about the relative local density of the data. It is performed on the same inputs as UMAP.

parameter sets tested logCP10klogCP10k, 1kHVG

Diffusion maps uses an affinity matrix to describe the similarity between data points, which is then transformed into a graph Laplacian. The eigenvalue-weighted eigenvectors of the graph Laplacian are then used to create the embedding. Diffusion maps is calculated on the logCPM expression matrix.

NeuralEE is a neural network implementation of elastic embedding. It is a non-linear method that preserves pairwise distances between data points. NeuralEE uses a neural network to optimize an objective function that measures the difference between pairwise distances in the original high-dimensional space and the two-dimensional space. It is computed on both the recommended input from the package authors of 500 HVGs selected from a logged expression matrix (without sequencing depth scaling) and the default logCPM matrix with 1000 HVGs.

parameter sets tested DefaultlogCP10k, 1kHVG

PCA or "Principal Component Analysis" is a linear method that finds orthogonal directions in the data that capture the most variance. The first two principal components are chosen as the two-dimensional embedding. We select only the first two principal components as the two-dimensional embedding. PCA is calculated on the logCPM expression matrix with and without selecting 1000 HVGs.

parameter sets tested logCP10klogCP10k, 1kHVG

PHATE or “Potential of Heat - diffusion for Affinity - based Transition Embedding” uses the potential of heat diffusion to preserve trajectories in a dataset via a diffusion process. It is an affinity - based method that creates an embedding by finding the dominant eigenvalues of a Markov transition matrix. We evaluate several variants including using the recommended square - root transformed CPM matrix as input, this input with the gamma parameter set to zero and the normal logCPM transformed matrix with and without HVG selection.

parameter sets tested defaultlogCP10k, 1kHVGlogCP10kgamma=0

PyMDE is a Python implementation of minimum-distortion embedding. It is a non-linear method that preserves distances between cells or neighborhoods in the high-dimensional space. It is computed with options to preserve distances between cells or neighbourhoods and with the logCPM matrix with and without HVG selection as input.

parameter sets tested logCP10klogCP10k, 1kHVG

PyMDE is a Python implementation of minimum-distortion embedding. It is a non-linear method that preserves distances between cells or neighborhoods in the high-dimensional space. It is computed with options to preserve distances between cells or neighbourhoods and with the logCPM matrix with and without HVG selection as input.

parameter sets tested logCP10klogCP10k, 1kHVG

t-SNE or t-distributed Stochastic Neighbor Embedding converts similarities between data points to joint probabilities and tries to minimize the Kullback-Leibler divergence between the joint probabilities of the low-dimensional embedding and the high-dimensional data. We use the implementation in the scanpy package with the result of PCA on the logCPM expression matrix (with and without HVG selection).

parameter sets tested logCP10klogCP10k, 1kHVG

UMAP or Uniform Manifold Approximation and Projection is an algorithm for dimension reduction based on manifold learning techniques and ideas from topological data analysis. We perform UMAP on the logCPM expression matrix before and after HVG selection and with and without PCA as a pre-processing step.

parameter sets tested logCP10klogCP10k, 1kHVG

UMAP or Uniform Manifold Approximation and Projection is an algorithm for dimension reduction based on manifold learning techniques and ideas from topological data analysis. We perform UMAP on the logCPM expression matrix before and after HVG selection and with and without PCA as a pre-processing step.

parameter sets tested logCP10klogCP10k, 1kHVG
Control method info 3

Randomly generated two-dimensional coordinates from a normal distribution.

Use 1000-dimensional diffusions maps as an embedding

Use of the original feature inputs as the 'embedding'.

Metric info 10
co-KNN AUChigher is betterZhang et al., 2021

co-KNN AUC is area under the co-KNN curve

co-KNN sizehigher is betterZhang et al., 2021

co-KNN size counts how many points are in both k-nearest neighbors before and after the dimensionality reduction

continuityhigher is betterZhang et al., 2021

Continuity measures error of hard extrusions based on nearest neighbor coranking

Density preservationhigher is betterNarayan et al., 2021

Similarity between local densities in the high-dimensional data and the reduced data.

Distance correlationhigher is betterSchober et al., 2018

Spearman correlation between all pairwise Euclidean distances in the original and dimension-reduced data

Distance correlation (spectral)higher is betterCoifman & Lafon, 2006

Spearman correlation between all pairwise diffusion distances in the original and dimension-reduced data

global propertyhigher is betterZhang et al., 2021

The global property metric is a summary of the global co-KNN

local continuity meta criterionhigher is betterZhang et al., 2021

The local continuity meta criterion is the co-KNN size with baseline removal which favors locality

local propertyhigher is betterZhang et al., 2021

The local property metric is a summary of the local co-KNN

trustworthinesshigher is betterVenna & Kaski, 2001

a measurement of similarity between the rank of each point's nearest neighbors in the high-dimensional data and the reduced data.

Dataset info 4

5k Peripheral Blood Mononuclear Cells (PBMCs) from a healthy donor. Sequenced on 10X v3 chemistry in July 2019 by 10X Genomics. 5247 cells x 20822 features with no cell type labels

1.6k hematopoietic stem and progenitor cells from mouse bone marrow. Sequenced by Smart-seq2. 1920 cells x 43258 features with 3 cell type labels

Myeloid lineage differentiation from mouse blood. Sequenced by SMARTseq in 2016 by Olsson et al. 660 cells x 112815 features with 4 cell type labels

90k cells from zebrafish embryos throughout the first day of development, with and without a knockout of chordin, an important developmental gene. Dimensions: 26022 cells, 25258 genes. 24 cell types (avg. 1084±1156 cells per cell type).

references
  1. Agrawal, A., Ali, A., & Boyd, S. (2021). Minimum-Distortion Embedding. Foundations and Trends® in Machine Learning, 14(3), 211–378. 10.1561/2200000090 ↗
  2. Coifman, R. R., & Lafon, S. (2006). Diffusion maps. Applied and Computational Harmonic Analysis, 21(1), 5–30. 10.1016/j.acha.2006.04.006 ↗
  3. Open Problems for Single Cell Analysis Consortium. (2022). Open Problems. link ↗
  4. van der Maaten, L., & Hinton, G. (2008). Visualizing Data using t-SNE. Journal of Machine Learning Research, 9(86), 2579–2605. link ↗
  5. McInnes, L., Healy, J., & Melville, J. (2018). UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv. 10.48550/arxiv.1802.03426 ↗
  6. Moon, K. R., van Dijk, D., Wang, Z., Gigante, S., Burkhardt, D. B., Chen, W. S., Yim, K., van den Elzen, A., Hirn, M. J., Coifman, R. R., Ivanova, N. B., Wolf, G., & Krishnaswamy, S. (2019). Visualizing structure and transitions in high-dimensional biological data. Nature Biotechnology, 37(12), 1482–1492. 10.1038/s41587-019-0336-3 ↗
  7. Narayan, A., Berger, B., & Cho, H. (2021). Assessing single-cell transcriptomic variability through density-preserving data visualization. Nature Biotechnology, 39(6), 765–774. 10.1038/s41587-020-00801-7 ↗
  8. Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 2(11), 559–572. 10.1080/14786440109462720 ↗
  9. Schober, P., Boer, C., & Schwarte, L. A. (2018). Correlation Coefficients. Anesthesia & Analgesia, 126(5), 1763–1768. 10.1213/ane.0000000000002864 ↗
  10. Venna, J., & Kaski, S. (2001). Neighborhood Preservation in Nonlinear Projection Methods: An Experimental Study. Artificial Neural Networks — ICANN 2001, 485–491. 10.1007/3-540-44668-0\_68 ↗
  11. Xiong, J., Gong, F., Wan, L., & Ma, L. (2020). NeuralEE: A GPU-Accelerated Elastic Embedding Dimensionality Reduction Method for Visualizing Large-Scale scRNA-Seq Data. Frontiers in Genetics, 11. 10.3389/fgene.2020.00786 ↗
  12. Zhang, Y., Shang, Q., & Zhang, G. (2021). pyDRMetrics - A Python toolkit for dimensionality reduction quality assessment. Heliyon, 7(2), e06199. 10.1016/j.heliyon.2021.e06199 ↗