FAIR Metadata

Transforming ELIXIR SCO benchmark experiments into FAIR-compliant, machine learning-ready resource

Overview

This repository provides FAIR-compliant metadata for the BE1 benchmark dataset from the ELIXIR Single Cell Omics Implementation Study (WP3) – a 10XGenomics single-cell RNA sequencing experiment providing controlled cancer heterogeneity using seven lung cancer cell lines with distinct driver mutations: EGFR, ALK, MET, ERBB2, KRAS, BRAF, ROS1.

The be1_query.py extracts single cell expression data from the GEO repository

Standard Purpose Compliance
Single Cell Schema v0.4 Darwin Core extension for single-cell RNA-seq ✅ Full (16 sheets)
CITATION.cff Software citation ✅ v1.2.0
Schema.org Dataset discovery ✅ JSON-LD compatible
EDAM Ontology Bioinformatics operations ✅ Integrated
PubMed/PMC Literature references ✅ PMID: 38307867

Quick Start

# Clone repository
git clone https://github.com/biofold/sco-benchmark-fair.git
cd sco-benchmark-fair

# Install requirements
pip install -r requirements.txt

# Basic gene query (uses embedded defaults)
python scripts/be1_query.py --gene EGFR,KRAS,BRAF

# Generate enriched metadata (as JSON)
python scripts/be1_query.py \
  --geo GSE243665 \
  --metadata sco-benchmark-experiment-1.json \
  --gene EGFR,KRAS,BRAF \
  --output results.csv \
  --verbose
        

Dataset Statistics

29,528
Total Cells
7
Cell Lines
8
Samples
16/16
Schema Sheets
Cell Line Driver Mutation Cells Mean Reads/Cell Median Genes Saturation
PC9EGFR Del194,49228,9842,86847.8%
A549KRAS G12S6,89829,8643,30760.1%
CCL-185-IGEML4-ALK fusion6,35429,8592,81060.6%
NCI-H1395BRAF G469A2,67327,9722,82248.0%
DV90ERBB2 V842I2,99826,7562,38741.4%
HCC78ROS1 fusion2,74824,7002,57038.7%
NCI-H596MET Del142,96528,6663,11356.7%
PBMCHealthy control500———

Total sequencing: 2.46 billion reads, ≥71.26% bases ≥Q30 on Illumina NovaSeq 6000

FAIR Compliance Evaluation

This repository has been evaluated using comprehensive FAIR principles analysis. Overall Score: 80/100

Findable (80/100) ✅

  • Strengths: Rich metadata with Single Cell Schema v0.4, persistent identifiers (GEO:GSE243665, DOI:10.1038/s41597-024-03002-8).
  • Improvements Needed (Low Priority, +5 pts each):
    • Add relevant repository topics on GitHub (e.g., single-cell, rnaseq, benchmark).
    • Create GitHub releases with version tags for stable versions.
    • Enable and populate the GitHub documentation wiki.
    • Include license information in metadata files.

Accessible (80/100) ✅

  • Strengths: Open access CC-BY-NC 4.0 license, standard protocols (HTTPS/FTP) for data access, multiple data formats (MTX, HDF5, FASTQ).
  • Improvements Needed (Low Priority, +20 pts):
    • Specify access rights (open, restricted, closed) explicitly in metadata.

Interoperable (75/100) ⚠️

  • Strengths: Single Cell Schema v0.4 (Darwin Core compliant), controlled vocabularies for library prep and sequencing, cross-references to PMIDs.
  • Improvements Needed:
    • (Low Priority, +15 pts) Use controlled vocabularies/ontologies more extensively for metadata fields.
    • (Low Priority, +10 pts) Include format information (MIME types, file extensions) in metadata.

Reusable (85/100) ✅

  • Strengths: Full provenance (Cell Ranger v7.1.0 pipeline), clear contact information, usage examples (R Shiny App, Python tool), explicit license and citation files.
  • Improvements Needed (Low Priority, +5 pts):
    • Add .github/ISSUE_TEMPLATE/ for structured bug reports and feature requests.
  • Medium Priority (+10 pts):
    • Include contact information (email, ORCID) more prominently in metadata.

Note: The scores and improvement items above are specifically for the biofold/sco-benchmark-fair repository. Implementing the listed "Improvements Needed" can raise the overall FAIR score to 100/100.

Metadata Architecture

Generated File Structure

sco-benchmark-fair/
├── scrna-schema/
│   ├── sco-benchmark-experiment-1.json      # Complete BE1 metadata (16 sheets)
│   └── schema.json                           # Single Cell Schema v0.4 definition
├── scripts/
│   ├── be1_query.py                          # Python query tool
│   ├── test_fair_gitrepo.py                  # Python FAIR repo evaluation
│   └── generate_fair_report.py               # Python FAIR Report Generator
├── CITATION.cff                              # Citation metadata
├── LICENSE.cff                                # CC BY 4.0 license
├── requirements.txt                           # Python dependencies
└── index.html                                 # FAIRification report (this page)
        

Data Schema (Single Cell v0.4)

{
  "manifest": {
    "key": "version_dwc_sc_rnaseq",
    "name": "Single-cell Ribonucleic Acid Sequencing [Darwin Core]",
    "standard": "dwc",
    "technology": "sc_rnaseq",
    "manifest_version": "0.4"
  },
  "study": [ /* study metadata */ ],
  "person": [ /* author information */ ],
  "sample": [ /* 8 sample entries */ ],
  "dissociation": [ /* protocol details */ ],
  "cell_suspension": [ /* per-line cell counts */ ],
  "lib_prep": [ /* 10X Genomics v3.1 */ ],
  "sequencing": [ /* NovaSeq 6000 parameters */ ],
  "analysis_derived_data": [ /* QC metrics */ ],
  "raw_data_processing": [ /* Cell Ranger v7.1.0 */ ],
  "downstream_processing": [ /* validation clustering */ ],
  "data_availability_checklist": [ /* access points */ ],
  "file": [ /* 24+ file listings */ ],
  "expression_data_process_setting": [ /* normalization */ ],
  "expression_data_file": [ /* matrix details */ ]
}
        

Usage Examples

Python Query Tool

import scanpy as sc
import pandas as pd

# Load A549 data
adata = sc.read_10x_mtx(
    './be1_data',
    prefix='GSE243665_A549_',
    var_names='gene_symbols'
)

# Query EGFR expression
egfr_idx = list(adata.var_names).index('EGFR')
expression = adata.X[:, egfr_idx].toarray().flatten()

results = pd.DataFrame({
    'cell_barcode': adata.obs_names,
    'EGFR_expression': expression,
    'EGFR_log1p': np.log1p(expression)
})
print(f"Mean EGFR: {results['EGFR_expression'].mean():.2f}")
                    

R Shiny App

# Install BE1 package
install.packages("devtools")
devtools::install_github("kendomaniac/BE1")

# Launch Shiny app
library(BE1)
# Run the mixing app at:
# http://aisc.hpc4ai.unito.it:3838/
                    

The Shiny app enables generation of controlled mixtures by blending cell lines at user-defined ratios.

Command Line Options

# Full metadata query with all options
python scripts/be1_query.py \
  --geo GSE243665 \                           # GEO accession
  --metadata scrna-schema/sco-benchmark-experiment-1.json \
  --gene EGFR,KRAS,BRAF \                      # Comma-separated genes
  --cell AAACCCAAGAGG-1 \                       # Cell barcode search
  --cell-line A549 \                            # Filter by cell line
  --min-umi 2 \                                 # Expression threshold
  --output results.csv \                         # Save results
  --verbose                                      # Detailed logging
        

Key Features & Innovations

Controlled Heterogeneity

Seven lung cancer cell lines with distinct driver mutations (EGFR, KRAS, BRAF, ALK, ROS1, MET, ERBB2) plus PBMC control for benchmarking cancer heterogeneity analysis.

Single Cell Schema v0.4

Complete Darwin Core extension implementation with 16 standardized metadata sheets covering all experimental aspects.

Interactive Tools

R Shiny app for generating controlled mixtures and Python query tool for expression extraction with comma-separated gene/cell support.

Multiple Data Access Points

Dataset available through GEO (GSE243665), SRA (PRJNA1019356), and figshare with comprehensive documentation.

FAIR Improvements Roadmap

Based on the FAIR assessment (80/100), here are prioritized improvements to reach a higher score. Each item includes potential point gain toward the target of 100/100.

Findable Improvements

Priority Action Points
Low Add relevant repository topics to GitHub settings (single-cell, rnaseq, benchmark, fair-data, cancer, scrna-seq) +5
Low Create GitHub releases with version tags for stable versions (v1.0.0, v1.1.0) +5
Low Enable and populate GitHub Pages documentation wiki +5
Low Include license information in metadata files +5
Subtotal Findable +20

Accessible Improvements

Priority Action Points
Low Specify access rights (open, restricted, closed) explicitly in metadata +20
Subtotal Accessible +20

Interoperable Improvements

Priority Action Points
Low Use controlled vocabularies/ontologies more extensively for metadata fields +15
Low Include format information (MIME types, file extensions) in metadata +10
Subtotal Interoperable +25

Reusable Improvements

Priority Action Points
Medium Include contact information (email, ORCID) more prominently in metadata +10
Low Add .github/ISSUE_TEMPLATE/ for structured bug reports and feature requests +5
Subtotal Reusable +15

Implementation Priorities

Quick Wins (Low Effort)

  • Add repository topics
  • Create GitHub releases
  • Add issue templates
  • Include format information

Medium Impact

  • Contact information in metadata
  • Enable documentation wiki
  • License information in metadata

High Value

  • Access rights specification (+20 pts)
  • Controlled vocabularies (+15 pts)
  • Zenodo DOI registration

Implementing all improvements will raise the FAIR score from 80/100 to 100/100, making the repository fully FAIR-compliant.

License and Attribution

License

Creative Commons Attribution 4.0 International (CC BY-NC 4.0)
This work is licensed under a Creative Commons Attribution 4.0 International License.

Citation

@article{arigoni2024single,
   title={A single cell RNAseq benchmark experiment embedding 'controlled' cancer heterogeneity},
   author={Arigoni, Maddalena and Ratto, Maria Luisa and Riccardo, Federica and Balmas, Elisa and Calogero, Lorenzo and Cordero, Francesca and Beccuti, Marco and Calogero, Raffaele A. and Alessandri, Luca},
   journal={Scientific Data},
   volume={11},
   number={1},
   pages={159},
   year={2024},
   publisher={Nature Publishing Group},
   doi={10.1038/s41597-024-03002-8},
   pmid={38307867},
   pmcid={PMC10837414}
}

@dataset{elixir2024fairmetadata,
   title={BE1 Benchmark Dataset for Single-Cell RNA Sequencing: Controlled Cancer Heterogeneity FAIR Metadata},
   author={{ELIXIR Single Cell Omics Community}},
   year={2024},
   version={1.0},
   publisher={Zenodo},
   doi={10.5281/zenodo.XXXXXXXX}  # Placeholder
}
        

Dataset Reference Publication

"A single cell RNAseq benchmark experiment embedding 'controlled' cancer heterogeneity"
📄 https://doi.org/10.1038/s41597-024-03002-8

Contact

Related Resources

Resource Description Link
ELIXIR Single Cell Omics Community platform elixir-europe.org
Source Code This repository GitHub
BE1 R Package Shiny app and supporting code kendomaniac/BE1
Single Cell Schema Darwin Core extension singlecellschemas.org
GEO Dataset Count matrices GSE243665
figshare Data Additional matrices and CCLE data 10.6084/m9.figshare.23939481

Contributing

We welcome contributions to improve FAIR compliance! See our GitHub Issues for current improvement tasks.

Support

For questions or support, please:

  1. Check the FAIR Improvements Roadmap section
  2. Review existing GitHub Issues
  3. Create a new issue with detailed description

This project is part of the ELIXIR Single Cell Omics Community's efforts to promote FAIR data practices in structural bioinformatics.