Machine Learning

Script Overview

ppi_ml_croissant.py is a Python script for training and evaluating machine learning models on the ELIXIR PPI benchmark dataset with advanced cross-validation, feature evaluation, and structural feature extraction from PDB/mmCIF files. It provides an end-to-end machine learning pipeline for protein-protein interaction (PPI) classification. It combines:

Quick Start

Basic Usage

# Basic usage with local dataset
python ppi_ml_croissant.py --local ./bioschemas_output

# With structural feature extraction (mmCIF format)
python ppi_ml_croissant.py --local ./bioschemas_output --extract-pdb-features

# With PDB format
python ppi_ml_croissant.py --local ./bioschemas_output --extract-pdb-features --pdb-format

# Quick mode (reduced output)
python ppi_ml_croissant.py --local ./bioschemas_output --quick

Advanced Usage

# Full pipeline with all features
python ppi_ml_croissant.py \
  --local ./bioschemas_output \
  --extract-pdb-features \
  --pdb-local-dir ./structure_files \
  --evaluate-features \
  --feature-analysis \
  --save-plots \
  --output-dir ./ml_results \
  --folds 5 \
  --random-state 42 \
  --log-level INFO

# Download structure files only
python ppi_ml_croissant.py --local ./bioschemas_output --extract-pdb-features --pdb-features-only

# Train/test split instead of CV
python ppi_ml_croissant.py --local ./bioschemas_output --test-split 0.2

Output

Typical Console Output

Generated Files Structure

ml_results/
├── group_kfold_cv_results.json          # Complete ML results with all metrics
├── structural_features.csv              # Extracted PDB/mmCIF interface features
├── feature_evaluation_report.json       # Comprehensive feature analysis
├── group_kfold_cv_results.png           # CV performance visualization
├── feature_evaluation_plots.png         # Feature analysis visualizations
├── feature_importance.png               # Feature importance ranking
└── feature_ranking_analysis.png         # Comparison of feature ranking methods

Example JSON Output (group_kfold_cv_results.json)

{
  "Random Forest": {
    "cv_metrics": {
      "accuracy": 0.827,
      "precision": 0.831,
      "recall": 0.824,
      "f1": 0.827,
      "roc_auc": 0.898
    },
    "overall_metrics": {
      "accuracy": 0.829,
      "precision": 0.833,
      "recall": 0.826,
      "f1": 0.829,
      "roc_auc": 0.901
    },
    "fold_metrics": {
      "accuracies": [0.819, 0.831, 0.825, 0.834, 0.826],
      "precisions": [0.822, 0.836, 0.828, 0.839, 0.830],
      "recalls": [0.816, 0.830, 0.822, 0.833, 0.825],
      "f1_scores": [0.819, 0.833, 0.825, 0.836, 0.827],
      "roc_aucs": [0.895, 0.901, 0.897, 0.903, 0.894]
    }
  },
  "cross_validation_settings": {
    "n_splits": 5,
    "method": "GroupKFold cross-validation (ClusterID as groups)",
    "random_state": 42,
    "best_model": "Random Forest"
  }
}

Visual Output Example

Pipeline Workflow

Command Line Options

Structural Feature Extraction

File Handling Features

# BioPython can parse gzipped files directly - no decompression needed!
- 1abc.pdb.gz    → Direct parsing
- 1abc.cif.gz    → Direct parsing
- 1abc.pdb       → Regular parsing
- 1abc.cif       → Regular parsing

# Case sensitivity handling (both work)
- 1ABC.pdb.gz    → Uppercase PDB ID
- 1abc.pdb.gz    → Lowercase PDB ID
- MixedCase.pdb.gz → Automatic detection

Example Structural Features Output

interface_id,extraction_success,pdb_file,chain_ids,pdb_avg_total_interface_residues,...
1ABC_1,True,1abc.cif.gz,A-B,24.5,...
1DEF_1,True,1def.pdb.gz,C-D,18.2,...
2GHI_1,False,2ghi.cif.gz,E-F,NaN,...  # Extraction failed
3JKL_1,True,3jkl.cif.gz,G-H,32.1,...

Machine Learning Models

Typical Performance Metrics (Example Output)

Complete Examples

Example 1: Full ML Pipeline with Structural Features

# Run complete ML pipeline with structural features
python ppi_ml_croissant.py \
  --local ./bioschemas_output \
  --extract-pdb-features \
  --pdb-local-dir ./structures \
  --pdb-format \
  --evaluate-features \
  --feature-analysis \
  --save-plots \
  --output-dir ./full_ml_run \
  --folds 5 \
  --random-state 123 \
  --log-level INFO

# Expected output in ./full_ml_run:
# - group_kfold_cv_results.json      # JSON results
# - structural_features.csv          # PDB features
# - feature_evaluation_report.json   # Feature analysis
# - group_kfold_cv_results.png       # Main visualization
# - 3-4 additional plot files        # Detailed analyses

Example 2: Quick Test Run

# Quick analysis for testing (minimal output)
python ppi_ml_croissant.py \
  --local ./bioschemas_output \
  --quick \
  --output-dir ./quick_test

# Console output shows:
# - Dataset statistics
# - Best model performance
# - Quick summary metrics
# Generated files are minimal

Example 3: Download Structure Files Only

# Just download structure files without ML
python ppi_ml_croissant.py \
  --local ./bioschemas_output \
  --extract-pdb-features \
  --pdb-features-only \
  --pdb-local-dir ./structure_files

# Output shows:
# - Files available locally
# - Files downloaded
# - Files missing
# No ML training performed

Integration with ELIXIR FAIR Metadata