G2P¶
Genotype-to-phenotype prediction for bacteria: eight binary phenotypes for 13,669
strains, each represented by the 2,047 Pfam protein-domain families present in its
genome, from the BacDive-AI release. Every trait is registered twice —
bacdive_<trait> uses a random split, bacdive_clade_<trait> holds out whole
gene-content clades, since strains of one species share ~99% of their gene content
and a random split places near-duplicates on both sides. Accuracy is a weak summary
at these prevalences (2.6%–55%), so the tables below report macro-F1. Note that
Aerobic and Anaerobic mean obligate and are mutually exclusive; facultative
organisms are negative for both.
Five entries, all on identical splits: g2p_multitask (shared 512/256 trunk, one
head per trait, masked loss), g2p_mlp (per-trait MLP with the same hidden sizes —
the control that isolates cross-trait sharing from simply using a neural network),
g2p_gbm, g2p_logistic and g2p_rf (the architecture of the published BacDive-AI
models). Bold marks the best macro-F1 per row.
Reference: BacDive-AI · BacDive · Zenodo
Random split — bacdive (macro-F1)¶
| Phenotype | %pos | n test | multitask | mlp | gbm | logistic | rf |
|---|---|---|---|---|---|---|---|
| Acidophilic | 2.7 | 500 | 0.7333 | 0.6939 | 0.7171 | 0.7449 | 0.5601 |
| Gram-positive | 37.6 | 919 | 0.9546 | 0.9468 | 0.9546 | 0.9460 | 0.9570 |
| Spore-forming | 31.5 | 580 | 0.9367 | 0.9420 | 0.9463 | 0.9365 | 0.9483 |
| Aerobic (obligate) | 55.4 | 1567 | 0.8911 | 0.9059 | 0.9106 | 0.8624 | 0.9072 |
| Anaerobic (obligate) | 21.1 | 1621 | 0.9612 | 0.9641 | 0.9668 | 0.9644 | 0.9655 |
| Thermophilic | 4.3 | 2647 | 0.9547 | 0.9315 | 0.9318 | 0.9444 | 0.9181 |
| Psychrophilic | 2.6 | 2614 | 0.7084 | 0.6699 | 0.6330 | 0.6828 | 0.5500 |
| Flagellated motility | 18.6 | 434 | 0.9763 | 0.9795 | 0.9897 | 0.9830 | 0.9897 |
| MEAN | 0.8895 | 0.8792 | 0.8812 | 0.8831 | 0.8495 |
Clade holdout — bacdive_clade (macro-F1)¶
| Phenotype | %pos | n test | multitask | mlp | gbm | logistic | rf |
|---|---|---|---|---|---|---|---|
| Acidophilic | 2.7 | 657 | 0.5938 | 0.4942 | 0.5571 | 0.5530 | 0.4942 |
| Gram-positive | 37.6 | 1166 | 0.9668 | 0.9689 | 0.9698 | 0.9517 | 0.9707 |
| Spore-forming | 31.5 | 670 | 0.8653 | 0.8909 | 0.8933 | 0.8698 | 0.8892 |
| Aerobic (obligate) | 55.4 | 1740 | 0.9179 | 0.9222 | 0.9260 | 0.8600 | 0.9188 |
| Anaerobic (obligate) | 21.1 | 1789 | 0.9684 | 0.9692 | 0.9700 | 0.9587 | 0.9645 |
| Thermophilic | 4.3 | 2638 | 0.9359 | 0.9273 | 0.8968 | 0.9247 | 0.7907 |
| Psychrophilic | 2.6 | 2577 | 0.5763 | 0.6426 | 0.6143 | 0.6484 | 0.4930 |
| Flagellated motility | 18.6 | 657 | 0.9214 | 0.9447 | 0.9715 | 0.9668 | 0.9754 |
| MEAN | 0.8432 | 0.8450 | 0.8498 | 0.8416 | 0.8121 |
All sixteen benchmarks at a glance¶
The heat map below covers all sixteen benchmarks — the eight bacdive
(random split) columns first, then the eight bacdive_clade (clade holdout)
columns — for all five models. Note it plots accuracy, the metric this
category scores on, not the macro-F1 of the tables above; the two disagree, which
is the point made below.
Gradient boosting is the strongest single model under clade holdout (0.8498);
multi-task sharing is third (0.8432) and is matched by the per-trait MLP (0.8450),
so on aggregate the sharing buys little over using a neural network at all. Its
advantage is confined to the starved traits: acidophily has 70 positives in the
whole dataset, and under clade holdout g2p_multitask reaches 0.594 where the MLP
and random forest sit at majority-class performance (0.494).