facebookresearch / tribev2

What is tribev2?

TRIBE v2 predicts fMRI brain responses to videos, audio, or text using a multimodal transformer that fuses LLaMA for language, V-JEPA2 for vision, and Wav2Vec-BERT for sound, mapping to fsaverage cortical surfaces. This GitHub repository contains code to load pretrained weights from Hugging Face, run inference on any media file, and train new models on fMRI datasets—all in Python with PyTorch Lightning. Users get vertex-wise predictions for the "average" brain in seconds via a simple API like `model.predict(events)`.

Why is it gaining traction?

It stands out with dead-simple inference (pip install, Colab-ready) and built-in brain visualizations via PyVista or Nilearn, skipping weeks of feature extraction boilerplate. Pretrained on diverse fMRI studies like Algonauts and BOLD Moments, it beats unimodal baselines on cortical encoding—ideal for quick prototyping without managing massive feature caches. The repo's clean structure avoids common GitHub pains like out of bounds symlinks or unmerged paths, with grids for Slurm training.

Who should use this?

Computational neuroscientists benchmarking brain encoding models on naturalistic stimuli. ML researchers in vision-language-neuroscience hybrids needing fMRI predictions for videos or narratives. Devs exploring in-silico neuroscience who want Hugging Face-style pretrained brain models without custom pipelines.

Verdict

Grab it for neuroscience ML experiments—pretrained weights and demos make it instantly useful despite 365 stars and 1.0% credibility score signaling early maturity. Docs are solid via README and Colab; add tests for production. (198 words)