Openly published, widely quoted in AI-in-medicine talks, and often shown with a benchmark score. Useful for seeing what such a score actually rests on.

GreyBrain Visual Model Card Verified · Hugging Face

Benchmark Results

aaditya/Llama3-OpenBioLLM-8B

Advancing Open-source Large Language Models in Medical Domain Online Demo | GitHub | Paper | Discord Introducing OpenBioLLM-8B: A State-of-the-Art Open Source Biomedical Large Language Model OpenBioLLM-8B is an advanced open source language model designed specifically for the biomedical domain. Developed by Saama AI Labs, this model leverages cutting-edge techniques to achieve state-of-the-art performance on a wide range of biomedical tasks. …

Open on Hugging Face

From the model card

Taken straight from the Hugging Face model card, quoted not rewritten.

Training data

from “Training procedure”

Training hyperparameters

learning_rate: 0.0002
lr_scheduler: cosine
train_batch_size: 12
eval_batch_size: 8
GPU: H100 80GB SXM5
num_devices: 1
optimizer: adamw_bnb_8bit
lr_scheduler_warmup_steps: 100
num_epochs: 4

Peft hyperparameters

adapter: qlora
lora_r: 128
lora_alpha: 256
lora_dropout: 0.05
lora_target_linear: true

-lora_target_modules:

q_proj
v_proj
k_proj
o_proj
gate_proj
down_proj
up_proj

Training results

Framework versions

Transformers 4.39.3
Pytorch 2.1.2+cu121
Datasets 2.18.0
Tokenizers 0.15.1
Axolotl
Lm harness for evaluation

Benchmark Results

…

Read the full model card

AI models can make mistakes. All details on this card are taken from the Hugging Face model card.