➡️ ADVANCED Vision Fine-tuning Repo: https://trelis.com/advanced-vision/
➡️ ADVANCED-inference Repo: https://trelis.com/enterprise-server-...
➡️ ADVANCED-fine-tuning Repo: https://trelis.com/advanced-fine-tuni...
➡️ Trelis Function-calling Models and Scripts: https://trelis.com/function-calling/
➡️ ADVANCED Transcription Repo: https://trelis.com/advanced-transcrip...
➡️ One-click Fine-tuning & Inference Templates: https://github.com/TrelisResearch/one...
➡️ Trelis Newsletter: https://Trelis.Substack.com
➡️ Trelis Resources and Support: https://Trelis.com/About

Affiliate Links (support the channel):
- Vast AI - https://cloud.vast.ai/?ref_id=98762
- RunPod - https://tinyurl.com/4b6ecbbn

*Video Resources*
Slides: https://docs.google.com/presentation/...
One-click RunPod / VastAI Templates: https://github.com/TrelisResearch/ins...
IDEFICS: https://huggingface.co/HuggingFaceM4/...
LLaVA: https://llava.hliu.cc/
Trelis Newsletter: Trelis.Substack.com

Chapters:
0:00 Fine-tuning Multi-modal Models
0:16 Overview
1:30 LLaVA vs ChatGPT
4:53 Applications
5:37 Multi-modal model architecture
9:05 Vision Encoder architecture
14:00 LLaVA 1.5 architecture
16:30 LLaVA 1.6 architecture
18:30 IDEFICS architecture
22:00 Data creation
24:11 Dataset creation
25:29 Fine-tuning
34:25 Inference and Evaluation
37:34 Data loading
40:00 LoRA setup
42:52 Recap so far
43.25 Evaluation pre-training
44:26 Training
45:40 Evaluation post-training
46:45 Technical clarifications
50:29 Summary