Library
Research papers and free AI books
We link only to copies the authors or publishers share themselves, such as arXiv or the book's own site, and say who hosts each one. New to papers? Start with Attention Is All You Need, read the figures and the conclusion first, then the method.
Papers: Language models
Attention Is All You Need
Ashish Vaswani and others · 2017 · I2 RAG builder
The paper that introduced the Transformer, the design behind every modern LLM. Read the architecture figure and section 3 first.
Download PDF: Attention Is All You Need (opens in a new tab)Hosted by arXiv, posted by the authorsOutrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Noam Shazeer and others · 2017 · P1 Quality engineer
The sparsely-gated mixture-of-experts layer: many expert networks, only a few used per input. The idea behind Mixtral and others.
Download PDF: Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer (opens in a new tab)Hosted by arXiv, posted by the authorsBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Jacob Devlin and others · 2018 · I2 RAG builder
Showed that pre-training a Transformer on plain text, then fine-tuning it, beats task-specific models. The start of today's embedding models.
Download PDF: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (opens in a new tab)Hosted by arXiv, posted by the authorsExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Colin Raffel and others · 2019 · I3 Agent builder
T5 treats every language task as text in, text out, and compares many pre-training choices side by side.
Download PDF: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (opens in a new tab)Hosted by arXiv, posted by the authorsLanguage Models are Unsupervised Multitask Learners
Alec Radford and others · 2019 · I2 RAG builder
GPT-2: a language model trained only to predict the next word picks up translation, summaries and question answering on its own.
Language Models are Few-Shot Learners
Tom B. Brown and others · 2020 · I2 RAG builder
GPT-3: a very large model can do new tasks from a few examples in the prompt, with no retraining. The birth of prompting.
Download PDF: Language Models are Few-Shot Learners (opens in a new tab)Hosted by arXiv, posted by the authorsScaling Laws for Neural Language Models
Jared Kaplan and others · 2020 · I3 Agent builder
Model quality improves predictably with more data, parameters and compute. Explains why labs keep building bigger models.
Download PDF: Scaling Laws for Neural Language Models (opens in a new tab)Hosted by arXiv, posted by the authorsSwitch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
William Fedus and others · 2021 · P1 Quality engineer
Switch Transformers simplified mixture-of-experts so trillion-parameter models became practical to train.
Download PDF: Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity (opens in a new tab)Hosted by arXiv, posted by the authorsLoRA: Low-Rank Adaptation of Large Language Models
Edward J. Hu and others · 2021 · I2 RAG builder
LoRA: fine-tune a huge model by training small add-on matrices instead of all its weights. Cheap and widely used.
Download PDF: LoRA: Low-Rank Adaptation of Large Language Models (opens in a new tab)Hosted by arXiv, posted by the authorsTraining language models to follow instructions with human feedback
Long Ouyang and others · 2022 · I3 Agent builder
How OpenAI turned GPT-3 into a helpful assistant with human feedback (RLHF). The recipe behind ChatGPT.
Download PDF: Training language models to follow instructions with human feedback (opens in a new tab)Hosted by arXiv, posted by the authorsTraining Compute-Optimal Large Language Models
Jordan Hoffmann and others · 2022 · I3 Agent builder
Chinchilla: most large models were under-trained. For a fixed budget, train a smaller model on more data.
Download PDF: Training Compute-Optimal Large Language Models (opens in a new tab)Hosted by arXiv, posted by the authorsChain-of-Thought Prompting Elicits Reasoning in Large Language Models
Jason Wei and others · 2022 · I1 LLM builder
Asking a model to think step by step before answering makes it far better at maths and logic problems.
Download PDF: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (opens in a new tab)Hosted by arXiv, posted by the authorsSelf-Consistency Improves Chain of Thought Reasoning in Language Models
Xuezhi Wang and others · 2022 · I2 RAG builder
Sample several chains of thought and take the most common answer. A simple trick that lifts accuracy.
Download PDF: Self-Consistency Improves Chain of Thought Reasoning in Language Models (opens in a new tab)Hosted by arXiv, posted by the authorsLLaMA: Open and Efficient Foundation Language Models
Hugo Touvron and others · 2023 · I2 RAG builder
Meta's LLaMA showed that smaller open models trained on public data can match much larger closed ones.
Download PDF: LLaMA: Open and Efficient Foundation Language Models (opens in a new tab)Hosted by arXiv, posted by the authorsLlama 2: Open Foundation and Fine-Tuned Chat Models
Hugo Touvron and others · 2023 · I3 Agent builder
How Llama 2 and its chat version were trained and made safer, in unusual detail for an open model.
Download PDF: Llama 2: Open Foundation and Fine-Tuned Chat Models (opens in a new tab)Hosted by arXiv, posted by the authorsMistral 7B
Albert Q. Jiang and others · 2023 · I2 RAG builder
A 7-billion-parameter model that beats larger ones, using grouped-query and sliding-window attention.
Download PDF: Mistral 7B (opens in a new tab)Hosted by arXiv, posted by the authorsGPT-4 Technical Report
OpenAI and others · 2023 · I2 RAG builder
OpenAI's technical report on GPT-4: its abilities, exam results, safety testing and limits.
Download PDF: GPT-4 Technical Report (opens in a new tab)Hosted by arXiv, posted by the authorsMamba: Linear-Time Sequence Modeling with Selective State Spaces
Albert Gu and Tri Dao · 2023 · P1 Quality engineer
Mamba: a sequence model that scales linearly with length instead of quadratically like attention. A possible Transformer rival.
Download PDF: Mamba: Linear-Time Sequence Modeling with Selective State Spaces (opens in a new tab)Hosted by arXiv, posted by the authorsQLoRA: Efficient Finetuning of Quantized LLMs
Tim Dettmers and others · 2023 · I2 RAG builder
QLoRA: fine-tune a large model on a single GPU by combining 4-bit quantisation with LoRA.
Download PDF: QLoRA: Efficient Finetuning of Quantized LLMs (opens in a new tab)Hosted by arXiv, posted by the authorsDirect Preference Optimization: Your Language Model is Secretly a Reward Model
Rafael Rafailov and others · 2023 · I3 Agent builder
Direct Preference Optimization aligns a model with human preferences without a separate reward model. Simpler than RLHF.
Download PDF: Direct Preference Optimization: Your Language Model is Secretly a Reward Model (opens in a new tab)Hosted by arXiv, posted by the authorsMixtral of Experts
Albert Q. Jiang and others · 2024 · I3 Agent builder
Mixtral: a mixture-of-experts model that uses only part of its weights for each word, so it is fast for its size.
Download PDF: Mixtral of Experts (opens in a new tab)Hosted by arXiv, posted by the authorsDeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
DeepSeek-AI and others · 2025 · I3 Agent builder
DeepSeek-R1: reasoning skills learned mostly through reinforcement learning, released openly. A key 2025 paper.
Download PDF: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (opens in a new tab)Hosted by arXiv, posted by the authors
Papers: Retrieval and RAG
Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs
Yu. A. Malkov and D. A. Yashunin · 2016 · I3 Agent builder
HNSW, the graph algorithm most vector databases use to find nearest neighbours quickly.
Download PDF: Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs (opens in a new tab)Hosted by arXiv, posted by the authorsSentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Nils Reimers and Iryna Gurevych · 2019 · I2 RAG builder
Sentence-BERT made it practical to turn whole sentences into embeddings for semantic search.
Download PDF: Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (opens in a new tab)Hosted by arXiv, posted by the authorsRetrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Patrick Lewis and others · 2020 · I2 RAG builder
The original Retrieval-Augmented Generation paper: fetch relevant documents first, then generate the answer from them.
Download PDF: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (opens in a new tab)Hosted by arXiv, posted by the authorsDense Passage Retrieval for Open-Domain Question Answering
Vladimir Karpukhin and others · 2020 · I2 RAG builder
Dense Passage Retrieval: search by embedding similarity instead of keywords. A foundation of modern RAG.
Download PDF: Dense Passage Retrieval for Open-Domain Question Answering (opens in a new tab)Hosted by arXiv, posted by the authorsColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
Omar Khattab and Matei Zaharia · 2020 · I3 Agent builder
ColBERT compares queries and documents word by word for more precise search that is still fast.
Download PDF: ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT (opens in a new tab)Hosted by arXiv, posted by the authorsLost in the Middle: How Language Models Use Long Contexts
Nelson F. Liu and others · 2023 · I2 RAG builder
Models pay less attention to information in the middle of a long prompt. Changes how you order RAG context.
Download PDF: Lost in the Middle: How Language Models Use Long Contexts (opens in a new tab)Hosted by arXiv, posted by the authors
Papers: Agents
ReAct: Synergizing Reasoning and Acting in Language Models
Shunyu Yao and others · 2022 · I2 RAG builder
ReAct: models that alternate between reasoning and taking actions such as searches. The pattern behind most agents.
Download PDF: ReAct: Synergizing Reasoning and Acting in Language Models (opens in a new tab)Hosted by arXiv, posted by the authorsTree of Thoughts: Deliberate Problem Solving with Large Language Models
Shunyu Yao and others · 2023 · I3 Agent builder
Tree of Thoughts lets a model explore and compare several lines of reasoning, like a search.
Download PDF: Tree of Thoughts: Deliberate Problem Solving with Large Language Models (opens in a new tab)Hosted by arXiv, posted by the authorsToolformer: Language Models Can Teach Themselves to Use Tools
Timo Schick and others · 2023 · I3 Agent builder
Toolformer: a model teaches itself when to call tools such as a calculator or search engine.
Download PDF: Toolformer: Language Models Can Teach Themselves to Use Tools (opens in a new tab)Hosted by arXiv, posted by the authorsReflexion: Language Agents with Verbal Reinforcement Learning
Noah Shinn and others · 2023 · I3 Agent builder
Reflexion: an agent writes notes on its own mistakes and uses them to do better next time.
Download PDF: Reflexion: Language Agents with Verbal Reinforcement Learning (opens in a new tab)Hosted by arXiv, posted by the authors
Papers: Deployment and ops
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Tri Dao and others · 2022 · P1 Quality engineer
FlashAttention makes attention much faster and lighter on memory by working with the GPU's memory layout.
Download PDF: FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (opens in a new tab)Hosted by arXiv, posted by the authors
Papers: Responsible AI
Constitutional AI: Harmlessness from AI Feedback
Yuntao Bai and others · 2022 · I3 Agent builder
Anthropic's Constitutional AI: a model critiques and improves its own answers against written principles.
Download PDF: Constitutional AI: Harmlessness from AI Feedback (opens in a new tab)Hosted by arXiv, posted by the authors
Papers: Machine learning
Efficient Estimation of Word Representations in Vector Space
Tomas Mikolov and others · 2013 · I1 LLM builder
word2vec: learning word vectors where similar words sit close together. The idea behind embeddings.
Download PDF: Efficient Estimation of Word Representations in Vector Space (opens in a new tab)Hosted by arXiv, posted by the authorsAuto-Encoding Variational Bayes
Diederik P Kingma and Max Welling · 2013 · I3 Agent builder
Variational autoencoders: learning a smooth hidden space you can sample new data from.
Download PDF: Auto-Encoding Variational Bayes (opens in a new tab)Hosted by arXiv, posted by the authorsPlaying Atari with Deep Reinforcement Learning
Volodymyr Mnih and others · 2013 · I3 Agent builder
DeepMind's DQN learned to play Atari games from pixels. The start of deep reinforcement learning.
Download PDF: Playing Atari with Deep Reinforcement Learning (opens in a new tab)Hosted by arXiv, posted by the authorsSequence to Sequence Learning with Neural Networks
Ilya Sutskever and others · 2014 · I2 RAG builder
Sequence-to-sequence learning: an encoder reads a sentence and a decoder writes the translation.
Download PDF: Sequence to Sequence Learning with Neural Networks (opens in a new tab)Hosted by arXiv, posted by the authorsNeural Machine Translation by Jointly Learning to Align and Translate
Dzmitry Bahdanau and others · 2014 · I2 RAG builder
The first attention mechanism: let the decoder look back at the most relevant input words.
Download PDF: Neural Machine Translation by Jointly Learning to Align and Translate (opens in a new tab)Hosted by arXiv, posted by the authorsAdam: A Method for Stochastic Optimization
Diederik P. Kingma and Jimmy Ba · 2014 · I2 RAG builder
Adam, the default optimiser for training neural networks, and why it adapts the step size per weight.
Download PDF: Adam: A Method for Stochastic Optimization (opens in a new tab)Hosted by arXiv, posted by the authorsGenerative Adversarial Networks
Ian J. Goodfellow and others · 2014 · I2 RAG builder
Generative Adversarial Networks: a generator and a judge trained against each other to create realistic images.
Download PDF: Generative Adversarial Networks (opens in a new tab)Hosted by arXiv, posted by the authorsDeep Residual Learning for Image Recognition
Kaiming He and others · 2015 · I2 RAG builder
ResNet's skip connections made very deep networks trainable. Still inside almost every model.
Download PDF: Deep Residual Learning for Image Recognition (opens in a new tab)Hosted by arXiv, posted by the authorsBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Sergey Ioffe and Christian Szegedy · 2015 · I2 RAG builder
Batch normalisation keeps layer inputs stable, so deep networks train faster and more reliably.
Download PDF: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift (opens in a new tab)Hosted by arXiv, posted by the authorsU-Net: Convolutional Networks for Biomedical Image Segmentation
Olaf Ronneberger and others · 2015 · I2 RAG builder
U-Net, the encoder-decoder network for image segmentation, later reused inside diffusion models.
Download PDF: U-Net: Convolutional Networks for Biomedical Image Segmentation (opens in a new tab)Hosted by arXiv, posted by the authorsYou Only Look Once: Unified, Real-Time Object Detection
Joseph Redmon and others · 2015 · I2 RAG builder
YOLO: real-time object detection in a single pass over the image.
Download PDF: You Only Look Once: Unified, Real-Time Object Detection (opens in a new tab)Hosted by arXiv, posted by the authorsLayer Normalization
Jimmy Lei Ba and others · 2016 · I3 Agent builder
Layer normalisation, the variant Transformers use because it works one example at a time.
Download PDF: Layer Normalization (opens in a new tab)Hosted by arXiv, posted by the authorsProximal Policy Optimization Algorithms
John Schulman and others · 2017 · I3 Agent builder
Proximal Policy Optimization, the reinforcement learning method used in RLHF.
Download PDF: Proximal Policy Optimization Algorithms (opens in a new tab)Hosted by arXiv, posted by the authorsDenoising Diffusion Probabilistic Models
Jonathan Ho and others · 2020 · I3 Agent builder
Denoising diffusion: generate images by learning to remove noise step by step. The basis of image generators.
Download PDF: Denoising Diffusion Probabilistic Models (opens in a new tab)Hosted by arXiv, posted by the authorsAn Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Alexey Dosovitskiy and others · 2020 · I2 RAG builder
The Vision Transformer cuts an image into patches and treats them like words.
Download PDF: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (opens in a new tab)Hosted by arXiv, posted by the authorsHigh-Resolution Image Synthesis with Latent Diffusion Models
Robin Rombach and others · 2021 · I3 Agent builder
Latent diffusion runs diffusion in a compressed space, which made Stable Diffusion possible on ordinary GPUs.
Download PDF: High-Resolution Image Synthesis with Latent Diffusion Models (opens in a new tab)Hosted by arXiv, posted by the authorsLearning Transferable Visual Models From Natural Language Supervision
Alec Radford and others · 2021 · I2 RAG builder
CLIP learns images and text in one shared space, so you can search images with words.
Download PDF: Learning Transferable Visual Models From Natural Language Supervision (opens in a new tab)Hosted by arXiv, posted by the authors
Free books
An Introduction to Statistical Learning
Gareth James and others · 2023 · I1 LLM builder
The friendliest book on regression, classification and trees, with Python labs. Free from the authors.
Read free online: An Introduction to Statistical Learning (opens in a new tab)Hosted by The authors, on statlearning.comDeep Learning
Ian Goodfellow, Yoshua Bengio and Aaron Courville · 2016 · P1 Quality engineer
The foundational deep learning textbook, readable free online chapter by chapter.
Read free online: Deep Learning (opens in a new tab)Hosted by The authors, on deeplearningbook.orgDive into Deep Learning
Aston Zhang and others · 2023 · I1 LLM builder
A hands-on deep learning textbook with runnable code in PyTorch on every page. Used in hundreds of universities.
Download PDF: Dive into Deep Learning (opens in a new tab)Hosted by The authors, on d2l.aiFoundations of Large Language Models
Tong Xiao and Jingbo Zhu · 2025 · I2 RAG builder
A free book on how LLMs are pre-trained, aligned, prompted and scaled.
Download PDF: Foundations of Large Language Models (opens in a new tab)Hosted by arXiv, posted by the authorsInterpretable Machine Learning
Christoph Molnar · 2022 · I2 RAG builder
How to explain what a model is doing: feature importance, SHAP, LIME and more. Free online.
Read free online: Interpretable Machine Learning (opens in a new tab)Hosted by The author, on GitHub PagesMathematics for Machine Learning
Marc Peter Deisenroth and others · 2020 · I1 LLM builder
The linear algebra, calculus and probability you need for ML, explained for people who are rusty on maths.
Download PDF: Mathematics for Machine Learning (opens in a new tab)Hosted by The authors, on mml-book.github.ioPattern Recognition and Machine Learning
Christopher M. Bishop · 2006 · P1 Quality engineer
The classic graduate textbook on the probability behind machine learning, shared free by Microsoft Research.
Download PDF: Pattern Recognition and Machine Learning (opens in a new tab)Hosted by Microsoft ResearchProbabilistic Machine Learning: An Introduction
Kevin P. Murphy · 2022 · I3 Agent builder
A broad, modern reference on machine learning from a probability point of view.
Read free online: Probabilistic Machine Learning: An Introduction (opens in a new tab)Hosted by The author, on probml.github.ioReinforcement Learning: An Introduction
Richard S. Sutton and Andrew G. Barto · 2018 · I3 Agent builder
The classic reinforcement learning textbook, by two of the field's founders.
Download PDF: Reinforcement Learning: An Introduction (opens in a new tab)Hosted by The authors, on incompleteideas.netSpeech and Language Processing (3rd edition draft)
Dan Jurafsky and James H. Martin · 2025 · I2 RAG builder
The standard NLP textbook, updated for Transformers and LLMs. Its chapters on embeddings and attention are excellent.
Download PDF: Speech and Language Processing (3rd edition draft) (opens in a new tab)Hosted by The authors, at StanfordThe Little Book of Deep Learning
François Fleuret · 2023 · I1 LLM builder
A short, phone-sized summary of deep learning, from training basics to Transformers. A good revision guide.
Download PDF: The Little Book of Deep Learning (opens in a new tab)Hosted by The author, on fleuret.orgUnderstanding Deep Learning
Simon J. D. Prince · 2023 · I2 RAG builder
A clear, beautifully illustrated modern textbook, from basics to Transformers and diffusion models. The full PDF is on the book's site.
Read free online: Understanding Deep Learning (opens in a new tab)Hosted by The author, on udlbook.github.io