An opinionated lexicon of LLM-related concepts. ## Model development - Pre-training - Causal vs. masked language modeling - [Quantization-aware training](https://huggingface.co/blog/optimize-llm) - [Flash Attention](https://huggingface.co/blog/optimize-llm) - [Mixture of Experts](https://huggingface.co/blog/moe) - Fine tuning - Supervised fine-tuning (SFT) - Instruction tuning - Alignment (with human preferences) - Reinforcement learning from human feedback (RLHF) - Reinforcement learning from AI feedback (RLAIF, Constitutional AI) - Hindsight instruction relabeling (HIR) - Direct preference optimization (DPO) - Reinforced self-training (ReST) ## Application design - Agency level during inference - Semantic search/retrieval (with embeddings or keyword generation) - Human-in-the-loop (offline) content generation - Fully automated (online) content generation - Retrieval-augmented Generation (RAG) - Chatbots - Agents & copilots ("AI", [ReAct Prompting](ReAct%20Prompting.md)) - Modality of inputs (audio, image, and/or text) ## Prompt engineering - Prompting techniques - Zero-shot prompting - [Chain-of-Thought](Chain-of-Thought%20Prompting.md) - Few-shot prompting - [Plan-and-Solve](Plan-and-Solve%20Prompting.md) - Linear prompt chaining (e.g., [LangChain](https://python.langchain.com/docs/get_started/introduction)) - Self-Consistency - Reasoning agents (e.g., [AutoGen](https://microsoft.github.io/autogen/)) - [ReAct](ReAct%20Prompting.md) - [Probabilitistic] Prompt testing - Prompt design - Manual (hard) prompt tuning - Automatic (hard) prompt *optimization* - [Automatic Prompt Engineer (APE)](Automatic%20Prompt%20Engineer%20(APE).md) - [OPRO](OPRO%20-%20Large%20Language%20Models%20as%20Optimizers.md) Prompt Optimizer - Automatic Prompt Optimization (APO) - ... - *Soft* prompt tuning (e.g., Prefix Tuning, [Prompt Tuning](Prompt%20Tuning.md), or [P-Tuning](P-Tuning.md)) - Constrained [token] sampling during content generation ("guidance") - [Function calling](https://platform.openai.com/docs/guides/function-calling) - Guardrails for content monitoring, validation & filtering (+ "re-prompting") ## Model engineering - Adapter learning techniques (incl. IA3, LoRA, LoKr, etc.) - Model [Distilling Step-by-Step](Distilling%20Step-by-Step.md) - Post-training model quantization ## Inference optimization - Quantization - In-flight batching - Speculative inference/decoding - [Mixture of Experts](https://huggingface.co/blog/moe)