Fundamentals of Generative AI
What you will learn
In this domain, you learn the concept of generative AI foundation models, the characteristics of Transformers and LLMs, the basics of prompt engineering, and core technologies such as Retrieval-Augmented Generation (RAG) and fine-tuning, along with the essential concepts for working with LLMs: tokens, context windows, inference parameters, and embeddings. It is a frequently tested domain that accounts for 24% of the AIF-C01 exam and is the central theme of modern AI use.
Key points
- Foundation Model - a general-purpose model pre-trained on large-scale data that can be repurposed for a variety of tasks
- Large Language Model (LLM) - a foundation model that handles text. Representative examples include models from OpenAI, Anthropic, and Meta, and Amazon's Nova
- Transformer - an architecture that uses the self-attention mechanism and is the standard structure of today's LLMs
- Prompt engineering - the technique of designing instructions (prompts) that elicit the desired output from an LLM
- Zero-shot / few-shot - techniques that have an LLM perform a task with no examples / a few examples
- RAG (Retrieval-Augmented Generation) - retrieves relevant information from an external knowledge base and adds it to the LLM's prompt to improve the accuracy of answers
- The difference between fine-tuning and pre-training - pre-training learns from scratch on large-scale data, while fine-tuning additionally trains an existing model for a specific use
- Hallucination - the phenomenon in which an LLM plausibly generates content that is not factual. It is suppressed with RAG and verification
- Tokens and context windows - a token is the smallest unit of text an LLM handles, and the context window is the upper limit on how many tokens can be processed at once
- Inference parameters - lower temperature and top-p values produce more deterministic, consistent output, while higher values produce more diverse, creative output
- Embeddings - numeric vector representations of the meaning of text. Combined with a vector database, they enable semantic similarity search
- Evaluating generated output - combine task-appropriate automatic metrics (ROUGE for summarization, BLEU for translation, and so on) with human evaluation
Terms and concepts
Foundation Model
A large-scale model pre-trained on a large amount of general-purpose data that can be applied to a variety of tasks through fine-tuning or prompts. Its uses are expanding to LLMs for text, image generation, audio, and multimodal applications. With Bedrock, you can use multiple foundation models via an API.
Prompt engineering
The technique of crafting the instructions that elicit the desired output from an LLM. Techniques include specifying a role ("you are a doctor"), specifying the output format ("answer in JSON"), adding examples (few-shot), and prompting step-by-step thinking (Chain-of-Thought).
RAG (Retrieval-Augmented Generation)
A technique that searches an external knowledge base (internal documents, databases, etc.) for information relevant to a question, adds that information to the LLM's prompt, and then generates an answer. It supplements information the LLM lacks or that is outdated, and it can suppress hallucinations. It is implemented in combination with Bedrock Knowledge Bases or Kendra.
Fine-tuning and pre-training
Pre-training is the process of training a foundation model from scratch on large-scale data and requires enormous computational resources. Fine-tuning is the process of additionally training a pre-trained model on a small amount of data from a specific domain (law, medicine, etc.), increasing expertise while keeping costs low.
Hallucination
The phenomenon in which an LLM plausibly generates information not in the training data or content that differs from the facts. The cause lies in the model's nature of generating "the characters most likely to come next." Its impact is mitigated by providing evidence with RAG, validating output with guardrails, and human review.
Tokens and context windows
A token is the smallest unit in which an LLM processes text. Text is often split more finely than into words, and a symbol or a fragment of a word can each count as one token. The context window is the upper limit on the number of tokens (input plus output) the model can handle in a single call. To work with long documents that exceed the limit, the standard approach is to split them into chunks and use RAG to retrieve only the relevant parts into the context. Pricing is also calculated per token, making tokens an important unit for cost management.
→ Designing a Generative AI Platform / → Handling Long Documents with RAG
Inference parameters (temperature / top-p)
Temperature is a parameter that adjusts the randomness of the output: low values (for example, 0.1) yield more deterministic, consistent output, while higher values yield more diverse, creative output. Top-p (nucleus sampling) is a technique that samples only from the highest-probability token candidates whose cumulative probability reaches a threshold. As a rule of thumb, set these low for fact-based Q&A and higher for brainstorming or creative writing.
Embeddings and vector databases
An embedding is the meaning of a piece of text converted into a numeric vector of hundreds to thousands of dimensions. Sentences with similar meanings sit close together in the vector space, enabling similarity search "by meaning." The retrieval side of RAG works exactly this way: vectorize a large document collection, store it in a vector database, and quickly pull out the entries closest to the vector of the question. On AWS, OpenSearch's vector search is a common home for this.
→ Vector Search with OpenSearch / → RAG with Bedrock Knowledge Bases
Evaluating generative AI models (ROUGE / BLEU)
The quality of generated output is evaluated with metrics suited to the task. ROUGE is used to evaluate summarization and measures the word overlap between a generated summary and a reference summary. BLEU is used to evaluate machine translation and measures agreement with reference translations. Because automatic metrics alone cannot fully capture fluency or factual accuracy, combining them with human evaluation is standard practice.
Check your understanding
Check what you have learned with 5 questions