AWS Certified AI Practitioner AIF-C01 Practice Question
A developer must send a 20-page FAQ document to a large language model that accepts a maximum of 4 000 tokens per request. Which action best illustrates the use of chunking before the document is stored for retrieval-augmented generation (RAG)?
Divide the document into multiple overlapping sections of roughly 500 tokens each and store each section separately.
Remove sentences that repeat information so the remaining text fits within the model's context window.
Convert every word in the document to its corresponding vocabulary ID before sending it to the model.
Generate one high-dimensional embedding vector that represents the entire document and store only that vector.
Chunking refers to breaking a large document into smaller, semantically coherent pieces that fit within the model's context window, often with a small overlap to preserve continuity. Splitting the FAQ into several overlapping blocks of about 500 tokens each is an example of chunking. Mapping words to vocabulary IDs describes tokenization, not chunking. Turning the whole document into a single embedding vector skips the step of dividing the text and therefore is not chunking. Simply deleting repetitive sentences shortens the text but does not systematically divide it into manageable segments, so it is also not chunking.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What are tokens in NLP, and why is there a maximum context window?
Open an interactive chat with Bash
Why are overlapping sections used in chunking for RAG?
Open an interactive chat with Bash
What is the difference between tokenization and chunking?
Open an interactive chat with Bash
Why is overlapping used when chunking a document?
Open an interactive chat with Bash
What is the difference between chunking and tokenization?
Open an interactive chat with Bash
When is chunking necessary in Retrieval-Augmented Generation (RAG)?
Open an interactive chat with Bash
AWS Certified AI Practitioner AIF-C01
Fundamentals of Generative AI
Your Score:
Report Issue
Bash, the Crucial Exams Chat Bot
AI Bot
Loading...
Loading...
Loading...
Pass with Confidence.
IT & Cybersecurity Package
You have hit the limits of our free tier, become a Premium Member today for unlimited access.
Military, Healthcare worker, Gov. employee or Teacher? See if you qualify for a Community Discount.
Monthly
$19.99
$19.99/mo
Billed monthly, Cancel any time.
3 Month Pass
$44.99
$14.99/mo
One time purchase of $44.99, Does not auto-renew.
MOST POPULAR
Annual Pass
$119.99
$9.99/mo
One time purchase of $119.99, Does not auto-renew.
BEST DEAL
Lifetime Pass
$189.99
One time purchase, Good for life.
What You Get
All IT & Cybersecurity Package plans include the following perks and exams .