What Is Chunking, and Why Does It Decide Which Paragraph AI Quotes?

When an AI assistant quotes a website, it rarely quotes the page. It quotes a paragraph, sometimes a sentence, lifted from somewhere in the middle.

The step that decides which paragraph is even available to be quoted has a name: chunking. It happens before any question is asked, and most business owners have never heard of it.

Here is what chunking in AI search is, why pages get cut into pieces, what the same page looks like cut two different ways, four common methods side by side, and what it changes about how you write.

What is chunking in AI search?

Chunking is cutting a page into short pieces so each piece can be searched and matched on its own. When a question arrives, the system looks for the pieces that fit it best, and those pieces are what the answer is built from.

The consequence is simple and easy to miss. Your page is not judged as a page. Each piece of it is judged alone, without the heading above it or the paragraph before it, unless the system has done something extra to carry that context along.

That is why one page can be quoted for one question and ignored for another. The piece that answered the first question was complete. The piece that should have answered the second was missing half of what it needed.

Why assistants cut pages into pieces

Because matching a question to a whole page is too blunt. A services page covers who you help, what it costs and when to start; a question usually asks about one of those, and the system wants the part that answers it.

Anthropic's engineering write-up on contextual retrieval describes the standard approach plainly: documents are split into smaller chunks, usually no more than a few hundred tokens, for efficient retrieval. It also names the cost. Its own example is a chunk reading "The company's revenue grew by 3% over the previous quarter", which on its own does not say which company or which quarter.

The fix Anthropic tested was to add a short summary of the surrounding document to each chunk before storing it, which cut failed retrievals by 49 percent in their tests, and by 67 percent with an extra re-ranking step. That tells you the problem is real and large. It does not tell you whether the assistant your buyer uses does the same thing, and that is the honest limit of what anyone outside these companies can say.

The same page, cut two ways

Where the cut falls decides whether each piece makes sense alone. The words below are identical in both columns; only the boundaries move.

ILLUSTRATIVE: ONE SERVICES PAGE, EIGHT LINESTWO CUTS, ONE PAGE

Cut by length

A new piece roughly every 40 words, wherever that lands

1. • Career coaching for mid-career moves
   • I work with managers in Denver changing industries after ten or more years in one field.
   • Most clients come to me after a restructuring or a promotion that went to someone else.
   • What it costs

   Ends on a heading with no answer under it

2. • Twelve weekly sessions are $1,800, paid monthly.
   • That includes a resume rewrite and two mock interviews.
   • When you can start

   A price with no name or service attached

3. • New clients start on the first Monday of each month.

   A start date with nothing saying for what

0 of 3 pieces make sense alone

Cut at the headings

A new piece wherever the page starts a new section

1. • Career coaching for mid-career moves
   • I work with managers in Denver changing industries after ten or more years in one field.
   • Most clients come to me after a restructuring or a promotion that went to someone else.

   Who the service is for, in one place

2. • What it costs
   • Twelve weekly sessions are $1,800, paid monthly.
   • That includes a resume rewrite and two mock interviews.

   The cost question and its answer together

3. • When you can start
   • New clients start on the first Monday of each month.

   The start question and its answer together

3 of 3 pieces keep a heading with its answer

Same eight lines, same order. Cut by length, a heading ends up in one piece and its answer in the next, and the price arrives without the name of the service it prices. Cut at the headings, every piece carries its own question and answer.

AnswerHalo answerhalo.com

CLICK TO ENLARGE

That page and its $1,800 price are illustrative. The pattern is the point: you do not control where the cut falls, but you do control whether a cut in an unlucky place still leaves something usable. A price sentence that says "Twelve weekly career coaching sessions with me are $1,800" survives any cut. "That includes the above" survives none.

Four common ways to cut a page

There are a handful of standard methods, and a page written in clear sections does well under all of them. The table sets them side by side with what each one does to a typical small-business page.

Read down the last column and it is the same advice four times: make each part of the page able to stand alone. That is also what makes a page easy for a person to skim, which is not a coincidence.

What chunking changes about how you write

Write in sections that each answer one question, and make every answer complete enough to be read alone. That covers most of it.

In practice, three edits do the most work. Turn headings into the questions buyers ask ("What does it cost?" rather than "Investment"). Put the answer in the first sentence under each heading. And replace pointing words such as "this", "the above" and "our program" with the actual name of the thing, at least once per section.

Google's guidance on AI features in Search says there are no special optimizations needed to appear in AI Overviews, and that important content should be available in textual form. Chunking is part of why that advice holds: text in clear sections is what every method cuts cleanly, and there is no hidden format that beats it.

The damaging admission: good chunking does not make a page worth quoting. If your page does not state a price, a location or who you serve, cutting it well only produces clean pieces with nothing in them. Chunking decides which of your facts are usable. It cannot supply the facts.

For the step that happens next, when pieces are pulled in to answer a question, see what retrieval-augmented generation is. For which kinds of pages supply the pieces that get quoted most, see the pages AI assistants quote most, and for the page to fix first, how to write a services page an AI can quote.

If you want to know whether ChatGPT names you today, start with the free check. It asks ChatGPT 25 real buyer questions from your niche at $0 and emails a plain-English readout: your score, who got named in your place, and the one fix to start with. It runs as soon as you ask for it, and the report lands within the hour between 7am and 5pm Central, or by 10am the next morning outside those hours.