Acceptance Length

Acceptance length is the average number of tokens the main model accepts from the drafter in one step of speculative decoding.

It depends heavily on the task: predictable content like math can accept ~339 tokens in the test’s metric, while creative writing only ~150. The higher the acceptance, the bigger the speed-up.