Deriving the quantity of items is an abstract form of categorization. To explore it, monkeys were trained to judge whether successive visual displays contained the same quantity of items. Many neurons in the lateral prefrontal cortex were tuned for quantity irrespective of the exact physical appearance of the displays. Their tuning curves formed overlapping filters, which may explain why behavioral discrimination improves with increasing numerical distance and why discrimination of two quantities with equal numerical distance worsens as their numerical size increases. A mechanism that extracts the quantity of visual field items could contribute to general numerical ability.