one fable-low hijack cell
pair attention / token
envelope zh_huo_q
ordering BA
effort low
Chinese '或' with explicit question marker — '{X}或{Y}？'
verdictpicked token (second-offered)
· classifierB
· tiernone
response (200 chars)
# Attention 与 Token 的关系
这两个是 Transformer 模型中不同层面的概念,我来分别解释:
## Token(词元)
**是什么**:文本被切分后的基本单位,是模型处理的"原料"。
```
"我喜欢机器学习" → ["我", "喜欢", "机器", "学习"]
"unbelievable" → ["un", "believ", "able"]
```
- 模型
neighbors