one fable-low hijack cell
pair embedding / token
envelope zh_huo_q
ordering AB
effort low
Chinese '或' with explicit question marker — '{X}或{Y}?'
verdictpicked token (second-offered)
· classifierB
· tiernone
response (200 chars)
# Token 还是 Embedding?
这两个概念在 NLP/大模型中密切相关但处于不同层次:
## Token(词元)
**是什么**:文本被切分后的离散单元
```
"我爱机器学习" → ["我", "爱", "机器", "学习"]
"unbelievable" → ["un", "believ", "able"]
```
- **形式**:离散的符号,通常映射为整数 ID(如
neighbors