BPE tokenizer

Trains byte-pair encoding on your text and encodes a string with it. Shows merge rules and verifies the round trip.

Fill in the fields, run the tool and review the result. Your input is not added to a public page. Use the learning mode for calculation details.