EmbeddingsDivision documentation
==============================


.. toctree::
   :maxdepth: 2
   :caption: Contents:

   modules

About
==================

Python script for manipulating the existing tokenizer.

The solution was tested on meta-llama/Llama-3.2-1B.

Installation
==================

.. code-block:: bash

    pip install embdiv


Requirements
==================
- Python 3.9+
- cachetools>=5.5.0,
- torch>=2.5.1,
- transformers>=4.45.2,

Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`