diff options
Diffstat (limited to 'README.md')
-rw-r--r-- | README.md | 34 |
1 files changed, 34 insertions, 0 deletions
@@ -25,3 +25,37 @@ Works with handwritten formulae as well! - [ ] Image Export - [ ] UI Overhaul - [ ] Optimizations + +## Misc + +### Quantization + +#### Encoder Model + +```bash +python -m onnxruntime.quantization.preprocess --input iTexSnip/models/encoder_model.onnx --output encoder-infer.onnx +``` + +```python +import onnx +from onnxruntime.quantization import quantize_dynamic, QuantType +og = "encoder-infer.onnx" +quant = "encoder-quant.onnx" +quantized_model = quantize_dynamic(og, quant, nodes_to_exclude=['/embeddings/patch_embeddings/projection/Conv']) +``` + +It might be better if we quantize the encoder using static quantization. + +#### Decoder Model + +```bash +python -m onnxruntime.quantization.preprocess --input iTexSnip/models/decoder_model.onnx --output decoder-infer.onnx +``` + +```python +import onnx +from onnxruntime.quantization import quantize_dynamic, QuantType +og = "decoder-infer.onnx" +quant = "decoder-quant.onnx" +quantized_model = quantize_dynamic(og, quant) +``` |