Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,170 @@
+---
+language: en
+license: apache-2.0
+model_name: t5-encoder-12.onnx
+tags:
+- validated
+- text
+- machine_comprehension
+- t5
+---
+<!--- SPDX-License-Identifier: Apache-2.0 -->
+
+# T5
+
+## Use-cases
+Transformer-based language model trained on multiple tasks including summarization, sentiment-analysis, q&a, translation etc.
+The implementation in this repo is an adaptation of the [onnxt5 repo](https://github.com/abelriboulot/onnxt5) which makes the export and use of T5 with ONNX easier.
+
+## Description
+[T5](https://arxiv.org/abs/1910.10683) is a transformer model which aims to provide great flexibility and provide better semantic
+understanding through the training of multiple tasks at once.
+
+## Model
+
+| Model | Download  | Download (with sample test data) | ONNX version | Opset version |
+| ----------- | ---------- |--------------| -------------- | -------------- |
+|T5-encoder       |[650.6 MB](model/t5-encoder-12.onnx) | [205.0 MB](model/t5-encoder-12.tar.gz)| 1.7 | 12
+|T5-decoder-with-lm-head |[304.9 MB](model/t5-decoder-with-lm-head-12.onnx) | [304.9 MB](model/t5-decoder-with-lm-head-12.tar.gz)| 1.7 | 12
+
+
+### Source
+Huggingface PyTorch T5 + script changes ==> ONNX T5-encoder
+
+Huggingface PyTorch T5 + script changes ==> ONNX T5-decoder-with-lm-head
+
+Script changes include:
+- reshaping the Huggingface models to combine the lm head with the decoder to allow for a unified model
+- reshaping the encoder to output the hidden state directly
+
+## Inference
+The script for ONNX model conversion and ONNX Runtime inference is [here](dependencies/T5-export.py).
+More complete utilities to export and use the models are maintained in the [onnxt5 repo](https://github.com/abelriboulot/onnxt5).
+
+### Input to model
+This implementation takes as inputs a prompt which begins by the task at hand here. Examples of some tasks include ```summarize: <PROMPT>```,
+```translate English to French: <PROMPT>```, ```cola sentence: <PROMPT>```, etc.
+For the full list of task you can refer to the appendix D of the [original paper](https://arxiv.org/pdf/1910.10683.pdf).
+
+
+### Preprocessing steps
+The easiest way to use the model is to use the onnxt5 utilities (installation instructions: ```pip install onnxt5```).
+
+In that case you can use the model with the following piece of code:
+```python
+from onnxt5 import GenerativeT5
+from onnxt5.api import get_encoder_decoder_tokenizer
+decoder_sess, encoder_sess, tokenizer = get_encoder_decoder_tokenizer()
+generative_t5 = GenerativeT5(encoder_sess, decoder_sess, tokenizer, onnx=True)
+prompt = 'translate English to French: I was a victim of a series of accidents.'
+output_text, output_logits = generative_t5(prompt, max_length=100, temperature=0.)
+# output_text: "J'ai été victime d'une série d'accidents."
+```
+
+Or if you wish to produce the embeddings of a sentence:
+```python
+from onnxt5.api import get_encoder_decoder_tokenizer, run_embeddings_text
+
+decoder_sess, encoder_sess, tokenizer = get_encoder_decoder_tokenizer()
+prompt = 'Listen, Billy Pilgrim has come unstuck in time.'
+encoder_embeddings, decoder_embeddings = run_embeddings_text(encoder_sess, decoder_sess, tokenizer, prompt)
+```
+
+Otherwise you can manually create the Generative model with the following:
+
+```python
+from onnxruntime import InferenceSession
+from transformers import T5Tokenizer
+from .dependencies.models import GenerativeT5
+
+tokenizer = T5Tokenizer.from_pretrained('t5-base')
+
+# Start from ORT 1.10, ORT requires explicitly setting the providers parameter if you want to use execution providers
+# other than the default CPU provider (as opposed to the previous behavior of providers getting set/registered by default
+# based on the build flags) when instantiating InferenceSession.
+# For example, if NVIDIA GPU is available and ORT Python package is built with CUDA, then call API as following:
+# InferenceSession(path/to/model, providers=['CUDAExecutionProvider'])
+decoder_sess = InferenceSession(str(path_t5_decoder))
+encoder_sess = InferenceSession(str(path_t5_encoder))
+generative_t5 = GenerativeT5(encoder_sess, decoder_sess, tokenizer, onnx=True)
+generative_t5('translate English to French: I was a victim of a series of accidents.', 21, temperature=0.)[0]
+```
+
+### Output of model
+For the T5-encoder model:
+
+**last_hidden_state**: Sequence of hidden-states at the last layer of the model. It's a float tensor of size (batch_size, sequence_length, hidden_size).
+
+For T5-decoder-with-lm-head model:
+
+**logit_predictions**: Prediction scores of the language modeling head. It's a float tensor of size (batch_size, sequence_length, vocab_size).
+
+### Postprocessing steps
+For the T5-encoder model:
+
+```python
+last_hidden_states = model(input_ids)[0]
+```
+
+For the T5-decoder-with-lm-head model:
+
+```python
+# To generate the encoder's last hidden state
+encoder_output = encoder_sess.run(None, {"input_ids": input_ids})[0]
+# To generate the full model's embeddings
+decoder_output = decoder_sess.run(None, {
+"input_ids": input_ids,
+"encoder_hidden_states": encoder_output
+})[0]
+```
+
+For the generative model, to generate a translation:
+```
+from onnxt5 import GenerativeT5
+from onnxt5.api import get_encoder_decoder_tokenizer
+decoder_sess, encoder_sess, tokenizer = get_encoder_decoder_tokenizer()
+generative_t5 = GenerativeT5(encoder_sess, decoder_sess, tokenizer, onnx=True)
+prompt = 'translate English to French: I was a victim of a series of accidents.'
+output_text, output_logits = generative_t5(prompt, max_length=100, temperature=0.)
+```
+<hr>
+
+## Dataset (Train and validation)
+The original model from Google Brain is pretrained on the [Colossal Clean Crawled Corpus](https://www.tensorflow.org/datasets/catalog/c4).
+The pretrained model is referenced in [huggingface/transformers](https://github.com/huggingface/transformers/blob/master/transformers/modeling_t5.py), trained on the same data.
+<hr>
+
+## Validation accuracy
+Benchmarking can be run with the following [script](https://github.com/abelriboulot/onnxt5/blob/master/notebooks/benchmark_performance.ipynb) with initial results in this [post](https://kta.io/posts/onnx_t5).
+<hr>
+
+
+## Publication/Attribution
+This repo is based on the work of Colin Raffel and Noam Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and
+Michael Matena and Yanqi Zhou and Wei Li and Peter J. Liu from Google, as well as the implementation of T5 from the
+huggingface team, the work of the Microsoft ONNX and onnxruntime teams, in particular Tianlei Wu, and the work of Thomas Wolf on generation of text.
+
+[Original T5 Paper](https://arxiv.org/pdf/1910.10683.pdf)
+```
+@article{2019t5,
+author = {Colin Raffel and Noam Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi Zhou and Wei Li and Peter J. Liu},
+title = {Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer},
+journal = {arXiv e-prints},
+year = {2019},
+archivePrefix = {arXiv},
+eprint = {1910.10683},
+}
+```
+
+## References
+This model is converted directly from [huggingface/transformers](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_t5.py).
+<hr>
+
+## Contributors
+[Abel Riboulot](https://github.com/abelriboulot)
+<hr>
+
+## License
+Apache 2.0 License
+<hr>
+