Mengyao00 commited on
Commit
f3cb564
·
verified ·
1 Parent(s): f51c1a8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -17,7 +17,7 @@ library_name: transformers
17
 
18
  ### **Description**
19
 
20
- The Llama Nemotron Retriever Embedding 1B model is optimized for **multilingual and cross-lingual** text question-answering retrieval with **support for long documents (up to 8192 tokens) and dynamic embedding size (Matryoshka Embeddings)**. This model was evaluated on 26 languages: English, Arabic, Bengali, Chinese, Czech, Danish, Dutch, Finnish, French, German, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Norwegian, Persian, Polish, Portuguese, Russian, Spanish, Swedish, Thai, and Turkish.
21
 
22
  In addition to enabling multilingual and cross-lingual question-answering retrieval, this model reduces the data storage footprint by 35x through dynamic embedding sizing and support for longer token length, making it feasible to handle large-scale datasets efficiently.
23
 
@@ -25,14 +25,14 @@ An embedding model is a crucial component of a text retrieval system, as it tran
25
 
26
  This model is ready for commercial use.
27
 
28
- The Llama Nemotron Retriever Embedding 1B model is a part of the NVIDIA NeMo Retriever collection of NIM, which provide state-of-the-art, commercially-ready models and microservices, optimized for the lowest latency and highest throughput. It features a production-ready information retrieval pipeline with enterprise support. The models that form the core of this solution have been trained using responsibly selected, auditable data sources. With multiple pre-trained models available as starting points, developers can also readily customize them for domain-specific use cases, such as information technology, human resource help assistants, and research & development research assistants.
29
 
30
  We are excited to announce the open sourcing of this commercial embedding model. For users interested in deploying this model in production environments, it is also available via the model API in NVIDIA Inference Microservices (NIM) at [llama-nemotron-embed-1b-v2](https://build.nvidia.com/nvidia/llama-3_2-nv-embedqa-1b-v2).
31
 
32
 
33
  ### **Intended use**
34
 
35
- The Llama Nemotron Retriever Embedding 1B model is most suitable for users who want to build a multilingual question-and-answer application over a large text corpus, leveraging the latest dense retrieval technologies.
36
 
37
  ### **License/Terms of use**
38
 
@@ -43,7 +43,7 @@ Use of this model is governed by the [NVIDIA Open Model License Agreement](https
43
  **Architecture Type:** Transformer
44
  **Network Architecture:** Fine-tuned Llama3.2 1B Retriever
45
 
46
- This NeMo Retriever embedding model is a transformer encoder - a fine-tuned version of Llama3.2 1b, with 16 layers and an embedding size of 2048, which is trained on public datasets. The AdamW optimizer is employed incorporating 100 warm up steps and 5e-6 learning rate with WarmupDecayLR scheduler. Embedding models for text retrieval are typically trained using a bi-encoder architecture. This involves encoding a pair of sentences (for example, query and chunked passages) independently using the embedding model. Contrastive learning is used to maximize the similarity between the query and the passage that contains the answer, while minimizing the similarity between the query and sampled negative passages not useful to answer the question.
47
 
48
  ### **Input**
49
 
@@ -139,13 +139,13 @@ print(scores.tolist())
139
 
140
  ### **Software Integration**
141
 
142
- **Runtime Engine:** NeMo Retriever embedding NIM
143
  **Supported Hardware Microarchitecture Compatibility**: NVIDIA Ampere, NVIDIA Hopper, NVIDIA Lovelace
144
  **Supported Operating System(s):** Linux
145
 
146
  ### **Model Version(s)**
147
 
148
- Llama Nemotron Retriever Embedding 1B v2
149
  Short Name: llama-nemotron-embed-1b-v2
150
 
151
  ## **Training Dataset & Evaluation**
 
17
 
18
  ### **Description**
19
 
20
+ The Llama Nemotron Embedding 1B model is optimized for **multilingual and cross-lingual** text question-answering retrieval with **support for long documents (up to 8192 tokens) and dynamic embedding size (Matryoshka Embeddings)**. This model was evaluated on 26 languages: English, Arabic, Bengali, Chinese, Czech, Danish, Dutch, Finnish, French, German, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Norwegian, Persian, Polish, Portuguese, Russian, Spanish, Swedish, Thai, and Turkish.
21
 
22
  In addition to enabling multilingual and cross-lingual question-answering retrieval, this model reduces the data storage footprint by 35x through dynamic embedding sizing and support for longer token length, making it feasible to handle large-scale datasets efficiently.
23
 
 
25
 
26
  This model is ready for commercial use.
27
 
28
+ The Llama Nemotron Embedding 1B model is a part of the NVIDIA NeMo Retriever collection of NIM, which provide state-of-the-art, commercially-ready models and microservices, optimized for the lowest latency and highest throughput. It features a production-ready information retrieval pipeline with enterprise support. The models that form the core of this solution have been trained using responsibly selected, auditable data sources. With multiple pre-trained models available as starting points, developers can also readily customize them for domain-specific use cases, such as information technology, human resource help assistants, and research & development research assistants.
29
 
30
  We are excited to announce the open sourcing of this commercial embedding model. For users interested in deploying this model in production environments, it is also available via the model API in NVIDIA Inference Microservices (NIM) at [llama-nemotron-embed-1b-v2](https://build.nvidia.com/nvidia/llama-3_2-nv-embedqa-1b-v2).
31
 
32
 
33
  ### **Intended use**
34
 
35
+ The Llama Nemotron Embedding 1B model is most suitable for users who want to build a multilingual question-and-answer application over a large text corpus, leveraging the latest dense retrieval technologies.
36
 
37
  ### **License/Terms of use**
38
 
 
43
  **Architecture Type:** Transformer
44
  **Network Architecture:** Fine-tuned Llama3.2 1B Retriever
45
 
46
+ This NeMo embedding model is a transformer encoder - a fine-tuned version of Llama3.2 1b, with 16 layers and an embedding size of 2048, which is trained on public datasets. The AdamW optimizer is employed incorporating 100 warm up steps and 5e-6 learning rate with WarmupDecayLR scheduler. Embedding models for text retrieval are typically trained using a bi-encoder architecture. This involves encoding a pair of sentences (for example, query and chunked passages) independently using the embedding model. Contrastive learning is used to maximize the similarity between the query and the passage that contains the answer, while minimizing the similarity between the query and sampled negative passages not useful to answer the question.
47
 
48
  ### **Input**
49
 
 
139
 
140
  ### **Software Integration**
141
 
142
+ **Runtime Engine:** Llama Nemotron embedding NIM
143
  **Supported Hardware Microarchitecture Compatibility**: NVIDIA Ampere, NVIDIA Hopper, NVIDIA Lovelace
144
  **Supported Operating System(s):** Linux
145
 
146
  ### **Model Version(s)**
147
 
148
+ Llama Nemotron Embedding 1B v2
149
  Short Name: llama-nemotron-embed-1b-v2
150
 
151
  ## **Training Dataset & Evaluation**