GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval
Paper
•
2112.07577
•
Published
YAML Metadata
Warning:
empty or missing yaml metadata in repo card
(https://huggingface.co/docs/hub/model-cards#model-card-metadata)
This is the zero-shot baseline model in the paper "GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval"
The training setup:
- Start from
distilbert-base-uncased; - Mine 50 hard negatives for each query on MS MARCO with
sentence-transformers/msmarco-distilbert-base-v3andsentence-transformers/msmarco-MiniLM-L-6-v3; - Do Margin-MSE training on the tuples (including queries, gold relevant, and hard negatives) with the teacher model
cross-encoder/ms-marco-MiniLM-L-6-v2for 70K steps with batch size 75, max. sequence-length 350.
- Downloads last month
- 3