beethogedeon/financial-sentiment-Gemma-300m

SetFit with unsloth/embeddinggemma-300m

This is a SetFit model that can be used for Text Classification. This SetFit model uses unsloth/embeddinggemma-300m as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: unsloth/embeddinggemma-300m
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 2048 tokens
• Number of Classes: 3 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
neutral	'Technopolis plans to develop in stages an area of no less than 100,000 square meters in order to host companies working in computer technologies and telecommunications , the statement said .' 'In Sweden , Gallerix accumulated SEK denominated sales were down 1 % and EUR denominated sales were up 11 % .' 'The company supports its global customers in developing new technologies and offers a fast route from product development to applications and volume production .'
negative	'The international electronic industry company Elcoteq has laid off tens of employees from its Tallinn facility ; contrary to earlier layoffs the company contracted the ranks of its office workers , the daily Postimees reported .' 'A tinyurl link takes users to a scamming site promising that users can earn thousands of dollars by becoming a Google ( NASDAQ : GOOG ) Cash advertiser .' 'Compared with the FTSE 100 index , which rose 36.7 points ( or 0.6 % ) on the day , this was a relative price change of -0.2 % .'
positive	'With the new production plant the company would increase its capacity to meet the expected increase in demand and would improve the use of raw materials and therefore increase the production profitability .' "According to the company 's updated strategy for the years 2009-2012 , Basware targets a long-term net sales growth in the range of 20 % -40 % with an operating profit margin of 10 % -20 % of net sales ." "FINANCING OF ASPOCOMP 'S GROWTH Aspocomp is aggressively pursuing its growth strategy by increasingly focusing on technologically more demanding HDI printed circuit boards PCBs ."

Evaluation

Metrics

Label	Accuracy
all	0.8958

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("beethogedeon/financial-sentiment-Gemma-300m")
# Run inference
preds = model("Officials did not disclose the contract value .")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	3	23.6836	62

Label	Training Sample Count
negative	301
neutral	1488
positive	793

Training Hyperparameters

• batch_size: (16, 2)
• num_epochs: (1, 16)
• max_steps: 500
• sampling_strategy: oversampling
• body_learning_rate: (2e-05, 1e-05)
• head_learning_rate: 0.01
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• l2_weight: 0.01
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.002	1	0.2539	-
0.1	50	0.2526	-
0.2	100	0.2431	-
0.3	150	0.2502	-
0.4	200	0.2365	-
0.5	250	0.2268	-
0.6	300	0.2277	-
0.7	350	0.1839	-
0.8	400	0.1644	-
0.9	450	0.1393	-
1.0	500	0.1401	-

Framework Versions

• Python: 3.11.5
• SetFit: 1.1.3
• Sentence Transformers: 5.1.2
• Transformers: 4.53.3
• PyTorch: 2.7.0
• Datasets: 4.1.0
• Tokenizers: 0.21.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}