Upload folder using huggingface_hub

README.md

@@ -0,0 +1,305 @@
+---
+language:
+- en
+license: mit
+tags:
+- text-classification
+- narrative-analysis
+- fiction
+- genre-classification
+- literary-analysis
+- creative-writing
+- longformer
+- narrative-embeddings
+datasets:
+- Mitchins/fiction-genre-validation-52
+metrics:
+- accuracy
+model-index:
+- name: Longformer Fiction Genre Classifier
+  results:
+  - task:
+      type: text-classification
+      name: Narrative Genre Classification
+    dataset:
+      name: Fiction Genre Validation Set (52 Stories)
+      type: Mitchins/fiction-genre-validation-52
+    metrics:
+    - type: accuracy
+      value: 67.31
+      name: Accuracy
+library_name: transformers
+pipeline_tag: text-classification
+---
+
+# Longformer Fiction Genre Classifier
+
+## Model Description
+
+This is **not** "yet another genre classifier."
+
+This model recognizes **narrative semantic genres** in long-form fiction—detecting what a story *feels like* and what narrative machinery it uses, rather than simply tagging marketing categories or book descriptions.
+
+### What Makes This Different
+
+Most genre classifiers online:
+- Train on 100-1000 samples (Goodreads tags, book blurbs)
+- Use bag-of-words or TF-IDF (no deep understanding)
+- Focus on short text (tweets, descriptions)
+- Single-label classification
+- Never tested on actual novels
+- No sliding-window inference
+- Predict bookstore shelves, not literary modes
+
+**This model:**
+- ✅ **Narrative-aware**: Trained on actual story text, not marketing copy
+- ✅ **Long-context transformer**: Longformer architecture (4096 token windows)
+- ✅ **Curriculum-trained**: Progressive training from short scenes → long narratives
+- ✅ **Evaluated on real fiction**: Tested on commercial novels and diverse short stories
+- ✅ **Window-based inference**: Produces genre heatmaps across a book
+- ✅ **Semantic genre detection**: Identifies literary modes (tone, structure, diction)
+- ✅ **Catches nuance**: Distinguishes political sci-fi from space opera, literary fantasy from epic fantasy
+
+### What This Model Does
+
+Instead of asking "What shelf would this book go on?", it asks:
+
+- What narrative modes is this text using?
+- What emotional tone and pacing patterns appear?
+- What socio-political structures and themes are present?
+- What genre conventions guide the storytelling?
+
+**Example**: *A Memory Called Empire* → The model correctly identifies it as science_fiction + literary + romance, not just "space opera." That's **literary-correct**, not bookstore-correct.
+
+## Intended Uses
+
+### Primary Use Cases
+
+1. **Fiction RAG systems**: Cluster and retrieve by narrative style/tone
+2. **Book recommendation engines**: "Find books that feel like X"
+3. **Writing assistants**: Analyze draft chapters for genre consistency
+4. **Story AI agents**: Condition generation on narrative mode
+5. **Dataset curation**: Filter fiction corpora by semantic genre
+6. **Subgenre classification**: Build specialized heads (e.g., "cozy mystery", "grimdark fantasy")
+7. **Narrative embeddings**: Use hidden states for "similar writing" search
+8. **Genre arc analysis**: Track how genre shifts across a book's chapters
+
+### What This Model Is NOT For
+
+- ❌ Classifying book blurbs or marketing copy (trained on story text)
+- ❌ Single-sentence genre detection (needs narrative context)
+- ❌ Non-fiction classification (trained exclusively on fiction)
+- ❌ Multi-label prediction (designed for dominant genre, though provides probabilities)
+
+## Model Architecture
+
+- **Base Model**: `allenai/longformer-base-4096`
+- **Architecture**: Longformer (efficient self-attention for long documents)
+- **Max Sequence Length**: 4096 tokens
+- **Parameters**: ~149M (backbone) + classification head
+- **Training Strategy**: Curriculum learning (short → long)
+- **Genres**: 13 semantic categories
+
+### Genre Labels
+
+The model predicts **13 semantic narrative genres**:
+
+- **adventure**: Narrative mode, not bookstore category
+- **contemporary**: Narrative mode, not bookstore category
+- **crime**: Narrative mode, not bookstore category
+- **fantasy**: Narrative mode, not bookstore category
+- **historical**: Narrative mode, not bookstore category
+- **horror**: Narrative mode, not bookstore category
+- **literary**: Narrative mode, not bookstore category
+- **mystery**: Narrative mode, not bookstore category
+- **romance**: Narrative mode, not bookstore category
+- **science_fiction**: Narrative mode, not bookstore category
+- **thriller**: Narrative mode, not bookstore category
+- **war**: Narrative mode, not bookstore category
+- **western**: Narrative mode, not bookstore category
+
+These represent **literary modes** and **narrative structures**, not marketing labels.
+
+## Training Data
+
+- **Training set**: Diverse corpus of fiction excerpts and scenes
+- **Curriculum strategy**: Progressive training from 500-token scenes to 4000-token chapters
+- **Validation set**: 52 original short stories (4 per genre × 13 genres)
+- **Writing styles**: Literary, indie, and blockbuster prose
+
+The training emphasizes:
+1. Narrative structure and pacing
+2. Diction and tone
+3. Thematic elements
+4. Character dynamics and focalization
+5. Genre conventions and tropes
+
+## Performance
+
+### Evaluation Results (52-Story Validation Set)
+
+**Overall Accuracy**: 67.31% (35/52 stories correct)
+
+This was achieved at only **8.6% through training** (checkpoint 2000/23226 steps), suggesting significant room for improvement with full training.
+
+### Genre-Specific Performance
+
+| Tier | Accuracy | Genres |
+|------|----------|--------|
+| Excellent (≥75%) | 75-100% | war (100%), western (100%), science_fiction (75%), horror (75%), romance (100%), literary (75%) |
+| Good (50-74%) | 50-74% | adventure (75%), contemporary (50%), historical (75%), mystery (75%), fantasy (50%) |
+| Challenging (<50%) | <50% | crime (25%), thriller (0%) |
+
+### Known Limitations
+
+1. **Thriller confusion**: Model struggles to distinguish thriller from mystery/crime (0% accuracy)
+2. **Crime vs. Mystery**: Confuses criminal perspective (crime) with investigative perspective (mystery)
+3. **Character-driven blur**: Literary, contemporary, and romance can overlap when character-focused
+4. **Experimental prose**: Indie/experimental writing styles reduce accuracy slightly
+
+### Strengths
+
+- Excellent at genres with clear setting/tone markers (war, western, sci-fi, horror)
+- Handles literary fiction with nuanced themes
+- Distinguishes romance as narrative driver vs. subplot
+- Recognizes historical context as central vs. background
+
+## How to Use
+
+### Basic Classification
+
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+import torch
+
+# Load model
+model_name = "Mitchins/longformer-fiction-genre"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForSequenceClassification.from_pretrained(model_name)
+
+# Classify a text
+text = """Your story text here (can be up to 4096 tokens)"""
+
+inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=4096)
+outputs = model(**inputs)
+
+# Get prediction
+probs = torch.softmax(outputs.logits, dim=-1)
+predicted_class = torch.argmax(probs, dim=-1).item()
+predicted_genre = model.config.id2label[predicted_class]
+confidence = probs[0][predicted_class].item()
+
+print(f"Genre: {predicted_genre} ({confidence:.2%} confidence)")
+```
+
+### Windowed Book Classification
+
+For full novels, use sliding windows to analyze genre distribution:
+
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+import torch
+
+def classify_book_windowed(text, window_size=3500, stride=1750):
+    """
+    Classify a full book using overlapping windows.
+    Returns genre distribution across the book.
+    """
+    tokenizer = AutoTokenizer.from_pretrained("Mitchins/longformer-fiction-genre")
+    model = AutoModelForSequenceClassification.from_pretrained("Mitchins/longformer-fiction-genre")
+
+    # Tokenize full text
+    tokens = tokenizer.encode(text, add_special_tokens=False)
+
+    # Create windows
+    windows = []
+    for i in range(0, len(tokens), stride):
+        window = tokens[i:i + window_size]
+        if len(window) > 100:  # Skip tiny windows
+            windows.append(window)
+        if i + window_size >= len(tokens):
+            break
+
+    # Classify each window
+    genre_votes = []
+    for window_tokens in windows:
+        inputs = {'input_ids': torch.tensor([window_tokens])}
+        outputs = model(**inputs)
+        pred = torch.argmax(outputs.logits, dim=-1).item()
+        genre_votes.append(model.config.id2label[pred])
+
+    # Aggregate results
+    from collections import Counter
+    return Counter(genre_votes)
+
+# Usage
+with open("your_book.txt", "r") as f:
+    book_text = f.read()
+
+genre_dist = classify_book_windowed(book_text)
+print("Genre distribution:", genre_dist)
+```
+
+### Extract Narrative Embeddings
+
+Use the model's hidden states as narrative embeddings:
+
+```python
+model = AutoModelForSequenceClassification.from_pretrained(
+    "Mitchins/longformer-fiction-genre",
+    output_hidden_states=True
+)
+
+inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=4096)
+outputs = model(**inputs)
+
+# Get final layer embedding (mean pooling)
+hidden_states = outputs.hidden_states[-1]  # Last layer
+embedding = hidden_states.mean(dim=1)  # Mean pool
+
+# Use for similarity search, clustering, etc.
+```
+
+## Citation
+
+If you use this model, please cite:
+
+```bibtex
+@model{longformer_fiction_genre,
+  title={Longformer Fiction Genre Classifier: A Narrative Semantic Genre Model},
+  author={Mitchell Currie},
+  year={2024},
+  publisher={HuggingFace},
+  howpublished={\url{https://huggingface.co/Mitchins/longformer-fiction-genre}}
+}
+```
+
+## Related Resources
+
+- **Validation Dataset**: [fiction-genre-validation-52](https://huggingface.co/datasets/Mitchins/fiction-genre-validation-52)
+- **Evaluation Results**: See FINAL_EVALUATION_52_STORIES.md in the repository
+- **Training Details**: See training documentation for curriculum strategy
+- **Inference Script**: Windowed classification script available in repository
+
+## Model Card Authors
+
+Mitchell Currie
+
+## License
+
+MIT License
+
+---
+
+## Future Directions
+
+This model represents a **building block** for narrative intelligence:
+
+1. **Fiction-trained multimodal RAG**: Combine with embeddings for narrative retrieval
+2. **Subgenre specialization**: Fine-tune heads for niche genres (cozy mystery, progression fantasy)
+3. **Genre-aware generation**: Condition story generation on semantic genre
+4. **Cross-genre detection**: Extend to multi-label for genre-blending
+5. **Style transfer**: Use embeddings to guide prose style transformation
+6. **Narrative arc tracking**: Analyze how genre/tone evolves through a story
+
+**Welcome to Narrative ML. This is just the beginning. 🧠📚**

config.json

@@ -0,0 +1,73 @@
+{
+  "architectures": [
+    "LongformerForSequenceClassification"
+  ],
+  "attention_mode": "longformer",
+  "attention_probs_dropout_prob": 0.1,
+  "attention_window": [
+    512,
+    512,
+    512,
+    512,
+    512,
+    512,
+    512,
+    512,
+    512,
+    512,
+    512,
+    512
+  ],
+  "bos_token_id": 0,
+  "eos_token_id": 2,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "adventure",
+    "1": "contemporary",
+    "2": "crime",
+    "3": "fantasy",
+    "4": "historical",
+    "5": "horror",
+    "6": "literary",
+    "7": "mystery",
+    "8": "romance",
+    "9": "science_fiction",
+    "10": "thriller",
+    "11": "war",
+    "12": "western"
+  },
+  "ignore_attention_mask": false,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "adventure": 0,
+    "contemporary": 1,
+    "crime": 2,
+    "fantasy": 3,
+    "historical": 4,
+    "horror": 5,
+    "literary": 6,
+    "mystery": 7,
+    "romance": 8,
+    "science_fiction": 9,
+    "thriller": 10,
+    "war": 11,
+    "western": 12
+  },
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 4098,
+  "model_type": "longformer",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "onnx_export": false,
+  "pad_token_id": 1,
+  "problem_type": "single_label_classification",
+  "sep_token_id": 2,
+  "torch_dtype": "float32",
+  "transformers_version": "4.54.0",
+  "type_vocab_size": 1,
+  "vocab_size": 50265
+}

final_eval_results.json

@@ -0,0 +1,10 @@
+{
+  "eval_loss": 0.07908330112695694,
+  "eval_accuracy": 0.981617379931701,
+  "eval_f1_macro": 0.9816376629462489,
+  "eval_f1_weighted": 0.9816396709662533,
+  "eval_runtime": 2373.738,
+  "eval_samples_per_second": 11.596,
+  "eval_steps_per_second": 0.966,
+  "epoch": 1.3562386980108498
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ecc09bf51c9c12b33542915c616bbafaed4be179b325cc992719df1300cb5b31
+size 594712020

phase3_results.json

@@ -0,0 +1,8 @@
+{
+  "train_runtime": 1.7731,
+  "train_samples_per_second": 279440.99,
+  "train_steps_per_second": 5822.721,
+  "total_flos": 4.4142344258995814e+17,
+  "train_loss": 0.0,
+  "epoch": 1.3562386980108498
+}

special_tokens_map.json

@@ -0,0 +1,15 @@
+{
+  "bos_token": "<s>",
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "unk_token": "<unk>"
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50264": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "tokenizer_class": "LongformerTokenizer",
+  "trim_offsets": true,
+  "unk_token": "<unk>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:93397c42c0040f6662c80addcd9b88f9479cde906b58393e345c3b95389c2de8
+size 5841