Classification and quantization
#8
by
kavyamali
- opened
Is this model capable enough to handle classification?
I tried the following prompt:
Task: Classify sentiment.
Input: "I waited 3 hours for nothing."
Respond with exactly one word:
positive / neutral / negative
END
Output: *End Message Display- Last thing remaining includes closing down whole conversation
.... (It started hallucinating here, continuing spitting random words till ctx-size runs out. I'm using 16k token length.)
Also, this model might be depreciated now, but is there a possibility of there being an upgrade to this model with more quantization?