Repository navigation
Wrong and irrelevant answers with very Basic Usage #115
Description
I followed Basic Usage section in documentation, and it appears that when sending very basic prompts for completion, the Model returns irrelevant completions/answers in the good scenario, and in worst scenario it returns delusional completions that looks like they were randomly picked without any connection, or related matching pattern or context. Within the response itself, there is endless repetition of sentences, that limited only by the requested number of maximum tokens to predict ( -n flag)
Basic prompt for completion ( deterministic):
python run_inference.py -m models/Llama3-8B-1.58-100B-tokens/ggml-model-tl2.gguf -p "internet is" -temp 0Response from the Model:
internet is a first step in the development of a new generation of robots that can be used in a wide variety of applications, medical, military, and industrial. The robot is designed to be able to perform a wide variety of tasks, including the ability to perform tasks that are not currently possible for humans. The robot is designed to be able to perform tasks that are not currently possible for humans. The robot is designed to be able to perform tasks that are not currently possible for humans. The robot is designed to be able to perform tasks that are not currently possible for humans. The robot is designed to be able to perform tasks that are not currently possibleBasic prompt for completion 2 ( with temperature =1 , to increase creativity):
python run_inference.py -m models/Llama3-8B-1.58-100B-tokens/ggml-model-tl2.gguf -p "internet is" -temp 1Response:
internet is a second of the five senses in the body and is closely linked to the other four. The other four senses include sight, smell, taste and touch. The human eye has the ability to see a wide variety of colors, from red to green to blue to purple. In the same way that humans can see a variety of colors, the human eye can also see a variety of shapes and sizes, such as large, small, square or triangular. It is a fact that the human eye is very useful and can see a lot. The human eye is able to see objects that are far away and objects that are close up to the eyeReacted by Ryo Nasu and sabeaussan
Activity
sabeaussan commentedon Feb 10, 2025
Same issue here, did you find any solution ?
sd983527 commentedon Apr 17, 2025
we recommend to download the new official model. thanks. https://huggingface.co/microsoft/bitnet-b1.58-2B-4T-gguf