ollama

Tree: 8afa6e83f2

Jesse Gross 499ae7311f ollamarunner: Base cached tokens on current prompt When we restore a sequence from the cache, we split the prompt into the already used tokens (stored in the cache) and new tokens that need to be processed. Currently, the references to the used tokens are coming from the stored previous sequence. However, even though we know that the used tokens are semantically equivalent to the prefix of the prompt, tokens can contain pointers which are no longer valid. As a result, it is better to get the used tokens from the prompt, which has currently valid pointers. This doesn't currently have any impact because it isn't possible to reuse the pointers (which are tensors) anyways. However, it becomes an issue once we can.		1 year ago
..
cache.go	ollamarunner: Base cached tokens on current prompt	1 year ago
cache_test.go	Runner for Ollama engine	2 years ago
image.go	chore: update mllama to use ollama engine (#10637)	1 year ago
image_test.go	Runner for Ollama engine	2 years ago
runner.go	chore: update mllama to use ollama engine (#10637)	1 year ago