ollama

Commit Graph

Author	SHA1	Message	Date
Daniel Hiltgen	27da2cddc5	Fix lingering Q4_0 help reference (#10720 )	11 months ago
Bruce MacDonald	feb8923ada	cmd: add ellipses to truncated show metadata (#10717 ) When a piece of information has been truncated in the show output an ellipses to indicate that more data has not been displayed	11 months ago
Jesse Gross	fe623c2cf4	ollamarunner: Multi-modal worst case graph We currently preallocate compute graph memory for the worst case batch of text tokens. This adds support for doing the same for images. Note that image models are more complicated than text models in how they process their inputs so there may be cases where this approach isn't completely generic for all models. It covers all currently supported models though.	12 months ago
Jesse Gross	3c14461d5d	ollamarunner: Separate text and multimodal graphs For some multimodal models (such as gemma3), we create a single graph that generates the image embedding and then use this in the text model. The embedding tensor is completely opaque to the runner. However, this doesn't work if we need to use the embedding in multiple batches. This can arise if the embedding is larger than the batch size. In these cases (as with llama4), we would like to create views that are more appropriately sized. However, if we do this then the original source tensor is used in multiple graphs, which isn't allowed. To avoid that problem, models with this pattern compute the embedding tensor on first use and recreate the individual views. There is no longer a single vision and text graph. This codifies the pattern of separating vision and text graphs. The logic of computing tensors on demand is moved to the runner, so models no longer have to worry about this. It also gives the runner visibility into the multimodal tensors, which is important for memory management.	11 months ago
Jesse Gross	499ae7311f	ollamarunner: Base cached tokens on current prompt When we restore a sequence from the cache, we split the prompt into the already used tokens (stored in the cache) and new tokens that need to be processed. Currently, the references to the used tokens are coming from the stored previous sequence. However, even though we know that the used tokens are semantically equivalent to the prefix of the prompt, tokens can contain pointers which are no longer valid. As a result, it is better to get the used tokens from the prompt, which has currently valid pointers. This doesn't currently have any impact because it isn't possible to reuse the pointers (which are tensors) anyways. However, it becomes an issue once we can.	11 months ago
Michael Yang	ef202789fa	fix pixel values padding (#10718 ) * panic if trying to pad 4d * fix pixel values padding	11 months ago
Michael Yang	55760195e6	fix mllama conversion (#10716 ) cross attention Q and K projections needs to have their heads swapped, similar to non-cross attention Q and K tensors	11 months ago
Bruce MacDonald	bd68d3ae50	ggml: update qwen25vl vision size estimate (#10711 )	11 months ago
Daniel Hiltgen	ff80718e9c	fix crash in old clients with quantization progress (#10710 ) Older clients assumed the digest was at least 19 characters long so increase the size of the dummy digest to avoid array out of bounds crashes.	11 months ago
Bruce MacDonald	0aa8b371dd	model: add Qwen2.5-VL support (#10385 )	11 months ago
Michael Yang	23125648b8	chore: update mllama to use ollama engine (#10637 )	11 months ago
tej	0478d440f0	Fixed over vram allcation dure to small initial layer sizes. Co-authored-by: Tej Kiran <kiran.tej@amd.com> Co-authored-by: Michael Yang <mxyng@pm.me> Co-authored-by: Tej Kiran <itej89@gmailcom>	11 months ago
Parth Sareen	8cc33f4c2b	llama: fix memory leak for grammar (#10696 )	11 months ago
Jeffrey Morgan	f46df4e5d2	llama: fix defrag patch to defragment when no slots are available (#10695 )	11 months ago
Daniel Hiltgen	c6bcdc4223	Revert "remove cuda v11 (#10569 )" (#10692 ) Bring back v11 until we can better warn users that their driver is too old. This reverts commit `fa393554b9`.	11 months ago
Jeffrey Morgan	4b903f088a	llama: fix crash on snowflake embedding model (#10690 )	11 months ago
Jeffrey Morgan	c7f4ae7b9c	server: add webp image input support (#10653 )	11 months ago
Michael Yang	526b2ed102	fix vocabulary (#10679 )	11 months ago
Bruce MacDonald	a7240c6d63	models: remove unused qwen2vl processing (#10677 )	11 months ago
Daniel Hiltgen	9d6df90805	Follow up to #10363 (#10647 ) The quantization PR didn't block all unsupported file types, which this PR fixes. It also updates the API docs to reflect the now reduced set of supported types.	11 months ago
Jeffrey Morgan	0cefd46f23	llama: update to commit de4c07f93 (#10655 )	11 months ago
Bruce MacDonald	ad035ad595	convert: quantize from safetensors needs kv (#10675 ) When creating a quantized model from safetensors we need the array KV values to be loaded.Changing this value to -1 loads the KV values on the returned layer to be used and saved during quantization.	11 months ago
Michael Yang	f95a1f2bef	feat: add trace log level (#10650 ) reduce prompt log to trace level	11 months ago
HardCodeDev	82a9e9462a	readme: add UnityCodeLama to community integrations (#10665 )	11 months ago
HardCodeDev	76724e2f29	readme: add OllamaPlusPlus C++ library to community integrations (#10664 )	11 months ago
frob	ecf14a220f	llama: allocate grammar buffer based on schema length (#10649 )	11 months ago
frob	69ce44b33c	envconfig: Remove no longer supported max vram var (#10623 ) Co-authored-by: Richard Lyons <frob@cloudstaff.com>	11 months ago
Michael Yang	5969674cf1	feat: add threshold to dump options (#10639 ) ml.Dump will preserve default values if not specified	11 months ago
AliAhmedNada	867d75b21e	readme: add ojira to community integrations (#10648 )	11 months ago
Bruce MacDonald	3fa78598a1	cmd: strip single quotes from image page (#10636 )	11 months ago
Michael Yang	0d6e35d3c6	fix: stream accumulator exits early (#10593 ) the stream accumulator exits as soon as it sees `api.ProgressResponse(status="success")` which isn't strictly correctly since some requests may have multiple successes, e.g. `/api/create` when the source model needs to be pulled.	11 months ago
Devon Rifkin	20c5fd39c8	Merge branch 'main' into drifkin/array-head-count-simple	11 months ago
Michael Yang	6e9a7a2568	lint: enable usetesting, disable tenv (#10594 )	11 months ago
Michael Yang	b585a58121	chore: remove unused ZipReader type (#10621 )	11 months ago
Jeffrey Morgan	fa9973cd7f	api: remove unused sampling parameters (#10581 )	11 months ago
Jesse Gross	3d9498a425	ollamarunner: Use correct constant to remove cache entries The correct constant to remove all entries to the end of the sequence for the Ollama engine is math.MaxInt32. -1 is used by the old engine. The impact of this is currently minimal because it would only occur in situations that are not supported by the implemented models or rarely used options.	11 months ago
Daniel Hiltgen	3098c8b29b	CI: trigger downstream release process (#10508 )	11 months ago
Daniel Hiltgen	5e380c3b42	sched: fix race leading to orphaned runners (#10599 ) If a model is loading, and the request context is canceled during the load by a client closing the connection, and another request is inbound for the same model with a different configuration (context size, etc.) thus requiring a reload, two unload events can be in flight. The first shuts down the original model load, but the second one caused the loss of the new reloading runner reference, thus triggering the leak. The primary fix is detecting the duplicate unload and ignoring the second instance. The load routine is also hardened to ensure we detect clobbering an already present runner and unload it with a warning.	11 months ago
Jeffrey Morgan	392de84031	api: remove unused RetrieveModelResponse type (#10603 )	11 months ago
Daniel Hiltgen	af31ccefc0	fix data race in WriteGGUF (#10598 ) err in the go routine should not be shared with the outer scope	11 months ago
Daniel Hiltgen	fa393554b9	remove cuda v11 (#10569 ) This reduces the size of our Windows installer payloads by ~256M by dropping support for nvidia drivers older than Feb 2023. Hardware support is unchanged. Linux default bundle sizes are reduced by ~600M to 1G.	11 months ago
Aharon Bensadoun	307e3b3e1d	readme: add Flufy to community integrations (#9719 )	11 months ago
Devon Rifkin	4090aca97b	server: send 405 instead of 404 for unallowed methods (#10275 ) Fixes: #5483	11 months ago
Michael Yang	92ce438de0	server: remove internal cmd (#10595 )	11 months ago
Daniel Hiltgen	424810450f	Move quantization to new backend (#10363 ) * Move quantization logic to GGML via new backend This moves the model aware logic to Go code and calls GGMLs quantization code for model creation. * Remove "add model quantizations" This is no longer needed now that quantization is implemented in Go+GGML code directly.	11 months ago
Michael Yang	95e744beeb	discover: fix compiler warnings (#10572 )	11 months ago
Jeffrey Morgan	3b2d2c8326	api: remove unused or unsupported api options (#10574 ) Some options listed in api/types.go are not supported in newer models, or have been deprecated in the past. This is the first of a series of PRs to clean up the API options	11 months ago
Michael Yang	d931ee8f22	create blobs in parallel (#10135 ) * default max term height * error on out of tree files	11 months ago
Jesse Gross	7073600797	ggml: Reduce log level of "key not found" Most of the time this is not an error.	11 months ago
Daniel Hiltgen	b1c40138da	win: lint fix (#10571 )	11 months ago

... 2 3 4 5 6 ...

4433 Commits (30f8a68c4cc55e0f3a717b891931847c97190843) All Branches Search

4433 Commits (30f8a68c4cc55e0f3a717b891931847c97190843)

All Branches