ollama

History

Jesse Gross f66216e399 ggml: Support heterogeneous KV cache layer sizes in memory estimation Gemma3 uses sliding windows for its context on 5/6 layers, significantly reducing memory usage but leading to uneven usage across layers, which makes allocation to the correct GPU difficult. We currently estimate very conservatively by assuming all layers are consistent at the max size. Llama3.2-vision is also inconsistent between self attention and cross attention layers - at moment, we calculate the correct total size and then average this across layers. In some cases, this may lead to crashes if a large layer is placed on a GPU sized by the average. This allows memory estimation to calculate per-layer KV cache size and take this account when placing layers onto GPUs. We already do this for weights that vary per-tensor, so this is a logical extension. Fixes #9730 Fixes #9890		1 year ago
..
internal	server/internal/client/ollama: fix file descriptor management in Pull (#9931)	1 year ago
testdata/tools	all: fix typos in documentation, code, and comments (#7021)	2 years ago
auth.go	fix nil deref in auth.go	2 years ago
create.go	server: validate local path on safetensor create (#9379)	1 year ago
create_test.go	server: validate local path on safetensor create (#9379)	1 year ago
download.go	server: increase timeout in stall detection from 5s to 30s (#8831)	2 years ago
fixblobs.go	server: replace blob prefix separator from ':' to '-' (#3146)	2 years ago
fixblobs_test.go	server: replace blob prefix separator from ':' to '-' (#3146)	2 years ago
images.go	next ollama runner (#7913)	2 years ago
layer.go	One corrupt manifest should not wedge model operations (#7515)	2 years ago
manifest.go	One corrupt manifest should not wedge model operations (#7515)	2 years ago
manifest_test.go	One corrupt manifest should not wedge model operations (#7515)	2 years ago
model.go	templates: add autotemplate for gemma3 (#9880)	1 year ago
model_test.go	Update the /api/create endpoint to use JSON (#7935)	2 years ago
modelpath.go	server: more support for mixed-case model names (#8017)	2 years ago
modelpath_test.go	server: more support for mixed-case model names (#8017)	2 years ago
prompt.go	gemma3: Allow multiple image in a single input	1 year ago
prompt_test.go	prompt: Don't trim whitespace from prompts	2 years ago
routes.go	add verbose mode to the show command (#9640)	1 year ago
routes_create_test.go	next ollama runner (#7913)	2 years ago
routes_delete_test.go	Update the /api/create endpoint to use JSON (#7935)	2 years ago
routes_generate_test.go	next ollama runner (#7913)	2 years ago
routes_list_test.go	Update the /api/create endpoint to use JSON (#7935)	2 years ago
routes_test.go	server/internal/client/ollama: hold DiskCache on Registry (#9463)	1 year ago
sched.go	ggml: Support heterogeneous KV cache layer sizes in memory estimation	1 year ago
sched_test.go	next ollama runner (#7913)	2 years ago
sparse_common.go	Don't hard fail on sparse setup error	2 years ago
sparse_windows.go	Don't hard fail on sparse setup error	2 years ago
upload.go	server: always print upload/download part info (#8832)	2 years ago