ollama

Tag: v0.9.6

Jesse Gross ea79003180 kvcache: Skip computing causal mask for worst case graph reservation Computing an attention mask for a large context and max batch is expensive - over 100ms. Models like Gemma3 that have multiple types of caches and custom attention masks need to do this 4 times, so this adds approximately 500ms to startup time when using 128k context When we are reserving the worst case graph, we don't need the mask, only its shape, so we can skip this.		10 months ago
..
cache.go	ollamarunner: Preallocate worst case graph at startup	12 months ago
causal.go	kvcache: Skip computing causal mask for worst case graph reservation	10 months ago
causal_test.go	ml: Panic rather than return error on tensor allocation failure	10 months ago
encoder.go	ollamarunner: Preallocate worst case graph at startup	12 months ago
wrapper.go	ollamarunner: Preallocate worst case graph at startup	12 months ago