ollama

Commit Graph

Author	SHA1	Message	Date
Michael Yang	f95a1f2bef	feat: add trace log level (#10650 ) reduce prompt log to trace level	11 months ago
Devon Rifkin	44b466eeb2	config: update default context length to 4096	11 months ago
Devon Rifkin	dd93e1af85	Revert "increase default context length to 4096 (#10364 )" This reverts commit `424f648632`.	11 months ago
Devon Rifkin	424f648632	increase default context length to 4096 (#10364 ) * increase default context length to 4096 We lower the default numParallel from 4 to 2 and use these "savings" to double the default context length from 2048 to 4096. We're memory neutral in cases when we previously would've used numParallel == 4, but we add the following mitigation to handle some cases where we would have previously fallen back to 1x2048 due to low VRAM: we decide between 2048 and 4096 using a runtime check, choosing 2048 if we're on a one GPU system with total VRAM of <= 4 GB. We purposefully don't check the available VRAM because we don't want the context window size to change unexpectedly based on the available VRAM. We plan on making the default even larger, but this is a relatively low-risk change we can make to quickly double it. * fix tests add an explicit context length so they don't get truncated. The code that converts -1 from being a signal for doing a runtime check isn't running as part of these tests. * tweak small gpu message * clarify context length default also make it actually show up in `ollama serve --help`	11 months ago
Eries Trisnadi	dc13813a03	server: allow vscode-file origins (#9313 )	1 year ago
Parth Sareen	314573bfe8	config: allow setting context length through env var (#8938 ) * envconfig: allow setting context length through env var	1 year ago
Blake Mizerany	68bac1e0a6	server: group routes by category and purpose (#9270 ) The route assembly in Handler lacked clear organization making it difficult scan for routes and their relationships to each other. This commit aims to fix that by reordering the assembly of routes to group them by category and purpose. Also, be more specific about what "config" refers to (it is about CORS if you were wondering... I was.)	1 year ago
Jeffrey Morgan	48708ca0d5	server: allow vscode-webview origin (#7273 )	1 year ago
Daniel Hiltgen	6719097649	llm: make load time stall duration configurable via OLLAMA_LOAD_TIMEOUT With the new very large parameter models, some users are willing to wait for a very long time for models to load.	2 years ago
Michael Yang	386af6c1a0	passthrough OLLAMA_HOST path to client	2 years ago
Michael Yang	85d9d73a72	comments	2 years ago
Michael Yang	78140a712c	cleanup tests	2 years ago
Michael Yang	8570c1c0ef	keepalive	2 years ago
Michael Yang	55cd3ddcca	bool	2 years ago
Michael Yang	d1a5227cad	origins	2 years ago
Michael Yang	4f1afd575d	host	2 years ago
Michael Yang	35b89b2eab	rfc: dynamic environ lookup	2 years ago
Daniel Hiltgen	955f2a4e03	Only set default keep_alive on initial model load This change fixes the handling of keep_alive so that if client request omits the setting, we only set this on initial load. Once the model is loaded, if new requests leave this unset, we'll keep whatever keep_alive was there.	2 years ago
Patrick Devine	c69bc19e46	move OLLAMA_HOST to envconfig (#5009 )	2 years ago
Patrick Devine	4cc3be3035	Move envconfig and consolidate env vars (#4608 )	2 years ago
Jeffrey Morgan	38255d2af1	Use flash attention flag for now (#4580 ) * put flash attention behind flag for now * add test * remove print * up timeout for sheduler tests	2 years ago
Daniel Hiltgen	824ee5446f	Fix envconfig unit test	2 years ago
Daniel Hiltgen	f56aa20014	Centralize server config handling This moves all the env var reading into one central module and logs the loaded config once at startup which should help in troubleshooting user server logs	2 years ago

20 Commits (dc5a645434f0ea6364c426c6ba112da1afa40cb2)