Class OllamaModel

Protected ReadonlyDEFAULT_REQUEST_SETTINGS

Protectedhost

Protected Readonlymodel

Protected Optional ReadonlytokenUsageService

ProtectedbudgetTokensToEffortLevel

• budgetTokensToEffortLevel(
      budgetTokens: undefined | number,
  ): "low" | "medium" | "high"

  Maps budget tokens to Ollama effort levels.

  Parameters

  • budgetTokens: undefined | number

    Optional budget tokens from thinking mode settings.

  Returns "low" | "medium" | "high"

  The effort level ('low', 'medium', or 'high').

ProtectedcheckThinkingSupport

• checkThinkingSupport(ollama: Ollama, model: string): Promise<boolean>

  Check if the Ollama server supports thinking.

  Use the Ollama 'show' request to get information about the model, so we can check the capabilities for the 'thinking' capability.

  Parameters

  • ollama: Ollama

    The Ollama client instance.

  • model: string

    The name of the Ollama model.

  Returns Promise<boolean>

  A boolean indicating whether the Ollama model supports thinking.

ProtecteddispatchRequest

• dispatchRequest(
      ollama: Ollama,
      ollamaRequest: ExtendedChatRequest,
      structured: boolean,
      cancellation?: CancellationToken,
      thinkingMode?: ThinkingModeSettings,
  ): Promise<LanguageModelResponse>

  Parameters

  • ollama: Ollama
  • ollamaRequest: ExtendedChatRequest
  • structured: boolean
  • Optionalcancellation: CancellationToken
  • OptionalthinkingMode: ThinkingModeSettings

  Returns Promise<LanguageModelResponse>

ProtectedgetSettings

• getSettings(request: LanguageModelRequest): Partial<ChatRequest>

  Retrieves the settings for the chat request, merging the request-specific settings with the default settings.

  Parameters

  • request: LanguageModelRequest

    The language model request containing specific settings.

  Returns Partial<ChatRequest>

  A partial ChatRequest object containing the merged settings.

ProtectedgetThinkingParameter

• getThinkingParameter(
      thinkingMode: undefined | ThinkingModeSettings,
      model: string,
  ): boolean | "low" | "medium" | "high"

  Determines the value for Ollama's 'think' parameter based on the request's thinking mode settings.

  Note: Most models support boolean values for 'think', but some models (e.g., GPT-OSS) require effort levels ('low', 'medium', 'high') and ignore boolean values.

  Parameters

  • thinkingMode: undefined | ThinkingModeSettings

    The thinking mode settings from the request.

  • model: string

    The model name to check for special handling.

  Returns boolean | "low" | "medium" | "high"

  The appropriate 'think' parameter value for the model.

ProtectedhandleNonStreamingRequest

• handleNonStreamingRequest(
      ollama: Ollama,
      chatRequest: ExtendedNonStreamingChatRequest,
      cancellation?: CancellationToken,
      thinkingMode?: ThinkingModeSettings,
  ): Promise<LanguageModelResponse>

  Parameters

  • ollama: Ollama
  • chatRequest: ExtendedNonStreamingChatRequest
  • Optionalcancellation: CancellationToken
  • OptionalthinkingMode: ThinkingModeSettings

  Returns Promise<LanguageModelResponse>

ProtectedhandleStreamingRequest

• handleStreamingRequest(
      ollama: Ollama,
      chatRequest: ExtendedChatRequest,
      cancellation?: CancellationToken,
      thinkingMode?: ThinkingModeSettings,
  ): Promise<LanguageModelStreamResponse>

  Parameters

  • ollama: Ollama
  • chatRequest: ExtendedChatRequest
  • Optionalcancellation: CancellationToken
  • OptionalthinkingMode: ThinkingModeSettings

  Returns Promise<LanguageModelStreamResponse>

ProtectedhandleStructuredOutputRequest

• handleStructuredOutputRequest(
      ollama: Ollama,
      chatRequest: ChatRequest,
  ): Promise<LanguageModelParsedResponse>

  Parameters

  • ollama: Ollama
  • chatRequest: ChatRequest

  Returns Promise<LanguageModelParsedResponse>

ProtectedinitializeOllama

• initializeOllama(): Ollama

  Returns Ollama

ProtectedrequiresEffortLevel

• requiresEffortLevel(model: string): boolean

  Checks if the model requires effort levels instead of boolean for the 'think' parameter.

  Parameters

  • model: string

    The model name to check.

  Returns boolean

  true if the model requires effort levels.

ProtectedtoOllamaMessage

• toOllamaMessage(message: LanguageModelMessage): undefined | Message

  Parameters

  • message: LanguageModelMessage

  Returns undefined | Message

ProtectedtoOllamaMessageRole

• toOllamaMessageRole(message: LanguageModelMessage): string

  Parameters

  • message: LanguageModelMessage

  Returns string

ProtectedtoOllamaTool

• toOllamaTool(tool: ToolRequest): ToolWithHandler

  Parameters

  • tool: ToolRequest

  Returns ToolWithHandler