Class TrajectoryEvalChain

A chain for evaluating ReAct style agents.

This chain is used to evaluate ReAct style agents by reasoning about the sequence of actions taken and their outcomes.

Hierarchy

AgentTrajectoryEvaluator
- TrajectoryEvalChain

Constructors

constructor

new TrajectoryEvalChain(fields): TrajectoryEvalChain
Parameters
- fields: LLMChainInput<EvalOutputType, BaseLanguageModel<any, BaseLanguageModelCallOptions>>
Returns TrajectoryEvalChain
Inherited from AgentTrajectoryEvaluator.constructor
- Defined in docs/api_refs/langchain/src/chains/llm_chain.ts:121

Properties

llm

llm: BaseLanguageModel<any, BaseLanguageModelCallOptions>

outputKey

outputKey: string = "text"

outputParser

outputParser: TrajectoryOutputParser = ...

prompt

prompt: BasePromptTemplate<any, BasePromptValue, any>

requiresInput

requiresInput: boolean = true

requiresReference

requiresReference: boolean = false

`Optional` criterionName

criterionName?: string

`Optional` evaluationName

evaluationName?: string = ...

`Optional` llmKwargs

llmKwargs?: BaseLanguageModelCallOptions

`Optional` memory

memory?: BaseMemory

`Optional` skipInputWarning

skipInputWarning?: string = ...

`Optional` skipReferenceWarning

skipReferenceWarning?: string = ...

Accessors

inputKeys

get inputKeys(): string[]
Returns string[]
Inherited from AgentTrajectoryEvaluator.inputKeys
- Defined in docs/api_refs/langchain/src/chains/llm_chain.ts:113

outputKeys

get outputKeys(): string[]
Returns string[]
Inherited from AgentTrajectoryEvaluator.outputKeys
- Defined in docs/api_refs/langchain/src/chains/llm_chain.ts:117

Methods

apply

apply(inputs, config?): Promise<ChainValues[]>
Call the chain on all inputs in the list
Parameters
- inputs: ChainValues[]
- Optional config: (BaseCallbackConfig | Callbacks)[]
Returns Promise<ChainValues[]>
Inherited from AgentTrajectoryEvaluator.apply
- Defined in docs/api_refs/langchain/src/chains/base.ts:252

call

call(values, config?): Promise<ChainValues>
Run the core logic of this chain and add to output if desired.

Wraps _call and handles memory.
Parameters
- values: ChainValues & BaseLanguageModelCallOptions
- Optional config: BaseCallbackConfig | Callbacks
Returns Promise<ChainValues>
Inherited from AgentTrajectoryEvaluator.call
- Defined in docs/api_refs/langchain/src/chains/llm_chain.ts:178

checkEvaluationArgs

checkEvaluationArgs(reference?, input?): void
Check if the evaluation arguments are valid.
Parameters
- Optional reference: string
  
  The reference label.
- Optional input: string
  
  The input string.
Returns void

Throws
If the evaluator requires an input string but none is provided, or if the evaluator requires a reference label but none is provided.
Inherited from AgentTrajectoryEvaluator.checkEvaluationArgs
- Defined in docs/api_refs/langchain/src/evaluation/base.ts:50

evaluateAgentTrajectory

evaluateAgentTrajectory(args, callOptions?, config?): Promise<ChainValues>
Evaluate a trajectory.
Parameters
- args: LLMTrajectoryEvaluatorArgs
- Optional callOptions: BaseLanguageModelCallOptions
- Optional config: BaseCallbackConfig | Callbacks
Returns Promise<ChainValues>
The evaluation result.
Inherited from AgentTrajectoryEvaluator.evaluateAgentTrajectory
- Defined in docs/api_refs/langchain/src/evaluation/base.ts:346

formatReference

formatReference(reference?): string
Parameters
- Optional reference: string
Returns string
- Defined in docs/api_refs/langchain/src/evaluation/agents/trajectory.ts:188

getAgentTrajectory

getAgentTrajectory(steps): string
Get the agent trajectory as a formatted string.
Parameters
- steps: AgentStep[]
  
  The agent trajectory.
Returns string
The formatted agent trajectory.
- Defined in docs/api_refs/langchain/src/evaluation/agents/trajectory.ts:173

invoke

invoke(input, config?): Promise<ChainValues>
Invoke the chain with the provided input and returns the output.
Parameters
- input: ChainValues
  
  Input values for the chain run.
- Optional config: BaseCallbackConfig
  
  Optional configuration for the Runnable.
Returns Promise<ChainValues>
Promise that resolves with the output of the chain run.
Inherited from AgentTrajectoryEvaluator.invoke
- Defined in docs/api_refs/langchain/src/chains/base.ts:83

predict

predict(values, callbackManager?): Promise<EvalOutputType>
Format prompt with values and pass to LLM
Parameters
- values: ChainValues & BaseLanguageModelCallOptions
  
  keys to pass to prompt template
- Optional callbackManager: CallbackManager
  
  CallbackManager to use
Returns Promise<EvalOutputType>
Completion from LLM.
Example
```
llm.predict({ adjective: "funny" })
```
Inherited from AgentTrajectoryEvaluator.predict
- Defined in docs/api_refs/langchain/src/chains/llm_chain.ts:244

prepOutputs

prepOutputs(inputs, outputs, returnOnlyOutputs?): Promise<Record<string, unknown>>
Parameters
- inputs: Record<string, unknown>
- outputs: Record<string, unknown>
- returnOnlyOutputs: boolean = false
Returns Promise<Record<string, unknown>>
Inherited from AgentTrajectoryEvaluator.prepOutputs
- Defined in docs/api_refs/langchain/src/chains/base.ts:99

run

run(input, config?): Promise<string>
Parameters
- input: any
- Optional config: BaseCallbackConfig | Callbacks
Returns Promise<string>
Inherited from AgentTrajectoryEvaluator.run
- Defined in docs/api_refs/langchain/src/chains/base.ts:140

serialize

serialize(): SerializedLLMChain
Returns SerializedLLMChain

Deprecated
Inherited from AgentTrajectoryEvaluator.serialize
- Defined in docs/api_refs/langchain/src/chains/llm_chain.ts:273

`Static` deserialize

deserialize(data): Promise<LLMChain<string, BaseLanguageModel<any, BaseLanguageModelCallOptions>>>
Parameters
- data: SerializedLLMChain
Returns Promise<LLMChain<string, BaseLanguageModel<any, BaseLanguageModelCallOptions>>>
Inherited from AgentTrajectoryEvaluator.deserialize
- Defined in docs/api_refs/langchain/src/chains/llm_chain.ts:257

`Static` fromLLM

fromLLM(llm, agentTools?, chainOptions?): Promise<TrajectoryEvalChain>
Create a new TrajectoryEvalChain.
Parameters
- llm: BaseChatModel<BaseLanguageModelCallOptions>
- Optional agentTools: StructuredTool<ZodObject<any, any, any, any, {}>>[]
  
  The tools used by the agent.
- Optional chainOptions: Partial<Omit<LLMEvalChainInput<EvalOutputType, BaseLanguageModel<any, BaseLanguageModelCallOptions>>, "llm">>
  
  The options for the chain.
Returns Promise<TrajectoryEvalChain>
- Defined in docs/api_refs/langchain/src/evaluation/agents/trajectory.ts:134

`Static` resolveTrajectoryPrompt

resolveTrajectoryPrompt(prompt?, agentTools?): BasePromptTemplate<any, BasePromptValue, any> | ChatPromptTemplate<any, any>
Parameters
- Optional prompt: BasePromptTemplate<any, BasePromptValue, any>
- Optional agentTools: StructuredTool<ZodObject<any, any, any, any, {}>>[]
Returns BasePromptTemplate<any, BasePromptValue, any> | ChatPromptTemplate<any, any>
- Defined in docs/api_refs/langchain/src/evaluation/agents/trajectory.ts:98

`Static` toolsDescription

toolsDescription(agentTools): string
Get the description of the agent tools.
Parameters
- agentTools: StructuredTool<ZodObject<any, any, any, any, {}>>[]
Returns string
The description of the agent tools.
- Defined in docs/api_refs/langchain/src/evaluation/agents/trajectory.ts:119

Class TrajectoryEvalChain

Hierarchy

Index

Constructors

Properties

Accessors

Methods

Constructors

constructor

Parameters

fields: LLMChainInput<EvalOutputType, BaseLanguageModel<any, BaseLanguageModelCallOptions>>

Returns TrajectoryEvalChain

Properties

llm

outputKey

outputParser

prompt

requiresInput

requiresReference

Optional criterionName

Optional evaluationName

Optional llmKwargs

Optional memory

Optional skipInputWarning

Optional skipReferenceWarning

Accessors

inputKeys

Returns string[]

outputKeys

Returns string[]

Methods

apply

Parameters

inputs: ChainValues[]

Optional config: (BaseCallbackConfig | Callbacks)[]

Returns Promise<ChainValues[]>

call

Parameters

values: ChainValues & BaseLanguageModelCallOptions

Optional config: BaseCallbackConfig | Callbacks

Returns Promise<ChainValues>

checkEvaluationArgs

Parameters

Optional reference: string

Optional input: string

Returns void

Throws

evaluateAgentTrajectory

Parameters

args: LLMTrajectoryEvaluatorArgs

Optional callOptions: BaseLanguageModelCallOptions

Optional config: BaseCallbackConfig | Callbacks

Returns Promise<ChainValues>

formatReference

Parameters

Optional reference: string

Returns string

getAgentTrajectory

Parameters

steps: AgentStep[]

Returns string

invoke

Parameters

input: ChainValues

Optional config: BaseCallbackConfig

Returns Promise<ChainValues>

predict

Parameters

values: ChainValues & BaseLanguageModelCallOptions

Optional callbackManager: CallbackManager

Returns Promise<EvalOutputType>

Example

prepOutputs

Parameters

inputs: Record<string, unknown>

outputs: Record<string, unknown>

returnOnlyOutputs: boolean = false

Returns Promise<Record<string, unknown>>

run

Parameters

`Optional` criterionName

`Optional` evaluationName

`Optional` llmKwargs

`Optional` memory

`Optional` skipInputWarning

`Optional` skipReferenceWarning

`Optional` config: (BaseCallbackConfig | Callbacks)[]

`Optional` config: BaseCallbackConfig | Callbacks

`Optional` reference: string

`Optional` input: string

`Optional` callOptions: BaseLanguageModelCallOptions

`Optional` config: BaseCallbackConfig | Callbacks

`Optional` reference: string

`Optional` config: BaseCallbackConfig

`Optional` callbackManager: CallbackManager

`Optional` config: BaseCallbackConfig | Callbacks

`Static` deserialize

`Static` fromLLM

`Optional` agentTools: StructuredTool<ZodObject<any, any, any, any, {}>>[]

`Optional` chainOptions: Partial<Omit<LLMEvalChainInput<EvalOutputType, BaseLanguageModel<any, BaseLanguageModelCallOptions>>, "llm">>

`Static` resolveTrajectoryPrompt

`Optional` prompt: BasePromptTemplate<any, BasePromptValue, any>

`Optional` agentTools: StructuredTool<ZodObject<any, any, any, any, {}>>[]

`Static` toolsDescription