Class LabeledPairwiseStringEvalChain

constructor

• new LabeledPairwiseStringEvalChain(fields): LabeledPairwiseStringEvalChain

• Parameters

  • fields: LLMChainInput<EvalOutputType, BaseLanguageModel<any, BaseLanguageModelCallOptions>>

  Returns LabeledPairwiseStringEvalChain

llm

outputKey

outputParser

prompt

requiresInput

skipReferenceWarning

Optional criterionName

Optional evaluationName

Optional llmKwargs

Optional memory

Optional skipInputWarning

inputKeys

• get inputKeys(): string[]

• Returns string[]

outputKeys

• get outputKeys(): string[]

• Returns string[]

apply

• apply(inputs, config?): Promise<ChainValues[]>

• Call the chain on all inputs in the list

  Parameters

  • inputs: ChainValues[]

  • Optional config: (BaseCallbackConfig | Callbacks)[]

  Returns Promise<ChainValues[]>

call

• call(values, config?): Promise<ChainValues>

• Run the core logic of this chain and add to output if desired.

  Wraps _call and handles memory.

  Parameters

  • values: ChainValues & BaseLanguageModelCallOptions

  • Optional config: BaseCallbackConfig | Callbacks

  Returns Promise<ChainValues>

checkEvaluationArgs

• checkEvaluationArgs(reference?, input?): void

• Check if the evaluation arguments are valid.

  Parameters

  • Optional reference: string

    The reference label.

  • Optional input: string

    The input string.

  Returns void

  Throws

  If the evaluator requires an input string but none is provided, or if the evaluator requires a reference label but none is provided.

evaluateStringPairs

• evaluateStringPairs(args, callOptions?, config?): Promise<ChainValues>

• Evaluate the output string pairs.

  Parameters

  • args: LLMPairwiseStringEvaluatorArgs

  • Optional callOptions: BaseLanguageModelCallOptions

  • Optional config: BaseCallbackConfig | Callbacks

  Returns Promise<ChainValues>

  A dictionary containing the preference, scores, and/or other information.

invoke

• invoke(input, config?): Promise<ChainValues>

• Invoke the chain with the provided input and returns the output.

  Parameters

  • input: ChainValues

    Input values for the chain run.

  • Optional config: BaseCallbackConfig

    Optional configuration for the Runnable.

  Returns Promise<ChainValues>

  Promise that resolves with the output of the chain run.

predict

• predict(values, callbackManager?): Promise<EvalOutputType>

• Format prompt with values and pass to LLM

  Parameters

  • values: ChainValues & BaseLanguageModelCallOptions

    keys to pass to prompt template

  • Optional callbackManager: CallbackManager

    CallbackManager to use

  Returns Promise<EvalOutputType>

  Completion from LLM.

  Example

  llm.predict({ adjective: "funny" })
  Copy

prepOutputs

• prepOutputs(inputs, outputs, returnOnlyOutputs?): Promise<Record<string, unknown>>

• Parameters

  • inputs: Record<string, unknown>

  • outputs: Record<string, unknown>

  • returnOnlyOutputs: boolean = false

  Returns Promise<Record<string, unknown>>

run

• run(input, config?): Promise<string>

• Parameters

  • input: any

  • Optional config: BaseCallbackConfig | Callbacks

  Returns Promise<string>

serialize

• serialize(): SerializedLLMChain

• Returns SerializedLLMChain

  Deprecated

Static deserialize

• deserialize(data): Promise<LLMChain<string, BaseLanguageModel<any, BaseLanguageModelCallOptions>>>

• Parameters

  • data: SerializedLLMChain

  Returns Promise<LLMChain<string, BaseLanguageModel<any, BaseLanguageModelCallOptions>>>

Static fromLLM

• fromLLM(llm, criteria?, chainOptions?): Promise<PairwiseStringEvalChain>

• Create a new instance of the PairwiseStringEvalChain.

  Parameters

  • llm: BaseLanguageModel<any, BaseLanguageModelCallOptions>

  • Optional criteria: CriteriaLike

    The criteria to use for evaluation.

  • Optional chainOptions: Partial<Omit<LLMEvalChainInput<EvalOutputType, BaseLanguageModel<any, BaseLanguageModelCallOptions>>, "llm">>

    Options to pass to the chain.

  Returns Promise<PairwiseStringEvalChain>

Static resolvePairwiseCriteria

• resolvePairwiseCriteria(criteria?): Record<string, string>

• Parameters

  • Optional criteria: CriteriaLike

  Returns Record<string, string>