Class HuggingFaceInference

Class implementing the Large Language Model (LLM) interface using the Hugging Face Inference API for text generation.

Example

const model = new HuggingFaceInference({
  model: "gpt2",
  temperature: 0.7,
  maxTokens: 50,
});

const res = await model.call(
  "Question: What would be a good company name for a company that makes colorful socks?\nAnswer:"
);
console.log({ res });

Hierarchy

  • LLM
    • HuggingFaceInference

Implements

Constructors

constructor

Properties

apiKey endpointUrl frequencyPenalty includeCredentials maxTokens model temperature topK topP

Constructors

constructor

Properties

apiKey

apiKey: undefined | string = undefined

API key to use.

endpointUrl

endpointUrl: undefined | string = undefined

Custom inference endpoint URL to use

frequencyPenalty

frequencyPenalty: undefined | number = undefined

Penalizes repeated tokens according to frequency

includeCredentials

includeCredentials: undefined | string | boolean = undefined

Credentials to use for the request. If this is a string, it will be passed straight on. If it's a boolean, true will be "include" and false will not send credentials at all.

maxTokens

maxTokens: undefined | number = undefined

Maximum number of tokens to generate in the completion.

model

model: string = "gpt2"

Model to use

temperature

temperature: undefined | number = undefined

Sampling temperature to use

topK

topK: undefined | number = undefined

Integer to define the top tokens considered within the sample operation to create new text.

topP

topP: undefined | number = undefined

Total probability mass of tokens to consider at each step

Settings

Member Visibility

Theme

On This Page

Generated using TypeDoc