Images

Send text and images in the same prompt. Choose a model with image input; libfx does not send images to a separate fallback model.

Send an image

In Node.js, read the image and encode it as base64. Set IMAGE_MODEL to the ID of an image-capable model from your model catalog, then run the example below:

import { readFile } from 'node:fs/promises'
import { createFxAgent } from 'libfx'

const agent = await createFxAgent({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  model: process.env.IMAGE_MODEL,
})
try {
  const data = await readFile('screenshot.png')
  const turn = agent.prompt([
    { type: 'text', text: 'Describe this screenshot.' },
    { type: 'image', data: data.toString('base64'), mimeType: 'image/png' },
  ])
  for await (const event of turn) {
    if (event.type === 'text_delta') process.stdout.write(event.delta)
  }
  await turn.result
} finally {
  await agent.close()
}

Use your own screenshot.png. In the browser, pass a File from your file input directly:

const turn = agent.prompt([
  { type: 'text', text: 'Describe this screenshot.' },
  { type: 'image', data: fileInput.files[0] },
])

Blob values also work. Their type must identify a supported image format. Reading a Blob is asynchronous; read failures reject turn.result. Cancelling or closing the agent during the read prevents the prompt from being sent.

Supported images

PNG, JPEG, GIF, and WebP are supported. The file contents must match mimeType or the Blob's type. Each prompt accepts up to 8 images, with at most 5 MiB of base64 data per image and 8 MiB total. Blob images use the same limits after encoding.

PNGs over 2,000 pixels per side shrink before they reach the model. The original image stays unchanged.

Images stay in conversation history and checkpoints, within the 4 MiB checkpoint limit.