Images

Send text and images in the same prompt. Choose a model with image input; libfx does not send images to a separate fallback model.

Send an image

In Node.js, pass the image bytes directly. Set IMAGE_MODEL to the ID of an image-capable model from your model catalog, then run the example below:

import { readFile } from 'node:fs/promises'
import { createFxAgent } from 'libfx'

const agent = await createFxAgent({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  model: process.env.IMAGE_MODEL,
})
try {
  const data = await readFile('screenshot.png')
  const turn = agent.prompt([
    { type: 'text', text: 'Describe this screenshot.' },
    { type: 'image', data, mimeType: 'image/png' },
  ])
  for await (const event of turn) {
    if (event.type === 'text_delta') process.stdout.write(event.delta)
  }
  await turn.result
} finally {
  await agent.close()
}

Use your own screenshot.png. In the browser, pass a File from your file input directly:

const turn = agent.prompt([
  { type: 'text', text: 'Describe this screenshot.' },
  { type: 'image', data: fileInput.files[0] },
])

Blob values also work. Their type must identify a supported image format. Reading a Blob is asynchronous; read failures reject turn.result. Cancelling or closing the agent during the read prevents the prompt from being sent.

Supported images

PNG, JPEG, GIF, and WebP are supported. The file contents must match mimeType or the Blob's type. Raw bytes, canonical base64, Blob, and File inputs work. Invalid base64 throws from prompt().

Each prompt accepts up to 8 images, with at most 3.75 MiB of raw data per image and 6 MiB total. Text, encoded images, and metadata must fit within an 8 MiB frame. Eligible images reach the model unchanged, up to 8,000 pixels per side; libfx does not resize them automatically.

Prepare images

Pass a resizeImage callback when creating the agent to resize or convert prompt images before sending them. For a Node application that uses sharp:

import sharp from 'sharp'
import { createFxAgent } from 'libfx'

const agent = await createFxAgent({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  model: process.env.IMAGE_MODEL,
  async resizeImage({ bytes }) {
    const bytesOut = await sharp(bytes)
      .resize({ width: 1568, withoutEnlargement: true })
      .png().toBuffer()
    return { bytes: bytesOut, mimeType: 'image/png' }
  },
})

Install sharp in your application to use this callback. In a browser, use your image-processing code and return an ArrayBuffer or typed array. Limits apply to the callback's output. Preparation is asynchronous; failures reject turn.result. The callback runs on prompt images only, not tool results.

Keep an original reference

Add a sourceRef to identify an original stored by your application:

{ type: 'image', data, mimeType: 'image/png', sourceRef: 'uploads:screenshot-1' }

You can also omit data and send the reference with its MIME type. libfx does not fetch the original. If an image cannot be sent, the model receives recovery feedback with the reference so your tools can provide a suitable copy. Your application must authorize access to the original.

Image bytes and references stay in conversation history and checkpoints, within the 4 MiB checkpoint limit. A reference-only block does not save the original file.