A list of multimodal inputs to be vectorized.

A single input in the list is a dictionary containing a single key `"content"`, whose value represents a sequence of text, images, and videos.

The value of "content" is a list of dictionaries, each representing a single piece of text or image. The dictionaries have four possible keys:
1. type: Specifies the type of the piece of the content. Allowed values are text, image_url, image_base64, video_url, or video_base64.
2. text: Only present when type is text. The value should be a text string.
3. image_base64: Only present when type is image_base64. The value should be a Base64-encoded image in the data URL format data:[<mediatype>];base64,<data>. Currently supported mediatypes are: image/png, image/jpeg, image/webp, and image/gif.
4. image_url: Only present when type is image_url. The value should be a URL linking to the image. We support PNG, JPEG, WEBP, and GIF images. The following constraints apply to the URL:
  - Limit the number of redirects.
  - Require that responses include a content-length header.
  - Respect robots.txt to prevent unauthorized scraping.
5. video_base64: Only present when type is video_base64. The value should be a Base64-encoded video in the data URL format data:[<mediatype>];base64,<data>. Currently supported mediatypes are: video/mp4.
6. video_url: Only present when type is video_url. The value should be a URL linking to the video. We support MP4 videos. The following constraints apply to the URL:
  - Limit the number of redirects.
  - Require that responses include a content-length header.
  - Respect robots.txt to prevent unauthorized scraping.

Note: Only one of the keys, base64 or url, should be present in each dictionary for image and video data. Consistency is required within a request, meaning each request should use either image_base64/video_base64 or image_url/video_url exclusively, not both.

Example payload where inputs contains an image as a URL

The inputs list contains a single input, which consists of a piece of text and an image (which is provided via a URL).


      {
        "inputs": [
          {   
            "content": [
              {   
                "type": "text",
                "text": "This is a banana."
              },
              {   
                "type": "image_url",
                "image_url": "https://raw.githubusercontent.com/voyage-ai/voyage-multimodal-3/refs/heads/main/images/banana.jpg"
              },
              {
                "type": "video_url",
                "video_url": "https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/360/Big_Buck_Bunny_360_10s_1MB.mp4"
              }
            ]   
          }   
        ],  
        "model": "voyage-multimodal-3.5"
      }

Example payload where inputs contains a Base64 image

Below is an equivalent example to the one above where the image content is a Base64 image instead of a URL. (Base64 images can be lengthy, so the example only shows a shortened version.)

  
      {
        "inputs": [
          {   
            "content": [
              {   
                "type": "text",
                "text": "This is a banana."
              },
              {   
                "type": "image_base64",
                "image_base64": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAA..."
              },
              {
                "type": "video_base64",
                "video_base64": "data:video/mp4;base64,AAAAIGZ0eXBpc29tAAACA..."
              }  
            ]   
          }   
        ],  
        "model": "voyage-multimodal-3.5"
      }

The following constraints apply to the inputs list:

The list must not contain more than 1,000 inputs.
Each image must not contain more than 16 million pixels or be larger than 20 MB in size.
Each video must not be larger than 20 MB in size.
With every 560 pixels of an image and every 1120 pixels of a video being counted as a token, each input in the list must not exceed 32,000 tokens, and the total number of tokens across all inputs must not exceed 320,000.

200Success

4XXClient error This indicates an issue with the request format or frequency. Please see our Error Codes guide.