interactive displays

enable displays

enable permission 67, bot.messages.components, in the dashboard, publish the configuration and obtain renewed consent. creating a public display also requires bot.messages.send; a private display requires bot.messages.ephemeral. updates require bot.messages.edit. deletion uses the normal message delete permission and route. displays always use bot authorship. existing administrator approvals cover only keys 2–66; permission_catalogue:67 is added by saving a new draft, and displays become available only after the installer approves it. displays confer no message-reading, account, command or media powers.

send a display

POST /chats/{chat}/messages with actor:bot, a required plain-text content fallback and display. send the same display field to POST /chats/{chat}/ephemeral-messages or POST /interactions/{id}/responses. the receipt has has_display:true. the display is stored with its message, and deleting or expiring the message removes its display and retained image references. content remains the searchable, notification and older-client summary; display updates do not create messages, unread activity or repeated notifications.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159{
  "actor": "bot",
  "content": "music player · night drive",
  "display": {
    "version": 1,
    "state": {
      "title": "night drive",
      "artist": "example artist",
      "track": "track-1",
      "playback": {
        "position": 30,
        "duration": 240,
        "at": 1792526400000,
        "rate": 1
      },
      "queue": [
        "next song",
        "another song"
      ]
    },
    "styles": {
      "heading": {
        "fontSize": 20,
        "fontWeight": "bold"
      },
      "secondary": {
        "fontSize": 14,
        "color": "muted"
      }
    },
    "actions": {
      "play": {
        "type": "bot",
        "audience": "chat_members",
        "context": {
          "bind": "track"
        }
      },
      "pause": {
        "type": "bot",
        "audience": "chat_members",
        "context": {
          "bind": "track"
        }
      },
      "seek": {
        "type": "bot",
        "audience": "chat_members",
        "context": {
          "bind": "track"
        }
      }
    },
    "root": {
      "id": "player",
      "type": "column",
      "children": [
        {
          "id": "header",
          "type": "row",
          "children": [
            {
              "id": "cover-placeholder",
              "type": "icon",
              "props": {
                "name": "music",
                "label": "music"
              }
            },
            {
              "id": "metadata",
              "type": "column",
              "style": {
                "grow": 1,
                "gap": 4
              },
              "children": [
                {
                  "id": "title",
                  "type": "text",
                  "style_ref": "heading",
                  "props": {
                    "text": {
                      "bind": "title"
                    }
                  }
                },
                {
                  "id": "artist",
                  "type": "text",
                  "style_ref": "secondary",
                  "props": {
                    "text": {
                      "bind": "artist"
                    }
                  }
                }
              ]
            }
          ]
        },
        {
          "id": "seek",
          "type": "slider",
          "props": {
            "label": "playback position",
            "timeline": {
              "bind": "playback"
            },
            "action": "seek",
            "step": 1
          }
        },
        {
          "id": "controls",
          "type": "row",
          "children": [
            {
              "id": "play",
              "type": "button",
              "props": {
                "label": "play",
                "icon": "play",
                "action": "play"
              }
            },
            {
              "id": "pause",
              "type": "button",
              "props": {
                "label": "pause",
                "icon": "pause",
                "action": "pause"
              }
            }
          ]
        },
        {
          "id": "queue",
          "type": "details",
          "props": {
            "label": "up next"
          },
          "children": [
            {
              "id": "queue-list",
              "type": "list",
              "props": {
                "items": {
                  "bind": "queue"
                }
              }
            }
          ]
        }
      ]
    }
  }
}

document and state

a version:1 document contains root, state and actions, with optional reusable styles. every node has a unique stable id and type, plus optional props, children, style, narrow_style and style_ref. identifiers contain 1–64 ASCII letters, digits, underscores or hyphens; reserved prototype keys are invalid. state is a flat map of strings, finite numbers, booleans, null, text lists, text tables or timelines. a property written as {bind:key} reads one state entry. bindings must exist and resolve to the property's expected type. bot actions may bind a scalar context such as the current track ID; the server rejects clicks against obsolete contexts. state is visible to everyone who can see the message, so keep credentials and private backend data on your server.

available components

layout components can nest up to eight levels. labels are required for controls, tabs, details, icons and progress indicators. strings are rendered as text; markdown uses the existing safe message renderer. command and link actions are supported on buttons. forms submit only when the user explicitly submits them.

scroll the table horizontally to see all columns.

layout and styling

style_ref selects a reusable entry in styles; inline style overrides it. narrow_style overrides both when the display is narrower than 420 pixels. allowed properties are gap/padding/radius (0–48px), fontSize (12–48px), borderWidth (0–4px), width/maxWidth (24–960px or fill), height (24–480px), grow (0–12), columns (up to six values such as 1fr 2fr or auto), align (start, center, end, stretch), justify (start, center, end, space-between, space-around), wrap (wrap, nowrap), textAlign (left, center, right) and fontWeight (normal, medium, bold). color, background, borderColor and gradientTo accept #rrggbb, #rrggbbaa or theme tokens surface, ink, muted, accent, line and transparent. gradientTo creates a diagonal gradient from background. use theme tokens for readable themes and maintain contrast with custom colors. displays stay within the chat width and preserve platform focus indicators. arbitrary CSS, HTML, JavaScript, remote fonts, iframes, browser storage and direct network fetches are not supported. available icon names are play, pause, skip, previous, plus, music, volume, refresh, check, x, heart and link.

album covers and images

upload images through the existing bot upload flow using actor:bot, purpose:attachment, bot.attachments.send and the same installation and chat. after completion, set the returned asset ID on the image node, directly or through state. PNG, JPEG, WebP, GIF and AVIF are supported, up to 40 million decoded pixels within the normal upload limits. SVG and external image URLs are rejected. retained display images cannot be deleted through the asset API while referenced. display assets are reserved for displays; upload a separate copy to send the image as a normal attachment. image bytes use authenticated audience checks, including private-message expiration and current membership; users' browsers do not fetch arbitrary bot-supplied image URLs.

partial and realtime updates

GET the display snapshot, then PATCH with If-Match quoting its revision and an Idempotency-Key. send either document for full replacement or operations for atomic patches. the server validates the complete result before committing anything. a conflict returns 412; refetch and recompute the change. snapshots contain document, revision, layout_revision, updated_at, server_time, active and installer. revision changes after every write; layout_revision changes only for structural/property changes or replacement. state patches preserve component IDs and local input, expansion, selected tab and focus. update the textual message separately when its searchable summary changes.

scroll the table horizontally to see all columns.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24{
  "operations": [
    {
      "op": "state",
      "values": {
        "title": "next track",
        "track": "track-2",
        "playback": {
          "position": 0,
          "duration": 212,
          "at": 1792526400000,
          "rate": 1
        }
      }
    },
    {
      "op": "props",
      "id": "play",
      "values": {
        "disabled": true
      }
    }
  ]
}

patch operations

state merges values into the state map. props merges values into one node's properties. replace replaces the named node with node; remove deletes a non-root node; insert inserts node into the named parent's children at zero-based index. operations run in order and roll back together on failure. use full document replacement to remove state keys, style definitions, actions or properties. keep bindings valid throughout structural edits and preserve IDs when retaining a control.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39{
  "operations": [
    {
      "op": "state",
      "values": {
        "queue": [
          "another song"
        ]
      }
    },
    {
      "op": "insert",
      "id": "player",
      "index": 0,
      "node": {
        "id": "notice",
        "type": "badge",
        "props": {
          "text": "live"
        }
      }
    },
    {
      "op": "replace",
      "id": "notice",
      "node": {
        "id": "notice",
        "type": "text",
        "props": {
          "text": "updated"
        }
      }
    },
    {
      "op": "remove",
      "id": "notice"
    }
  ]
}

realtime clocks and external data

a timeline is {position,duration,at,rate}. position and duration are seconds; at is a Unix timestamp in milliseconds; rate is 0–4, with zero meaning paused. the client interpolates and clamps the position locally from the server clock. update on play, pause, seek, track change or occasional drift correction, not every animation frame. bots consume their own external API, webhook or stream and submit state patches to Sometimes. the browser never runs bot code. updates are coalesced and sent to current viewers; missed or out-of-order revisions trigger a snapshot fetch. reconnect, visibility return and periodic reconciliation refresh access and state. hidden/offscreen displays stop their timers and refresh when visible.

clicks and bot responses

actions map IDs to {type:bot,audience:installer|chat_members,context?}. a button or input refers to its action ID in props.action. chat_members permits current eligible human members to interact with this bot-owned display; installer restricts the control to the installing user. private bots still require the clicking member to be the owner or invited. the existing interaction.created gateway event has kind:component, actor_id for the actual clicker, installer_id for the installation owner, message_id, component_id, action_id, layout_revision, context and options:{value}. command_id is null. acknowledge within three seconds through POST /interactions/{id}/ack, then update the display or send ordinary interaction responses within 15 minutes. public ACKs/replies require bot.messages.send; private ones require bot.messages.ephemeral. commands is not required for component actions. an ephemeral response goes only to the clicker; its audience is interaction when that person differs from the installer. component clicks never grant access to that person's account, chat history or delegated user actions. bots must authorize the operation represented by each action and treat actor_id as the requesting identity. replay and responses recheck current eligibility, blocks, installation access and grants.

insert commands and open links

a command action is {type:command,audience,command_id} and must refer to the installation's registered command snapshot. clicking inserts that bot's command into the clicking user's composer without submitting it. their existing draft is saved and can be restored; it returns after successful command submission. the user needs their own eligible installation with commands enabled. input arguments are completed and sent through the normal private command flow. a link action is {type:link,audience,url} with an HTTPS URL without embedded credentials; the link opens only after a click. controls cannot silently navigate, send chat messages or submit commands.

limits, lifecycle and fallback

documents are at most 32 KiB of UTF-8 JSON with 128 nodes, depth 8, 32 actions and 32 patch operations per request. text values are at most 4,000 characters; labels/alt/placeholder are at most 200; lists/tables have at most 100 rows, 12 table columns and 500 characters per cell. updates are limited to five per second per installation/chat, in addition to normal REST limits. member interactions allow ten per five seconds per member/application. unknown fields, invalid types, unsupported versions and unresolved bindings reject the whole request. paused, revoked, excluded, disabled or inaccessible displays lose live controls. public historical layouts remain readable to eligible chat members; private displays disappear when access ends or they expire. fallback content is shown until a valid layout loads. controls expose keyboard labels, loading/errors and retry without losing draft input. data bound into a display is treated as message content by the normal text filter.

Sometimes client protocol

these routes use a verified human session, not a bot credential. every request enforces membership and the display audience. POST interactions accepts a unique id, component_id, action_id, layout_revision, context and value; reusing an ID with changed input fails. bot_display_changed on the existing human chat socket contains conversation_id/id and either an invalidation or from_revision, revision, layout_revision, operations, updated_at and server_time. clients apply only contiguous revisions and refetch otherwise; these frames never include hidden private state.

scroll the table horizontally to see all columns.

run the music player example

download music-player.mjs and music-display.json into one directory and run with Node.js 24+. configure commands, bot.messages.components, bot.messages.send, bot.messages.edit and bot.messages.ephemeral, register a queue command with one required string option named song, then publish and install. set SOMETIMES_BOT_TOKEN, SOMETIMES_INSTALLATION, SOMETIMES_CHAT and SOMETIMES_QUEUE_COMMAND. for artwork, also authorize bot.attachments.send, then optionally set SOMETIMES_COVER_ASSET to a completed image upload for that chat. run node music-player.mjs. the example creates one player, handles play/pause/seek, queues songs through the command button, persists its message/cursor and reconnects. it models playback state; connect its action handler to your audio service or the existing bot call audio API to actually play music. display permission alone does not stream audio or bypass audio consent.