Concept demo · made for RYZA
Q, with a co-pilot in every room.
Q put several people’s screens, voices and remote control in one room in 2020. The missing piece was someone in the room who could hear, read the screens and help. This is that piece, built into Q’s own dashboard and group room: captions in each person’s language, the right screen on its own, answers about any area of a shared screen, app-level remote control, commands, and notes nobody has to write. Press play for a narrated run of about nine minutes, or use it by hand.
What the co-pilot adds
Everything Q did, and someone helping.
01
Live captions, in each person’s language
Every voice captioned with the speaker’s name, and each reader picks English, Spanish, French or Urdu. The original stays underneath a translation.
02
Smart screen routing
Who is talking, which window just changed and what was said decide which share goes to the main view. Sure enough, it switches; not sure, it suggests and the host decides.
03
Ask about any area of a shared screen
Drag a box around a stack trace, a keyframe or a chart and ask. Only the boxed pixels go to the model, with tokens and keys blurred first.
04
A co-pilot that shows its working
Vision boxes with a confidence on every share it may look at, an attention score on every stream, each answer’s steps and sources, and heads-ups it finds across meetings, like a decision that clashes with what the client asked for last week.
05
App-level remote control
As Q’s desktop app did it: one window is shared and controlled, never the desktop. Clicks are scaled into that window on the other machine, held while another app is in front, and stopped with Esc twice.
06
The co-pilot’s fix, typed by the app
The suggestion is applied through the same channel as real input events, each key shown as it goes and logged, then control goes straight back.
07
Voice and chat commands
“Switch focus to Alex”, “translate captions to Urdu”, /ask, /summarize. Plain words become one structured action; anything disruptive waits for the host.
08
Summary, recap and memory
A rolling summary with decisions and owners, a catch-up for late joiners, a recap per person in their language, and six months of sessions you can search.
Q in 2020, and with the co-pilot
| Q had | With Ryza |
|---|---|
| Multi-screen sharing | The share that matters moves to the main view by itself |
| Remote control of a shared window | Scoped to one app, held when it is not in front, timed grants, the co-pilot’s fix typed through it |
| Voice channels | Live captions and translation in four languages |
| Group chat | /ask, /summarize and /translate in the thread |
| — | Questions about a boxed area of any screen |
| — | Rolling summary, recap and six months of searchable sessions |
Where it lands first
- Newsrooms. Reporters in the field and producers in the studio: captions, the right feed on air, clip notes written for you.
- Education. Group projects and labs: captions for access, and a shy student can box a diagram and ask.
- Customer support. Tier two sees the customer’s screen, boxes the error and gets the likely cause before asking a question.
- Healthcare. Consults across several devices; the summary becomes the visit note, under the clinic’s own retention rules.
How it would be built
On Q’s foundation, not beside it.
Our founder, Ali, was lead developer on Q at RYZA in 2020 and 2021, so the co-pilot is designed around the way Q already moves screens, voice and control, rather than bolted on as a separate bot.
Media
WebRTC through an SFU with TURN, so four or more screens and voices hold up on real networks; Q’s data-channel protocol for remote control.
Speech
Streaming speech-to-text with speaker labels per track, then translation per reader, so nobody hears a dubbed voice.
Understanding
A model with vision for boxed areas, the transcript for /ask, and JSON actions for commands that the room checks before running.
Memory
Recaps and transcripts indexed per group, kept for the period the customer sets, and searchable by topic, person or decision.
Privacy
Per-share consent, secret redaction before any snapshot, no face analysis, a no-training agreement with the model provider, and a log of every co-pilot action.
Clients
Web first, with the desktop capture apps for Mac and Windows that Q already had, so remote control keeps working outside the browser.
Simulated here
- /The people, the session, the screens and the captions are invented, and the co-pilot’s answers are written in advance so the run is the same every time.
- /The command bar understands the phrases it suggests; the product sends what you say to a model and gets one action back.
- /Nothing is sent anywhere. The room runs in your browser only.
A first phase
- /Captions, the rolling summary and the recap on Q’s current rooms, with one design partner using it every week.
- /Then screen questions and smart routing, behind per-share consent and redaction, measured against what the partner’s team actually asks.
- /Then commands and meeting memory, once the first two are trusted.