This is a demonstration, not a feature you can turn on. It ran on a specific demo device with software staged for the occasion. It isn’t part of a standard AcuSight deployment, and there’s no setting in the product to enable it. Read this as a look at what’s possible, not as a guide to follow.
Watch it work
Two parts: cameras describing a live work cell, then the Co-Pilot answering questions about the same site.
Three cameras run side by side. Two are doing ordinary inspection work — pill bottles graded pass or fail, cable glands boxed green and red. The third is an overhead camera watching an operator at a bench, and it’s the one doing something new: as they work, a running commentary builds up in the events list beside the video.
Everything lands in the same place. A written description and a defect detection sit in one list, on one timeline, filtered the same way. Nothing separate to go and check.
How it works
Writing a good description is expensive. The model that does it — a vision-language model, or VLM — is too slow to run on every frame of video, and there’d be little point in trying, since most frames look just like the one before. This demonstration used one called LLaVA-OneVision. So two faster models run in front of it and decide when it’s worth the effort. First — is anyone there? A lightweight object detector — the same everyday “label and a box” technique the rest of the platform uses, here a YOLO model — checks whether the operator is at the bench. An empty seat needs no commentary, so nothing downstream runs until someone’s present. Then — is anything worth describing? With an operator present, CLIP acts as a lookout. You give it a short list of things to watch for, in plain words — phone, PPE — and it flags any frame that matches. CLIP writes nothing itself; it’s an image-and-text matcher, turning both the picture and your words into the same form and measuring how closely they line up. That’s all it needs to answer “does this look like one of these?” — quickly, and without being trained on those specific things. Finally — what’s happening? When CLIP flags a frame, it wakes LLaVA-OneVision, which takes a proper look and writes the detail. Each model is a cheaper filter in front of a costlier one — a quick presence check, then a quick lookout, then the expert, called over only when there’s genuinely something to describe. Pointing the expert at every frame would be enormously wasteful, and no more useful.There’s no separate chatbot or language model involved in producing these descriptions. LLaVA-OneVision looks at the image and writes the words itself, in one step.
Why it’s useful
An ordinary detector can only find what it was trained on. Adding a new thing to look for means collecting images, labeling them, and training a model — worth doing for the defects that matter to your line, and a lot of effort for a passing question. This approach is looser and quicker to point at something new. The lookout’s list is just words, so asking it to watch for something else is a change of wording rather than a retraining cycle. And because the descriptions come out as ordinary events, everything the platform already does with events works on them too — they show up live, sit on the playback timeline so you can jump to the moment something happened, and roll up onto dashboards. It’s a tool for understanding what happened and finding the root cause of a problem — not for making split-second calls. Pass-or-fail decisions on a moving line stay with a trained detector, which answers in an instant. The descriptions are there for a person to read afterwards, or for the Co-Pilot to answer questions about — the second half of the video.Next steps
Ask the Co-Pilot
The second half of the video — asking questions about your site in plain English.
Recording & livestream
The Live and Playback views the demonstration runs in.
What is AcuSight?
The platform this runs on, start to finish.
Glossary
Plain definitions for the terms in these docs.