Wowza Video Intelligence Framework

Real-time AIon live video.

Add object detection, scene understanding and vision-language analysis to live streams in Wowza Streaming Engine. Run inference on your GPUs and send structured results to the systems you already operate.

Request a demoStart building
10 msInference latency starts at
80Common object classes built in
0Cloud dependency required
Trucks lined up at a container port, each one marked and labelled as a truck by the detection modeltruck 0.93truck 0.94truck 0.92truck 0.95truck 0.91truck 0.97camera-01 · LIVE

Illustrative detection view on a stock photo; boxes from RF-DETR, scores are examples.

What you can build

Turn frames into signals your software can use.

Select an analysis mode

Find people, vehicles, equipment or domain-specific classes. Receive confidence scores, structured data and bounding boxes.

Request a demoRead the docs

01 / 04 · OBJECT DETECTION

City intersection from above with detection boxes around cars and busesbusbuscarcarcarcarcarcarcarcam-042 · LIVE
  • Bus detectedobject detection
  • Car detectedobject detection
  • Custom eventyour own model

People, vehicles, equipment or your own classes

02 / 04 · SCENE UNDERSTANDING

City interchange from above at night
Traffic is flowing on all routes
A lane is blocked–
People are on the roadway–

Illustrative view on a stock photo, not a product screenshot.

Log whether conditions are met, in real time

03 / 04 · VISION-LANGUAGE MODEL

City interchange from above at night
YOUR PROMPTIs any vehicle stopped on the shoulder?
MODEL RESPONSENo stopped vehicles. Traffic is moving in all lanes.

Illustrative view on a stock photo, not a product screenshot.

Ask in plain language, get structured answers

04 / 04 · CUSTOM WORKFLOWS

Webhook alertsJSON log filesVideo overlaysID3 timed metadataJava listener APIWowzaSTREAMING ENGINE

One detection can reach every channel at once

Non-blocking by design

AI that never gets in the way of the video.

Frames are sampled out-of-band. If inference becomes unavailable, detection pauses and your streams carry on.

Live video, untouchedHLS / LL-HLS, WebRTC, MPEG-DASH
Control centers, IP streams, CDN
SAMPLED FRAMES

You set the interval

Specify frame sample intervals per stream, and control what you spend on inference.

YOUR GPUS

Inference runs locally

Built-in models or your own, on hardware you control. Frames are not retained by default.

STRUCTURED EVENTS

Your systems react

VMS, SIEM, dispatch, monitoring and dashboards, through APIs and webhooks.

Quickstart

From zero to a verified detection in under a day.

A clear first path. Prove the integration first, then scale the model, stream count and automation around a measurable outcome.

1
DeployUse a containerized or native path to add VIF to your Wowza Streaming Engine environment.
2
Connect a live streamUse a camera, encoder, drone or other live source already feeding into the engine.
3
Enable an analysis modeChoose object, scene, VLM or synthetic video detection, then tune sampling and confidence.
4
Verify and use resultsPull an annotated thumbnail, watch stream health and connect an output to a downstream system.
enable-vif.sh
# Enable object detection on a live stream
curl -u admin:•••••• \
  -X PUT http://localhost:8087/v1/_defaultServer_/plugin/vif/\
  applications/live/streams/camera-01 \
  -H 'Content-Type: application/json' \
  -d '{ "active": true, "skip_frames": 10 }'

✓ Stream camera-01 is active
  detector  object / RF-DETR
  output    webhook + JSONL + overlay
Deployment

Put inference where your video already lives.

Start with the all-in-one container and upgrade to split-server without changing your stream configuration.

Engine + inferenceONE CONTAINER

All-in-one Docker

Ideal for evaluation and lower-volume production.

EngineGPU host

Split server

A dedicated GPU host, for independent scaling of inference.

EngineEngineGPU host

Multi-instance

Multiple engines share inference resources, or the other way round.

EngineGPU host

Air-gapped

Fully on premises. Docker images can be pre-loaded, and all output channels keep working.

Why teams run it themselves

Your video, your models, your budget.

Privacy and control

Raw video stays inside your environment while intelligence, alerts and metadata flow out.

  • Raw video remains on-site
  • Encryption in transit and at rest
  • Built for regulated and air-gapped sites

Your own models

Start with built-in models or deploy your own. Swap them without rebuilding your streaming infrastructure.

  • Custom and ONNX-compatible models
  • Fine-tune for your requirements
  • No proprietary AI lock-in

Predictable AI spend

Avoid per-frame pricing, cloud egress fees and unpredictable inference costs.

  • Run inference on your infrastructure
  • Control sampling and resources
  • Scale without runaway cloud costs
FAQ

Video Intelligence Framework, answered.

What is VIF?

A real-time video intelligence layer for Wowza Streaming Engine. It samples frames from live streams, runs GPU-accelerated inference and returns structured results, without blocking video delivery.

Can it run entirely on premises or air-gapped?

Yes. VIF is fully deployable on premises with zero cloud dependency. All inference runs locally on your hardware, and no video data needs to leave your network.

What AI models does it support?

It ships with RF-DETR, a transformer-based object detection model that recognizes 80 common object classes, with inference latency starting at 10 milliseconds. It also supports custom-trained RF-DETR models, CLIP-based scene matching, VLM integrations and NVIDIA’s Synthetic Video Detector.

If inference goes offline, are my streams affected?

No. VIF is non-blocking by design and operates asynchronously. Detection coverage pauses until the inference service is restored, but ingest, processing and delivery are never impacted.

What happens to video frames after inference?

VIF processes frames in memory and does not retain them by default. The system produces structured event metadata: timestamp, camera ID, detection type, confidence score and bounding box.

How do I manage false positives?

Per-stream controls: confidence thresholds, minimum consecutive frames, tracking persistence and event batching. They can be changed at runtime through the dashboard or the REST API.

Start with one stream.And one useful event.

Talk to an engineerRead the docs