English | 中文
JEV is fast, structured, and built to make decisions.
JEV is the first System One Model from TypeSafe AI, built for fast, structured decisions inside software. JEV Sees is an independent open-source project built on JEV and typesafe-sdk; it is not an official TypeSafe AI vision product.
JEV Sees works with images, video, and RGB-D cameras. It finds the objects in view, keeps their IDs consistent across video frames, and asks JEV your question about each relevant object.
| Image · bus color | Video · pedestrian risk | Video · animal motion |
|---|---|---|
 |
 |
 |
| Question: What color is the bus? Answer: blue |
Question: Is this pedestrian currently involved in a car accident? Answer: one probability per pedestrian |
Question: What is this animal's current motion state? Answer: walking |
git clone https://github.com/CharlesFeng0314/JEV_sees.git
cd JEV_sees
python -m pip install -e .JEV Sees requires Python 3.10+. The first vision run downloads the default Florence-2 and CLIP weights if they are not already available locally.
Create a key at console.typesafe.ai/keys.
macOS / Linux:
export TYPESAFE_API_KEY="your-key"PowerShell:
$env:TYPESAFE_API_KEY = "your-key"This is a complete program, not a wrapper around one of the repository examples:
from jev_sees import Sees
result = Sees().for_each("bus").choice(
"assets/bus.jpg",
"What color is this bus?",
choices=["yellow", "red", "white", "blue", "black", "uncertain"],
save="bus_answer.png",
)
for row in result.rows:
print(row["object_id"], row["choice"], row["confidence"])Typical output:
object_001 blue 1.0
The saved image includes the detected bus, its stable object ID, and JEV's answer.
All three examples use the same flow: media + a question + an explicit answer type. Each script calls JEV and prints structured output in the terminal.
- Input:
assets/bus.jpg - Question: “What color is the bus?”
- Answer type:
Choicefrom colors supplied by the application
python examples/bus_color.pyobject_id label choice confidence
object_001 bus blue 1.00
Source: examples/bus_color.py
- Input: the 42–50 second segment of
assets/traffic.mp4 - Question: “Is this object currently involved in a car accident?”
- Answer type: probability for every pedestrian in each sampled frame
python examples/traffic_relations.pyframe_index video_time_s object_id label scope_probability probability
504 42.00 object_001 person 0.88 0.04
510 42.50 object_001 person 0.92 0.04
This asks about what is happening in the current frame; it is not a future-accident forecast. Source: examples/traffic_relations.py. Video attribution: assets/CREDITS.md.
- Input:
assets/black_cat_walking.mp4 - Question: “What is this animal's current motion state?”
- Answer type:
Choicefromstationary,walking,running, anduncertain
python examples/animal_motion.pyframe_index video_time_s object_id label choice confidence
0 0.00 object_001 black cat uncertain 0.60
120 5.00 object_001 black cat walking 0.23
Source: examples/animal_motion.py
JEV Sees does not guess the answer format from the wording of a question. Your code chooses it explicitly:
probability(...)returns an independent value from 0 to 1 for every selected object. It uses JEV's officialNoulquestion type.choice(...)selects from options supplied by your application and returns the selected option, confidence, and probability distribution.score(...)evaluates every selected object against a rubric supplied by your application.
from jev_sees import Sees
sees = Sees()
risk = sees.for_each("pedestrian").probability(
"assets/traffic.mp4",
"Is this pedestrian currently involved in a car accident?",
start=504,
end=600,
)
motion = sees.for_each("animal").choice(
"assets/black_cat_walking.mp4",
"What is this animal's current motion state?",
choices=["stationary", "walking", "running", "uncertain"],
)
occlusion = sees.for_each("package").score(
"warehouse.jpg",
"How obstructed is this package?",
rubric=["0: clear", "1: partly obstructed", "2: heavily obstructed"],
)Call Sees().probability(...), .choice(...), or .score(...) without for_each(...) to ask about every detected object. Choice, Noul, Score, and TypeSafeClient are also re-exported for applications that want to construct official JEV questions directly.
Pass an image path, a PIL image, or an RGB NumPy array. Each result row identifies the object that was judged.
Object IDs remain stable across consecutive sampled frames, so answers can be graphed, thresholded, or used to trigger an action. By default, JEV Sees samples roughly twice per second. Use every=1 for every frame or every=N for every Nth frame.
RGB-D is supported for live camera loops through observe(rgb, depth, intrinsics=...). When camera intrinsics are available, objects can include metric 3D positions and distances. The same objects can then be passed to JEV with ask(...).
result.answers: JEV answers keyed by object IDresult.rows: one summary row per objectresult.frames: answers grouped by sampled video framesave="output.gif": annotated image, GIF, or video with live answersjson="output.json"andexcel="output.xlsx": structured exports
Every selected object in one sampled frame is included in the same JEV request.
Image and video perception run locally. The default detector/captioner is microsoft/Florence-2-base-ft; select another compatible checkpoint with:
Sees(florence_model="microsoft/Florence-2-large-ft")If JEV_SEES_ROBO_ROOT points to a project containing weights/clip/ViT-B-32.pt, JEV Sees uses that local CLIP weight. Otherwise it downloads the weight on first use. Model weights are not committed to this repository.
Run the test suite with:
python -m unittest discover -s tests -vIf import jev_sees fails after installation, make sure python -m pip install -e . and your program use the same Python interpreter.
JEV Sees began as the visual layer of JEV Control Your Roboarm and is now a standalone project for images, video, cameras, and robotics.
The project is currently v0.1.0 and experimental. The public API, perception stack, and examples may continue to change.
MIT