A Practical Test Of Jev’s AI Vision On 30 Images

🔍 Read the full analysis: A Practical Test Of Jev’s AI Vision On 30 Images on ThorstenMeyerAI.com

Buying for a business?Offer from Amazon

Get business pricing on tech for your team

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

Thorsten Meyer tested TypeSafe’s Jev 1.14 on 30 featured images from his website after the company added image inputs and a bounding-box response type, reporting strong headline matching for product photos but no complete results in the source account for news and essay images. The test offers an early, small-scale look at a capability TypeSafe says is not yet ready for production use and keeps behind explicit opt-in headers.

Meyer, writing on ThorstenMeyerAI.com on October 15, says he used the 30 most recent featured images on the site: 10 product roundups, 15 news posts and five essays. Each image was paired with three yes-or-no questions: whether it fit its own headline, a randomly selected other headline, and an unrelated headline about repotting a monstera plant. The product images also received a multiple-choice question asking which of 10 products each image showed. Meyer ran the tests on jev-1.14.0 and jev-1.13.0, for about 60 requests in total.

For the 10 product images, Meyer reports that Jev 1.14 gave each image’s own headline a fit score of 0.65 to 0.89, with a median of 0.76. Random alternative headlines scored no higher than 0.18, with a median of 0.06; the unrelated plant headline scored no higher than 0.10. Jev 1.14 identified the pictured product correctly in nine of 10 cases. Its error involved a photo showing a router alongside a cylindrical mesh unit: the model chose “Wi-Fi 7 router,” with confidence of 0.60.

Meyer says all correct product identifications from Jev 1.14 had confidence scores of at least 0.77, while Jev 1.13 identified all 10 products but gave two answers confidence below 0.5. That suggests confidence could help flag uncertain classifications in this sample, but it does not establish how well such a threshold would work on other image sets. The supplied account cuts off as Meyer begins describing the news-image results; it provides no final findings for those 15 images or the five essays.

What the Product Image Scores Show

The test addresses a practical editorial task: checking whether a featured image matches the headline it accompanies. Meyer’s product results suggest Jev can distinguish relevant product photos from deliberately mismatched headlines in a small, controlled sample. If that performance holds in broader evaluations, a decision model that returns a score or category could help flag images for human review rather than requiring every pairing to be checked manually.

The limits matter as much as the reported scores. This was one site’s 30 images, and the source provides detailed results only for the 10 product photos. Meyer’s headline-fit questions also test matching against a specific set of alternatives, not image understanding in every editorial setting. The report should be read as an early practical test, not a general benchmark or proof of production reliability.

Jev Adds Opt-In Image Inputs

Before this release, Jev was presented as a decision model: a caller supplies state and typed questions, and the model returns answers such as yes-or-no probabilities, choices from a list, or scores. TypeSafe published Jev 1.14 on October 14, 2026, adding image support and a new `box` type that returns one bounding box for a described target, along with confidence. The company’s post says image support also works on Jev 1.13, which it described as multimodal already.

Image input is enabled with the `TypeSafe-Danger-Zone-Images: true` header. TypeSafe also made longer context available behind a separate opt-in header, raising limits from 64,000 tokens per request and 32,000 per question to 262,000 for each. The company warns that reliability falls on long inputs. Jev 1.14 is available as `jev-preview`; TypeSafe said `jev-latest` would move to it about a week after release, around October 21.

TypeSafe has cautioned that the new capabilities remain rough. Its release post said there was “still way too much jaggedness” for the company to vouch for production use cases, which is why the features require explicit headers. The company also shared internal intelligence estimates rather than public benchmark scores and advised users to run their own evaluations.

“There is still way too much jaggedness for us to vouch for these for production use cases.”

— TypeSafe, in its Jev 1.14 release post, as quoted by Thorsten Meyer

News and Essay Results Are Missing

The source account ends while Meyer is introducing the news-image results. It does not report the fit scores or outcomes for the 15 news images, the five essays, or a complete comparison between Jev 1.14 and Jev 1.13 on headline matching. It is also unclear whether the reported product results were independently checked or repeated across multiple runs.

The sample is too small and narrow to establish performance across other publishers, image styles, or content topics. TypeSafe’s own warning says it does not yet endorse these features for production. The report provides no independent benchmark or evidence that the scores will remain calibrated on new data.

Further Tests Needed Before Deployment

Meyer’s report identifies no planned follow-up date or additional test set. Readers seeking a fuller account would need the missing results for news and essay images, along with details on repeatability and how the confidence scores behave across larger samples.

For developers, the near-term next step is to evaluate the opt-in image features on their own material and check error rates before relying on them in workflows. TypeSafe said `jev-latest` was expected to move to Jev 1.14 around October 21, 2026; it also outlined deprecation notice periods for model versions, while saying a formal policy was still to come.

Key Questions

What did Jev’s image test cover?

Thorsten Meyer tested 30 featured images from his website: 10 product roundups, 15 news posts and five essays. The reported results in the supplied account cover the product images.

How did Jev 1.14 perform on product images?

Meyer reports that product-image fit scores were 0.65 to 0.89 for each image’s own headline, and Jev 1.14 identified nine of 10 pictured products correctly. These are results from a small site-specific test, not a general benchmark.

Did Jev 1.13 also identify the products?

According to Meyer, Jev 1.13 identified all 10 products, though two answers had confidence scores below 0.5. Jev 1.14 got nine correct, with all of its correct answers at 0.77 confidence or higher.

Is TypeSafe recommending Jev image features for production?

No. TypeSafe said the new capabilities still have too much “jaggedness” for it to vouch for production use and requires users to enable image support with an explicit opt-in header.

What is not known from the report?

The supplied source cuts off before giving the results for the 15 news images and five essays. It also does not establish how the scores would generalize to other publishers or larger image collections.

Source: ThorstenMeyerAI.com