UpDish

Overview

UpDish analyzes a photo of a meal, identifies the visible food components, estimates their visual proportions, lets the user review or correct the result, and turns the final components into a plate evaluation. Evaluations are stored locally so previous meals can be searched and revisited.

Problem

A generative vision model can produce an answer that sounds more certain than the image allows. The real challenge was not simply getting a model response; it was deciding what the model should be allowed to infer and how the application should behave when the result is incomplete or wrong.

Constraining model output

The meal-detection service requests a structured JSON response containing a meal name and a list of visible components with integer portion percentages.

The prompt explicitly tells the model not to infer hidden ingredients, weight, calories, or food volume that is not visible. It also asks the component proportions to represent one complete meal and total 100%. That turns prompting into part of the interface contract rather than leaving the rest of the app to parse arbitrary prose.

Treating AI output as a draft

The detected components are not treated as unquestionable truth. After analysis, the user can review and modify the result before the app continues to the final evaluation. This keeps the model useful without making the entire experience dependent on a prediction the user cannot correct.

Designing the failure path

The analysis flow handles cancellation, missing connectivity, dropped connections, empty model responses, invalid structured output, and cases where no food can be identified. These states are surfaced as product states instead of being reduced to one generic error.

Keeping useful history on device

UpDish uses SwiftData to persist previous meal evaluations and provides search across the saved history. That makes the model call one part of a longer native workflow rather than the entire product.

Engineering Takeaways

UpDish taught me that integrating an AI model is mostly a systems and interface problem: constrain what the model can say, validate what comes back, give users a correction path, and design failure states as seriously as the happy path.