Do AI-Restored Photos Change Too Much? An Honest Look

"Remini made my grandma look like a stranger." It's the canonical complaint on r/photorestoration, posted every week with a side-by-side of a real, wonderful face and a smoothed-out approximation that shares almost no actual features with the person. The AI isn't broken. It is doing exactly what it was trained to do.

Most AI restoration apps — Remini, MyHeritage, VanceAI, and dozens of GFPGAN-based indie tools — are built on open-source face-restoration models that learned from millions of modern high-resolution faces. Hand them a blurry 1960s portrait and they pull features toward "average young adult." That is why your great-grandfather emerges looking 40 years younger, with symmetric features he never had, and skin like polished ceramic.

Newer frontier image models reduce that drift — significantly, in our testing — but they do not eliminate it. Any AI restoration can change a face too much. The rest of this post is about knowing when that has happened and what to do about it.

Why Some Tools Are Worse Than Others

The difference between a faithful restoration and a stranger-in-grandma's-dress is mostly a generational one. Older open-source stacks — the ones powering most restoration apps you have heard of — chain several narrow models together. A denoiser, a face enhancer, a colorizer, an upscaler. Each step was trained on a comparatively small slice of data, and each adds its own drift. By the time the photo comes out the other end, the errors have compounded.

Newer frontier image models were trained on enormous, varied datasets and do the restoration in fewer, smarter passes. They hold identity noticeably better. Our tool uses this newer generation — Seedream for most passes and Nano Banana 2 Ultra for the highest-fidelity work — which is why side-by-side comparisons with older apps usually favor ours. That does not mean no drift ever. It means less drift, and the QA checklist below still applies.

The Three Ways AI Restoration Goes Wrong

Over-restoration isn't one problem — it is three distinct failure modes, each with a different cause. Learn to spot them and you can usually catch them before sharing.

Failure 1: Hallucinated features

Generative restoration fills in what it thinks should be there based on surrounding context. If the mouth is low-resolution, the AI might invent teeth in a closed-mouth smile. If one eye is partially shadowed, the model may paint in an eye color the person never had. This is the mode most likely to make grandma unrecognizable, and it is the signature failure of older pipeline tools that stack a face enhancer on top of a coarse upscaler.

What to do: run the restoration a second time — most tools are non-deterministic and you will get a noticeably different result. Compare both against other photos of the same person and keep the one that holds identity best. If the tool offers a lower-intensity or "faithful" mode, start there before pushing harder.

Failure 2: Plastic skin

Real human skin has pores, fine lines, asymmetry, and micro-texture. Over-aggressive face restoration erases all of it. The result looks like a wax figure or a CGI character — uncanny and oddly sterile. This is particularly jarring on elderly subjects whose wrinkles are part of their identity.

What to do: plastic skin is usually the denoiser running too hot — a problem baked into many older restoration pipelines. Look for a tool that separates damage repair from face enhancement so you can use the former without the latter. On our restorer, a lower-intensity pass usually keeps the texture intact.

Failure 3: Inaccurate colorization

Colorization is a plausibility guess, not a historical record. Green is a common guess for unknown fabrics. Military uniforms get generically colorized without awareness of service branch or era. Wedding dresses come out white even when the bride wore mauve — because white dresses dominate the training data. Every AI colorizer does this, and no frontier model has solved it because the information simply is not in the photograph.

What to do: if accurate colors matter, ask an elderly relative before you colorize. A restored black-and-white is more honest than a confidently wrong colorization. When family memory is gone, consider leaving the image monochrome.

What Conservators Actually Do

The American Institute for Conservation has an ethics code with a core principle called minimal intervention. A good restoration should be distinguishable from the original on close inspection. The intent is to stabilize the artifact and recover lost detail, not invent a new one.

Applied to AI: that means removing scratches and color shifts and fading — the clearly-damaged parts — while keeping facial features, skin texture, pose, and expression exactly as they were. That is the target a good restorer aims at. It is also the bar by which you should judge any tool's output.

The "Would Your Family Recognize Them" Checklist

Before you share a restored photo at a funeral, family reunion, or print it large, run this mental check against any other photos of the same person you have:

When to Stay Conservative vs. Go Further

Frequently asked questions