What is the way to tell stable diffusion to output deterministic results?

[StudenteChamp2]

I am using build 5a34bc7. Here the command that i use:
sd-cli.exe --backend vulkan0 --diffusion-model flux-2-klein-4b-Q8_0.gguf --llm Qwen3-4B-UD-Q4_K_XL.gguf --vae flux2.full_encoder_small_decoder.safetensors -p "turn the image into a high quality photograph" -r input_image.png -o output_photo.png --cfg-scale 2 --steps 4 --offload-to-cpu --fa --seed 42

Let's take this example

I get this result:

With a change of the camera position/orientation:

I get this:

As you can see the scene changed. For instance the color of the entities circled in green changed! And objects in red in first render simply vanished.

I MADE another test. This time telling stable diffusion what to do.
Here the test scene:

The prompt:
Turn the image into a high quality photograph. Apply a white porcelain material on the teapot! Use white material for the water pitcher. Use light wood texture for the table. Use dark wood texture for the floor. Use a red metallic material for the light shades. Use a white porcelain material on the cups. Use a beach painting for the wall canvas. Use white material on the seats. Use light wood texture on the seat supports. Do not turn on the lights.

The negative prompt:
Lamp lights.

First camera output:

Everything looks as expected!

Second camera output:

I get almost the same scene. But the color of the areas circled in green changed :(

Third camera output:

Everything looks fine expect that the right shade color specified in the prompt is not respected.

My guess is that to get deterministic results i need to tell stable diffusion what it needs to do for the part of the scenes!
Tell me if i am wrong :)

For instance by adding " The light shade interiors must remain red" to the prompt the third test passes:

Here the second test input images:|

[wbruna]

First, what you want is 'consistent', not 'deterministic' outputs. (I'm not being pedantic: searching the web for 'deterministic' will direct you to different things, like seeds and samplers).

As you noticed, if you leave the model free to guess, it will; and if it has no context to work with, each guess will be different. One possibility is specifying elements using additional reference images, including several different views, so the model doesn't have to guess. A quick example (Klein 4b isn't great at this, and I don't have time to look up the exact syntax it prefers, but hopefully you get the idea):

A concept character sheet showing Hanna, a redhead female, wearing glasses, a wristwatch, a grey blouse and dark blue pants.	Change the first image, placing Hanna, the female from the second image, sitting at the table.

Notice that, even if it didn't generate a terribly consistent reference in the first place, details like the outfit, glasses and wristwatch were kept. So, in your case, you could render elements like the light shade first, then compose the final scene passing it as an additional reference.

Passing a first render as an additional reference for the second one may also work, but it may demand a tighter prompt to keep the model from getting confused between the references.