fix(scan): derive manifest schema/spec from table metadata (cf. iceberg-java specsById)

[raghav-reglobe]

What

When reading a manifest, iceberg-rust parses the table schema and partition spec from that manifest's own schema / partition-spec Avro key-value metadata and uses them to decode entries — hard-failing if schema is not a valid Iceberg schema. This PR derives the schema + spec from the authoritative table metadata (by the manifest's schema-id / partition-spec-id) instead, falling back to the manifest's own keys when no table metadata is available.

Why

1. Redundant + brittle. The scan already holds TableMetadata (ObjectCache::get_manifest_list takes it; schema_by_id/partition_spec_by_id exist). A manifest's embedded schema is a redundant copy.
2. Ecosystem alignment. iceberg-java's ManifestReader takes specsById from table metadata and has deprecated reading the schema from manifest file metadata — the warning is literally "Pass specsById to avoid reading from file metadata" (removed in 1.12.0). pyiceberg and iceberg-go don't read it on the scan path either. iceberg-rust is currently the only implementation that hard-depends on it.
3. Observable impact. Tables whose manifest schema key holds a non-conformant value are readable by pyiceberg, Doris, and Spark (iceberg-java) but not iceberg-rust. For example duckdb-iceberg serializes the manifest_entry Avro schema there (Avro type names like array/record), producing data did not match any variant of untagged enum SchemaEnum.

How

ObjectCache::get_manifest now takes &TableMetadataRef and threads it through ManifestFile::load_manifest_with → Manifest::try_from_avro_bytes_with → ManifestMetadata::parse_with, which prefers table_metadata.{schema_by_id, partition_spec_by_id}. The existing public parse / parse_avro / load_manifest / try_from_avro_bytes are preserved (they delegate with None), so behaviour is unchanged when no table metadata is available.

Tests

• New test_manifest_metadata_parse_prefers_table_metadata_over_bad_schema: a manifest with a non-conformant schema key parses successfully via table metadata, while the manifest-only path rejects it.
• Full lib suite green (1359 passed), clippy + fmt clean.

Closes #2682.

[roberto-trani]

This None default is what the write path still hits: SnapshotProducer loads existing manifests through plain load_manifest() (transaction/snapshot.rs), so a fast_append commit to a table with a DuckDB written delete manifest still fails with the SchemaEnum parse error.
Verified on this branch against a real S3 Tables table: reads are fixed but commits still break.

Could the transaction path be plumbed through load_manifest_with too?

[raghav-reglobe]

Good catch — confirmed. validate_duplicate_files was still loading existing manifests through the self-describing path, so the duplicate check on append hit the same parse error the scan path used to. Pushed a commit routing it through load_manifest_with (table metadata preferred, same as the scan path), plus a regression test that corrupts a written manifest's schema user metadata in place and asserts fast_append with the duplicate check enabled succeeds.

That was the only remaining production site loading existing manifests — the other load_manifest() callers are tests reading manifests they just wrote. If your S3 Tables repro still breaks anywhere after this, I'd be interested in the error.