refactor(mason): keep custom Python tools code-first

[junchoi-db]

What

• Keep agent.toml as the declarative source of truth for Databricks-managed infrastructure: sandbox, managed MCP, Unity Catalog functions, memory, session, and durability bindings.
• Keep custom Python tools framework-native and code-first. Users write decorated code directly; Mason no longer scaffolds it with mason tools add python or records Python entrypoints in agent.toml.
• Preserve customer-authored MCP servers as code and combine them with managed agent.toml bindings through the existing LangGraph and OpenAI adapters.
• Remove the obsolete Python scaffold assets and reject legacy Python manifest rows with explicit migration guidance.

CUJ

mason init my-agent
cd my-agent

# Databricks-managed integrations: CLI CRUDs agent.toml.
mason tools add sandbox --scope table:samples.nyctaxi.trips
mason tools add mcp system.ai.web_search
mason tools add uc-function catalog.schema.lookup_ticket
mason tools list

# Custom Python: write framework-native code in agent/tools/.
mason dev
mason deploy my-agent

LangGraph tools use @tool; OpenAI Agents tools use @function_tool. Both Mason templates auto-discover decorated modules under agent/tools/, so there is no second registration step. mason tools list intentionally reports only managed bindings from agent.toml.

An older manifest row such as source = { kind = "python", ... } now fails with guidance to remove the row; the decorated source file remains active.

Architecture boundary

Customer Python tools and customer MCP servers -> customer/framework code
Managed sandbox, MCP, and UC functions         -> agent.toml
Memory, session, and durability resources      -> agent.toml
Dependencies                                   -> pyproject.toml
Framework/template provenance                  -> .mason/project.toml

This PR deliberately has no generated agent/databricks_tools.py, integration registry, AST rewriting, or runtime attachment layer. The existing framework adapters read agent.toml directly.

Verification

Local verification on the rebased branch:

• Mason unit suite: 393 passed.
• Focused CLI/manifest/runtime suite: 50 passed.
• LangGraph template custom-tool auto-discovery: passed.
• OpenAI template custom-tool auto-discovery: passed.
• Durable LangGraph template suite: 5 passed; managed TOML tools are combined with local tools.
• ruff check: passed.
• ruff format --check: passed.
• ty check with both runtime and runtime-openai extras: passed.

Live df1 E2E at d18bd893:

• 16 passed, 0 failed, 0 skipped.
• Matrix: CLI-managed vs directly authored agent.toml × local mason dev vs managed mason deploy × sandbox, MCP, custom Python, and UC function.
• CLI projects listed exactly the three managed bindings; custom Python existed only as a framework-native file under agent/tools/.
• Semantic results: MASON_SANDBOX_OK, a real system.ai.web_search HTTPS result, MASON_PYTHON_OK, and MASON_UC_OK:matrix.
• A unique freshness marker appeared in both local processes and managed App logs.
• Both temporary Apps were deleted and the temporary UC function was dropped after the run.

Shareable Alternative 2 CUJ and E2E report

The live deployment matrix uses LangGraph. OpenAI and durable-LangGraph auto-discovery are covered by the local template/runtime tests above.

This pull request and its description were written by Isaac.