[Improvement-18593] [API&DAO] Optimize TaskInstanceMapper to exclude large text fields from list queries

[njnu-seafish]

Was this PR generated or assisted by AI?

YES. Leverage LLMs to verify that the removed large text fields are indeed unused.

Purpose of the pull request

close #18593

Brief change log

Optimize TaskInstanceMapper to exclude large text fields from list queries

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(or)

Pull Request Notice

Pull Request Notice

If your pull request contains incompatible change, you should also add it to docs/docs/en/guide/upgrade/incompatible.md

[ruanwenjun]

Please provide benchmark results to show exactly how much performance this type of refactoring can improve. Otherwise, we should return all fields in Listing interface, rather than add new VO/DTO.

[ruanwenjun]

Remove the text column at listing api.

[njnu-seafish]

Remove the text column at listing api.

Sorry, I'm not quite sure I understand this review comment. I've already run the SQL benchmark with large fields excluded.

[njnu-seafish]

Please provide benchmark results to show exactly how much performance this type of refactoring can improve. Otherwise, we should return all fields in Listing interface, rather than add new VO/DTO.

Thanks for the feedback. We ran a benchmark on MySQL 8.0 (InnoDB) with 10,000 workflow instances and 50,000 task instances to quantify the improvement. The test simulates a typical list/paging query (LIMIT 20) and runs 10 iterations per variant.

Excluded columns:

Benchmark results (10-run average, LIMIT 20 per page):

Analysis:
SQL latency: t_ds_workflow_instance uses an index scan (start_time_index), so reducing the column set directly cuts the amount of row data read — yielding a 42.8% improvement. t_ds_task_instance does a full table scan + filesort (no covering index for project_code + submit_time), so the SQL execution time is dominated by sorting rather than column projection, and the latency gain is a more modest 9.3%.

Data transfer savings (not reflected in SQL latency): Each page of 20 rows avoids transferring ~64 KB and ~23 KB of large text respectively. In a real-world deployment, this overhead is amplified through the full stack: JDBC ResultSet materialization → MyBatis entity mapping → Jackson JSON serialization for the API response. For high-traffic list endpoints (e.g. task instance paging under a busy project), this reduction meaningfully lowers GC pressure and response payload size.

We believe the lightweight projection is worthwhile even for the t_ds_task_instance case: the 9.3% SQL latency improvement plus the ~23 KB/page transfer savings represent a real gain for a frequently accessed endpoint, with no functional impact (the omitted fields are only needed for task execution, detail views, or log retrieval, not for list/paging display).

[njnu-seafish]

@ruanwenjun When you have a moment, could you please review the code again? Thanks so much!

[njnu-seafish]

This E2E test failure appears to be unrelated to the changes in this PR.

[ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 295.474 s <<< FAILURE! - in org.apache.dolphinscheduler.e2e.cases.tasks.PythonTaskE2ETest
[ERROR] testRunPythonTasks_UsingResourceFile Time elapsed: 35.433 s <<< ERROR!
org.openqa.selenium.StaleElementReferenceException:
stale element reference: stale element not found in the current frame
(Session info: chrome=125.0.6422.76)
Session ID: 4f19366fefac1dd7215b898187c3bff0
at org.apache.dolphinscheduler.e2e.cases.tasks.PythonTaskE2ETest.testRunPythonTasks_UsingResourceFile(PythonTaskE2ETest.java:280)
2026-09-03 10:33:49,440 pool-1-thread-1 DEBUG Stopping LoggerContext[name=5c29bfd, org.apache.logging.log4j.core.LoggerContext@66ce957f]
2026-09-03 10:33:49,440 pool-1-thread-1 DEBUG Stopping LoggerContext[name=5c29bfd, org.apache.logging.log4j.core.LoggerContext@66ce957f]...
2026-09-03 10:33:49,441 pool-1-thread-1 DEBUG Shutting down OutputStreamManager SYSTEM_OUT.false.false
2026-09-03 10:33:49,441 pool-1-thread-1 DEBUG OutputStream closed
2026-09-03 10:33:49,441 pool-1-thread-1 DEBUG Shut down OutputStreamManager SYSTEM_OUT.false.false, all resources released: true
2026-09-03 10:33:49,441 pool-1-thread-1 DEBUG Appender Console stopped with status true
2026-09-03 10:33:49,441 pool-1-thread-1 DEBUG Stopped XmlConfiguration[location=/home/runner/work/dolphinscheduler/dolphinscheduler/dolphinscheduler-e2e/dolphinscheduler-e2e-core/target/classes/log4j2.xml] OK
2026-09-03 10:33:49,441 pool-1-thread-1 DEBUG Stopped LoggerContext[name=5c29bfd, org.apache.logging.log4j.core.LoggerContext@66ce957f] with status true
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] PythonTaskE2ETest.testRunPythonTasks_UsingResourceFile:280 » StaleElementReference
[INFO]
[ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 1