Dev

AGENTS.md

@@ -27,7 +27,7 @@ Groovy 沙箱默认启用但不是 JVM 隔离：勿把脚本编写权开放给
 
 ## 三、凭据外部化
 
-真实值一律环境变量注入，禁止写回仓库 yaml：`DATAPOLY_ADMIN_PASSWORD`、`DATAPOLY_REDIS_PASSWORD`、`DATAPOLY_DS_AES_KEY`（轮换前须用旧密钥导出重录）、`DATAPOLY_CORS_ALLOWED_ORIGINS`、compose 的 `MYSQL_ROOT_PASSWORD`/`MYSQL_PASSWORD`（演示默认 123456）。演示凭据 admin/123456、test/test 正式部署必须修改；actuator 已收窄为 health,info。
+真实值一律环境变量注入，禁止写回仓库 yaml：`DATAPOLY_ADMIN_PASSWORD`、`DATAPOLY_REDIS_PASSWORD`、`DATAPOLY_DS_AES_KEY`（轮换前须用旧密钥导出重录）、`DATAPOLY_CORS_ALLOWED_ORIGINS`、compose 的 `MYSQL_ROOT_PASSWORD`/`MYSQL_PASSWORD`（演示默认 123456）。演示凭据 admin/123456、test/test 正式部署必须修改；actuator 已收窄为 health,info。飞书登录凭证 `DATAPOLY_FEISHU_APP_ID`/`DATAPOLY_FEISHU_APP_SECRET` 同样只从环境变量注入：compose 取被忽略的 `install/.env`，发行版 `conf/config.ini` 留空即关闭（`datapolyctl.sh` 只在键有值时导出，空串会让布尔属性宽松绑定失败）。
 
 ## 四、端点防护
 
@@ -36,7 +36,8 @@ Groovy 沙箱默认启用但不是 JVM 隔离：勿把脚本编写权开放给
 ## 五、其他约束
 
 - 新增 JDBC 代码资源必须 try-with-resources；firewall 规则行被删时网关按"全拒绝"处理（fail-closed，属预期）。
-- DataTask 投递 Sink 仓库内置零实现，外部以 Spring Bean / `META-INF/services` 注册（SPI `com.cs.common.datatask.DataTaskSink`）；宿主可自行维护本地扩展：在顶层 `datapoly-extension/`（已被 .gitignore 排除，独立 git 仓库）下用 `backend/` 放 Maven 扩展模块（依赖钉版在模块自身 pom、不进根 reactor，由入库脚本 build-extension.sh 在宿主机 JDK 25 构建后投放 lib-extra/，随发行版装配进各服务 classpath）、`front/` 放扩展 UI；API 扩展点 `ApiAssignmentPostProcessor` 注册方式相同、须同步执行且保持轻量。详见 docs/*/data-task.md。
-- 默认前端扩展目录 `datapoly-extension/front`（同被 .gitignore 排除）经 datapoly-manager-ui 编译期装配：webpack `@extension` 别名自动探测该目录 `src/index.js`（见 build/webpack.base.conf.js）、`src/extension-stub` 为缺省回退、扩展路由与 i18n 词条在 manager-ui 入口深合并——这四处钩子文件（build/webpack.base.conf.js、src/extension-stub、src/router、src/main.js）勿移除或改名；目录不存在时 CI 与普通构建不受影响。front 自带 `package.json` 可直接启动（`npm run dev`，复用宿主 webpack 链，前置为宿主 node_modules 已安装、Node 24——dev server 与生产构建均已实测）。
+- DataTask 投递 Sink 仓库内置零实现，外部以 Spring Bean / `META-INF/services` 注册（SPI `com.cs.common.datatask.DataTaskSink`；服务端导出类投递——如 MaxCompute `UNLOAD` 直写对象存储——另实现可选接口 `DataTaskStatementSink`：`handlesStatement` 判定、`executeStatement` 执行，引擎跳过整条行式管线并在阻塞期间代为续租，取消只在提交前拦得住）；宿主可自行维护本地扩展：在顶层 `datapoly-extension/`（已被 .gitignore 排除，独立 git 仓库）下用 `backend/` 放 Maven 扩展模块（依赖钉版在模块自身 pom、不进根 reactor，由入库脚本 build-extension.sh 在宿主机 JDK 25 构建后投放 lib-extra/，随发行版装配进各服务 classpath）、`front/` 放扩展 UI；API 扩展点 `ApiAssignmentPostProcessor` 注册方式相同、须同步执行且保持轻量。详见 docs/*/data-task.md。
+- 默认前端扩展目录 `datapoly-extension/front`（同被 .gitignore 排除）经 datapoly-manager-ui 编译期装配：webpack `@extension` 别名自动探测该目录 `src/index.js`（见 build/webpack.base.conf.js）、`src/extension-stub` 为缺省回退、扩展路由与 i18n 词条在 manager-ui 入口深合并、登录页扩展区渲染 `@extension` 的 `loginExtras` 组件数组（`src/views/login/index.vue`，stub 为空数组）——这五处钩子（build/webpack.base.conf.js、src/extension-stub、src/router、src/main.js、src/views/login/index.vue 的 loginExtras 挂载点）勿移除或改名；目录不存在时 CI 与普通构建不受影响。front 自带 `package.json` 可直接启动（`npm run dev`，复用宿主 webpack 链，前置为宿主 node_modules 已安装、Node 24——dev server 与生产构建均已实测）。
+- 用户角色见 `DATAPOLY_SYSTEM_USER.user_role`（`ADMIN`/`USER`，v1.4.0 迁移：新行默认 `USER`、存量行回填 `ADMIN`），登录响应 `AccessToken.role` 带回该值；角色只决定界面可见范围，鉴权仍以 token 为准，按角色的端点拦截须同时校验数据库中的角色而非前端传来的值。
 - 宿主扩展 jar 经根目录 `lib-extra/` 投放点进入发行版 `lib/common/`（`package.xml` 打包该目录 `*.jar`；目录只占位入库，jar 永不入库）。扩展为独立 git 仓库（内部 GitLab，front+backend 一体）：`build-extension.sh` 按环境变量 `DATAPOLY_EXTENSION_GIT_URL`（真实地址不入库，CI 注入）+ `DATAPOLY_EXTENSION_GIT_REF`（默认 master）浅克隆到 `datapoly-extension/`（仍被 .gitignore 排除），目录已存在则按本地工作区构建（宿主机 JDK 25 优先、低于 25 不可用）、`DATAPOLY_EXTENSION_FORCE_SYNC=1` 强制覆盖本地改动；`build.sh`/`docker-maven-build.sh` 会先调用该脚本，未配置且目录不存在时无操作（纯开源构建零影响）。本地环境变量注入（env.sh）与防误提交钩子集中在被忽略的 `dev-local/`。
 - 一次性 token 在校验时即消费（含查库兜底路径；2026-09 修复兜底不消费导致的重放）；并发首用竞态下多 executor 仍可能各放行一次（无分布式锁，已知限制）。

CHANGELOG.md

@@ -12,6 +12,21 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ### Added
 
+- DataTask statement sinks (optional capability, backward compatible): the new
+  interface `com.cs.common.datatask.DataTaskStatementSink` lets a delivery
+  provider claim a definition and run its rendered statement itself, for
+  exports that must complete inside the source engine (MaxCompute
+  `UNLOAD ... INTO LOCATION 'oss://...'`) instead of streaming rows through the
+  executor. `DataTaskJobEngine` consults the sink once, right after rendering
+  and before any JDBC work: on a claim it skips the whole row pipeline (no
+  session, no result set, no row limit or reshaping) and records the returned
+  `SinkOutcome` like any other artifact. Because such a statement can outlive
+  `lease-seconds`, a lazily started daemon refreshes the job lease every
+  `lease-seconds/3` while it blocks; the cancel probe can only stop a statement
+  before submission, so a late cancel keeps the artifact and is recorded as
+  `artifactInfo.cancelRequested`. Existing sinks are untouched — a sink that
+  never claims a definition behaves exactly as before. Documented in
+  `docs/{zh,en}/data-task.md` §5.
 - DataTask sink contract (net-neutral extension): `SinkRequest` now carries
   `columnMetadata` (per-column JDBC type hints, shaped through the same
   projection as the columns) and the terminal `DataTaskEvent` includes the

build-docker/install/docker-compose.yml

@@ -44,6 +44,12 @@ services:
       DATAPOLY_DS_ENCRYPT: false
       # 部署后请务必设置：覆盖种子 admin 口令（S5）
       DATAPOLY_ADMIN_PASSWORD: ${DATAPOLY_ADMIN_PASSWORD:-}
+      # 飞书登录（datapoly-extension-feishu，relaxed binding 映射为 datapoly.feishu.*）：
+      # 真实值由 install/.env 注入，勿写回本文件；三项凭证缺一或开关关闭时登录页不显示飞书按钮
+      DATAPOLY_FEISHU_ENABLED: ${DATAPOLY_FEISHU_ENABLED:-false}
+      DATAPOLY_FEISHU_APP_ID: ${DATAPOLY_FEISHU_APP_ID:-}
+      DATAPOLY_FEISHU_APP_SECRET: ${DATAPOLY_FEISHU_APP_SECRET:-}
+      DATAPOLY_FEISHU_REDIRECT_URI: ${DATAPOLY_FEISHU_REDIRECT_URI:-}
     #      DATAPOLY_MANAGER_URL: http://www.example.com:8090
     #      DATAPOLY_GATEWAY_URL: http://www.example.com:8091
     depends_on:

datapoly-common/src/main/java/com/cs/common/datatask/DataTaskStatementRequest.java

@@ -0,0 +1,85 @@
+// Use of this source code is governed by a BSD-style license
+package com.cs.common.datatask;
+
+import com.cs.common.enums.ProductTypeEnum;
+import lombok.Builder;
+import lombok.Data;
+
+import javax.sql.DataSource;
+import java.util.List;
+import java.util.Map;
+import java.util.function.BooleanSupplier;
+
+/**
+ * Everything a {@link DataTaskStatementSink} needs to decide about, and then run, one
+ * definition's statement outside the engine's row pipeline: the rendered SQL, the
+ * definition's delivery configuration, the datasource it was authored against and the
+ * cooperative cancellation probe.
+ *
+ * <p>The engine builds this object right after rendering the template and before any
+ * JDBC work, so a sink can inspect it in
+ * {@link DataTaskStatementSink#handlesStatement(DataTaskStatementRequest)} without
+ * paying for a connection.</p>
+ */
+@Data
+@Builder
+public class DataTaskStatementRequest {
+
+    private Long jobId;
+
+    private String taskName;
+
+    /** Registration type of the sink this definition selected. */
+    private String sinkType;
+
+    /**
+     * Opaque JSON configuration stored on the task definition; its schema belongs to
+     * the sink implementation. Secrets placed here are persisted in plain text in the
+     * meta store — providers should reference server-side credentials instead.
+     */
+    private String sinkConfig;
+
+    /**
+     * Statement rendered from the definition template. For server-side execution this
+     * text must be complete: drivers of these engines generally accept no bind
+     * parameters, so callers author such definitions with {@code ${param}} inlining
+     * ({@code dollarAllowed}) rather than {@code #{param}}.
+     */
+    private String sql;
+
+    /**
+     * Bind values the renderer left behind as placeholders; a non-empty list means
+     * {@link #sql} is not executable as-is and the definition must be rewritten with
+     * inlined parameters.
+     */
+    private List<Object> sqlParameters;
+
+    /**
+     * Whether {@link #sql} is a query (SELECT/WITH, ignoring leading whitespace, parens and
+     * comments). Server-side exports wrap the statement in their own command — MaxCompute's
+     * {@code UNLOAD FROM (<sql>)}, for instance — which only accepts a query, so a sink
+     * claiming such definitions must leave everything else (DML, DDL, engine-specific
+     * statements) on the ordinary path.
+     */
+    private boolean query;
+
+    /** Boundary values that produced {@link #sql}, for logging and diagnostics. */
+    private Map<String, Object> params;
+
+    private Long datasourceId;
+
+    private ProductTypeEnum product;
+
+    /** Pooled datasource of the definition, shared with the synchronous API path. */
+    private DataSource dataSource;
+
+    /** Account id of the submitter, as carried by the job record. */
+    private String submittedBy;
+
+    /**
+     * True once the job row left RUNNING or a cancel was requested — the same predicate
+     * the row pipeline uses. Long-running statements should poll it before submitting;
+     * an already-submitted server-side job will finish regardless.
+     */
+    private BooleanSupplier cancelled;
+}

datapoly-common/src/main/java/com/cs/common/datatask/DataTaskStatementSink.java

@@ -0,0 +1,40 @@
+// Use of this source code is governed by a BSD-style license
+package com.cs.common.datatask;
+
+/**
+ * Optional capability of a {@link DataTaskSink}: some deliveries must not be produced by
+ * this JVM at all. Server-side exports (MaxCompute {@code UNLOAD}, warehouse
+ * {@code INSERT OVERWRITE DIRECTORY}, ...) complete inside the source engine and write
+ * straight to the target store, so the engine must hand over the statement instead of
+ * executing it and pushing rows.
+ *
+ * <p>{@link #handlesStatement(DataTaskStatementRequest)} is consulted once per job, right
+ * after the template is rendered and before any JDBC execution. Returning {@code true}
+ * makes the engine skip the row pipeline entirely — no {@link SinkSession}, no row limit,
+ * no column reshaping — and call {@link #executeStatement(DataTaskStatementRequest)}
+ * instead; the outcome is recorded on the job like any other sink outcome. Returning
+ * {@code false} leaves the definition on the ordinary path, so one sink can serve both
+ * modes (e.g. stream to object storage for most engines, delegate to the engine's own
+ * export command for one of them).</p>
+ *
+ * <p>Implementations own resource cleanup for the delegated path: the engine has no
+ * session to abort when the statement fails or the job is cancelled mid-flight.</p>
+ */
+public interface DataTaskStatementSink extends DataTaskSink {
+
+    /**
+     * Non-statement implementations never reach this method — a sink that claims a
+     * definition via {@link #handlesStatement} is driven through
+     * {@link #executeStatement} only. Sinks serving both modes override it normally.
+     */
+    @Override
+    default SinkSession openSession(SinkRequest request) throws Exception {
+        throw new UnsupportedOperationException(
+                type() + ": this sink is driven through executeStatement, not openSession");
+    }
+
+    /** @return true to have the engine delegate this definition's statement to the sink */
+    boolean handlesStatement(DataTaskStatementRequest request);
+
+    SinkOutcome executeStatement(DataTaskStatementRequest request) throws Exception;
+}

datapoly-common/src/main/java/com/cs/common/dto/AccessToken.java

@@ -30,4 +30,7 @@ public class AccessToken implements Serializable {
 
     @Schema(description = "有效期(时间段，单位:秒)")
     private Long expireSeconds;
+
+    @Schema(description = "角色(ADMIN/USER)")
+    private String role;
 }