Skip to content

fix(mssql): cast table_diff values to VARCHAR(MAX) - #6133

Open
raams95 wants to merge 1 commit into
SQLMesh:mainfrom
raams95:fix/mssql-table-diff-varchar-length
Open

raams95 wants to merge 1 commit into
SQLMesh:mainfrom
raams95:fix/mssql-table-diff-varchar-length

Conversation

@raams95

@raams95 raams95 commented Oct 13, 2026 •

Copy link
Copy Markdown

Description

When table_diff has more than one key column it builds a single __sqlmesh_join_key by casting each key to a string and concatenating them. On SQL Server that cast was CAST(x AS VARCHAR) with no length, and T-SQL silently treats that as VARCHAR(30) inside CAST. This causes two problems:

This overrides normalize_value in the MSSQL adapter to cast to VARCHAR(MAX) instead, along the lines of what was suggested in #5146. Other engines are unchanged. Fabric inherits the same behaviour since it uses the MSSQL adapter.

Fixes #4478
Fixes #5146

Test Plan

  • Added test_concat_columns_uses_varchar_max in tests/core/engine_adapter/test_mssql.py, which checks the SQL generated for a composite key with a UNIQUEIDENTIFIER and an NVARCHAR(100) column. It fails on main (CAST(id AS VARCHAR)) and passes with this change.
  • pytest tests/core/engine_adapter/test_mssql.py tests/core/engine_adapter/test_fabric.py tests/core/test_table_diff.py passes, apart from 4 table_diff console tests that fail the same way on main on my Windows machine (box-drawing characters in the rich output).
  • ruff check, ruff format --check and mypy are clean on the changed files.
  • I haven't run this against a live SQL Server instance.

Checklist

  • I have run make style and fixed any issues (ruff and ruff-format pass; mypy and the migrations hook report the same environment-only errors on main on my Windows machine)
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test) (no new failures compared to main on my Windows machine; the Windows-only failures on main are unchanged)
  • My commits are signed off (git commit -s) per the DCO

CAST(x AS VARCHAR) without a length is truncated to 30 characters in
T-SQL, so composite join keys built by table_diff could collide or fail
for long strings and UNIQUEIDENTIFIER columns.

Fixes SQLMesh#4478, fixes SQLMesh#5146

Signed-off-by: JanakiRamChittineni <341781521+raams95@users.noreply.github.com>
@mday-io

mday-io commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

Thanks, this is the right fix. The T-SQL docs confirm the default length is 30 inside CAST, and VARCHAR(MAX) is supported on Fabric too. That's also why it shouldn't be NVARCHAR, since Fabric has no nvarchar; worth a word in the comment so nobody "fixes" it later. Two test asks before merging:

I'll approve the CI run so the MSSQL integration tests run against it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants