Skip to content

Name identifiers through the engine - #72

Draft
vito wants to merge 18 commits into
mainfrom
format-identifiers
Draft

Name identifiers through the engine#72
vito wants to merge 18 commits into
mainfrom
format-identifiers

Conversation

@vito

@vito vito commented Oct 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

What

The Python SDK names identifiers the way the engine does, by asking it, with no casing converters of its own. Follows dagger/go-sdk#90 and dagger/typescript-sdk#120. Supersedes #62.

  • Styling. Generation spells each Python identifier styled from a schema with naming(version:) { formatIdentifiers(casing: SNAKE) } at the schema's version: one call up front for every name in the schema, then a direct call for any name not in that map. No converter is kept, even for schemas before the cut: experimental_with_all_gp_us becoming experimental_with_all_gpus is an intended 1.0 break. The codegen execs reach the session as nested clients.
  • Predictions. At run time the module asks CurrentModule.naming (core: add ModuleSource.naming for SDK codegen dagger#14996), once per call, for the names of its own interfaces, their functions and arguments, and its enum members. Those are the names that calls through its own interfaces select. No prediction mirror is kept.
  • Main object. The module build sets DAGGER_MAIN_OBJECT, for every SDK version a module vendors, from ModuleNaming.mainObject over the classes its source decorates with @object_type. A short ast script lists those classes without importing the module. The build fails when the engine takes none of them. The naming used is modSource.naming in the runtime, currentModule.naming in the entrypoints, and at generation the naming of the source generation writes (the runtime's new installed function). The strcase mainObjectName is gone.
  • Clients. A client schema is read with clientSchemaIntrospectionJson(clientVersion:). withEngineVersion renamed a module's own declarations: a pinned module's client said NamingAppHttpFetcher where the engine serves NamingAppHTTPFetcher.
  • Floor. v1.0.0-beta.25, declared rather than checked: engineVersion in the SDK's and the runtime's manifests, and [imports.Dagger] dagger in their dang.toml. The SDK's own engine refusal and its checks are gone. Engines compare base versions only, so beta.18 to beta.24 still load the SDK and fail wherever it uses what they lack. v1.3.0 remains the release for them. engine-e2e builds the floor from dagger/dagger f921c4d until the release exists.
  • User-visible rename. At beta.25 the engine names the SDK module's own types by the new rules, so its main type is PythonSDK, no longer PythonSdk (in a Dang caller, for example). Fields spell acronyms Capitalized, so the constructor stays pythonSdk, and the CLI's python-sdk doesn't change. The SDK's own Dang code also sees core by the new rules (clientSchemaIntrospectionJson).
  • Fixed along the way. Arguments of calls through an own interface were looked up by API name, so from_ or a Name() override raised KeyError.
  • Not needed. There is no self-call code to remove.
  • Not changed. Package and project names are still lowercased locally (packageNameFor in runtime/build.dang, packageName in render-template, partition.package_name in codegen). These are Python package names, not schema identifiers.

Tests

Through engine-e2e:<check> on the floor built from f921c4d, at c5d2178, and before that from c8f0ab8 at ddc8bef, with the same results. engine-e2e's assert-floor-engine precondition was bypassed locally (not committed), because it pulls the unpublished registry.dagger.io/engine:v1.0.0-beta.25:

  • Pass: sdk-test-check (558 unit tests), init-naming-check, naming-check, and all 11 generate-scope-* checks.
  • Fail: runtime-call-check, which needs the same unpublished image for its playground.

On an engine built from 9e25570, at 3d1c622, run directly with dagger check -m .dagger/modules/e2e (the nested engine that engine-e2e builds has no DNS in this sandbox):

  • Pass: init-naming-check, naming-check, runtime-call-check, and all 11 generate-scope-* checks.

Earlier, on an engine built from 017633c, at 4aa8a36:

  • Pass:
    • naming-check, new: it generates a module with an own interface and enum, without a dang.toml (beta.20 rules) and pinned to beta.22, then a caller with a client of it. It asserts the names on each side and calls through the interface.
    • runtime-call-check (an older vendored SDK reading DAGGER_MAIN_OBJECT), shared-entrypoint-build-check, generate-scope-{init,clients}, static-scope-init, static-types-load, legacy-runtime, remote-entrypoint, sdk-test. Before the floor and main-object commits, these also passed: generate-scope-{plain,global-client,view}, mod-generate-clients, resources-generate, toml-generate.
    • Unit tests and ruff.
    • init-naming-check, new: it inits http-server without a dang.toml (class HttpServer) and pinned to beta.22 (class HTTPServer), and loads both; likewise k8s-tools (K8STools / K8sTools) and e2e-test (E2ETest both sides), which the 9e25570 run above covers. It also inits http-server on the shared entrypoint and on the runtime.
  • Known failures:
    • sdk-bindings-check: sdk/src/dagger/client/gen.py hasn't been regenerated yet.
    • Playground-based e2e checks need a registry.dagger.io/engine:v1.0.0-beta.25 image, which isn't published.

Engine issues (not worked around)

  • An own interface whose function takes an own enum makes the module's own session refuse to load its schema (core/interface.go:369). Enum conversion through interfaces is covered by unit tests only.
  • Naming.formatIdentifiers styles with the engine's rules on both sides of the cut: every v1 prerelease, pre-cut ones included, shares one naming dictionary, so version: doesn't bring back the old rules. So dagger init names a new module's class with typeNames([moduleName]) of the module as generation writes it (the manifest, plus the static entrypoint's directory).

If the cut moves

  • The declared floor: dagger-module.toml, dagger.json, dang.toml, runtime/dagger-module.toml, runtime/dang.toml. If the floor ends up before the cut, the SDK's own Dang code is named by the legacy rules again: clientSchemaIntrospectionJson goes back to clientSchemaIntrospectionJSON in scope.dang, and PythonSDK goes back to PythonSdk in the e2e module.
  • .dagger/modules/e2e/main.dang: floorVersion, and the dagger = "v1.0.0-beta.25" pin in namingScope
  • .dagger/modules/engine-e2e/{main.dang,dagger-module.toml}: the floor commit
  • sdk/src/dagger/mod/_entrypoint.py: the generated entrypoint's header, and the goldens under sdk/tests/mod/golden/
  • README.md, AGENTS.md, docs/collections.md
  • sdk/src/dagger/client/gen.py (regenerate on the floor engine), sdk/src/dagger/_engine/_version.py CLI_VERSION

vito added 10 commits October 30, 2026 14:19
@vito
mod: key interface call args by Python name
a87cefd 
A call through one of the module's own interfaces binds its arguments by
their Python names, but looked their schema names up by the API name, so an
argument renamed with Name() or a trailing underscore (from_) failed with a
KeyError.

Signed-off-by: Alex Suraci <alex@dagger.io>
@vito
mod: ask the engine how it names the module
4d5cbb7 
Calls through the module's own interfaces select the schema names the engine
gave the module's declarations, which the runtime used to guess: the main
class's name followed by the interface's, function and argument names
camelCased by snake_to_camel, and the module's enum members by their Python
names. The engine names them by the rules of the version it names the
module by, so the guesses were wrong past the identifier casing cut
(NamingAppHTTPFetcher, ARM64) and before it too (strcase writes e2e_test
e2ETest, arm64 ARM_64, FooHTTPClient FooHttpclient).

Now the runtime asks: one CurrentModule.naming query per call, before
anything is invoked, for the interfaces' type names, their functions' and
arguments' names, and the members of the module's enums. A module without
interfaces asks nothing. A name the engine wasn't asked about, or can't
name, fails the call. Own enum members go over the wire by the engine's
names and come back by them.

Signed-off-by: Alex Suraci <alex@dagger.io>
@vito
e2e: call a module named on either side of the cut
2ee2c8d 
naming-check generates naming-app, a static entrypoint module with an own
interface and enum whose schema names differ between the legacy rules and
the engine's identifier naming rules, without a dang.toml (named by
v1.0.0-beta.20) and with one pinning v1.0.0-beta.22. It generates
naming-caller with a client of it, checks the Python names generation wrote
and the names the client has for naming-app's declarations, and runs
naming-caller's check, which calls naming-app through its client so that
naming-app's runtime calls its own interface by the names the engine gave
it.

The caller is a second module, as a module calls itself through a client
of its own. The interface takes no own enum: the engine refuses an own
interface whose function takes one in the module's own session, so the
module could not load its schema at all.

Signed-off-by: Alex Suraci <alex@dagger.io>
@vito
mod: take the main object the engine names
da9b546 
The runtime picked the main class by a strcase port, in Dang, of the
engine's builtin Python runtime: DAGGER_MAIN_OBJECT, the module's name
converted with strcase.ToCamel. Which class is the main object is the
engine's to say, by the naming rules it names the module by, so the
runtime asks it (ModuleNaming.mainObject), with every object type the
module's code registers as a candidate, in the query that asks for its
interfaces' names: one query per call. A module the engine names no main
object for fails to load, saying which classes it had.

The module is the one the runtime runs in (CurrentModule.naming), except
when generation renders a module's static entrypoint, which runs in the
SDK's own module: mod.dang hands the renderer, as a nested client, the
ID of the module source the engine will name (ModuleSource.naming). It
is asked from Python so that this module, which an engine before the
floor still loads in order to refuse it, names no field such an engine
lacks.

The code is imported as the default package, or through a configured
entry point, without a class name to import.

Signed-off-by: Alex Suraci <alex@dagger.io>
@vito
codegen: style identifiers through the engine
7849d6d 
The generator spelled Python names with a converter of its own: acronym
runs title-cased, then graphql-core's camel_to_snake, which cuts
experimentalWithAllGPUs into experimental_with_all_gp_us. The engine knows
where its names' words are, so it spells them now: formatIdentifiers(casing:
SNAKE) at the schema's version, asked once up front for every type, field,
argument and input field name of the schema, and directly for any other
name the generator styles (a name a deprecation notice refers to). A name
the engine can't format fails the generation. No converter is kept, so
schemas before the identifier casing cut get the engine's spelling too
(experimental_with_all_gpus): an intended break of 1.0.

The generator reaches the engine as a nested client of the session, with
the standard library's HTTP client, so the codegen execs run with
experimentalPrivilegedNesting. The keyword and builtin suffix stays the
SDK's. The unit tests style made-up schemas with a stand-in, since they run
no engine; the e2e checks generate with one.

Signed-off-by: Alex Suraci <alex@dagger.io>
@vito
sdk: raise the engine floor to v1.0.0-beta.22
203efed 
Generation asks the engine how to spell the schema's Python names
(formatIdentifiers) and a module asks it what it named its own declarations
(CurrentModule.naming), which the release at the identifier casing cut is
the first to answer. Engines from v1.0.0-beta.18 to v1.0.0-beta.21 are
offered v1.3.0, the last release that runs on them.

engine-e2e builds the floor from the dagger/dagger commit that carries the
API until v1.0.0-beta.22 is released, and v1.0.0-beta.21 as the release
below it.

Signed-off-by: Alex Suraci <alex@dagger.io>
@vito
engine-e2e: build the floor with the naming API
6daa85c 
The module runtime asks CurrentModule.naming, which the engine stack has
from 017633c on.

Signed-off-by: Alex Suraci <alex@dagger.io>
@vito
scope: read clients at their own naming
1776126 
A client's schema was read with withEngineVersion(view), which makes the
module declare the scope's version, and since the identifier casing cut
that also names its declarations by that version's rules: the client of a
module the engine names by v1.0.0-beta.22 had NamingAppHttpFetcher where
the engine serves NamingAppHTTPFetcher. clientSchemaIntrospectionJSON's
clientVersion describes core at the scope's view and leaves the module at
the version it declares.

Signed-off-by: Alex Suraci <alex@dagger.io>
@vito
template: name the main class through the engine
c2e3b4c 
A new module's main class was named with strcase.ToCamel of the module's
name, a casing converter of the SDK's own. The engine spells it now
(formatIdentifiers, PASCAL), asked by the template renderer as a nested
client: k8s-tools is K8sTools, as the engine's naming rules name it. The
unused ModuleImport (strcase.ToKebab) and mod._utils.to_pascal_case go
too.

Signed-off-by: Alex Suraci <alex@dagger.io>
@vito
codegen: record every input field's GraphQL name
4bf2884 
The query builder derived an input field's GraphQL name from its Python
name with snake_to_camel, and the generator recorded only the names that
conversion missed. Python names are the engine's spelling now, so nothing
converts them back: the generator records every field's GraphQL name and
the query builder sends that one, refusing a field without one.

Signed-off-by: Alex Suraci <alex@dagger.io>
@vito
vito force-pushed the format-identifiers branch 3 times, most recently from ef92ea6 to 34be346 Compare October 30, 2026 23:43
vito added 2 commits October 30, 2026 23:46
@vito
sdk: declare the engine floor, don't check it
4cc9856 
The SDK and its runtime declare engineVersion v1.0.0-beta.22, in their
manifests and in dang.toml, which decides a Dang module's version. The SDK's
own engine refusal, its checks and the below-floor engine go. Engines compare
base versions only, so beta.18 to beta.21 still load the SDK and fail where
it uses what they lack; that is accepted.

At that version the engine names the SDK's own types by its new rules: its
main type is PythonSDK, where it was PythonSdk, which the e2e module now
names. Fields capitalize acronyms, so the constructor stays pythonSdk. The
SDK's Dang code sees the core API the same way, so it reads
clientSchemaIntrospectionJson.

Signed-off-by: Alex Suraci <alex@dagger.io>
@vito
build: name the main object in the module build
e8f07e5 
The Python runtime asked the engine for the main object itself, which only
an SDK new enough to ask could do: a module vendoring an older SDK reads
DAGGER_MAIN_OBJECT and found it unset. The build sets it again, for every
SDK version, from ModuleNaming.mainObject over the classes the module's
source decorates with @object_type, which a short ast script lists without
importing the module. The build fails when the engine takes none of them.

The naming is the module's: the runtime's modSource.naming, the
entrypoints' currentModule.naming, and at generation the naming of the
source generation writes, through the runtime's new installed function.
The Python side goes back to asking only for its interfaces' names, and
generation no longer hands the renderer a module source ID.

Signed-off-by: Alex Suraci <alex@dagger.io>
@vito
vito force-pushed the format-identifiers branch from 34be346 to e8f07e5 Compare October 30, 2026 23:47
vito added 6 commits October 31, 2026 00:05
@vito
sdk: fix what review of the naming port found
6fd08f3 
- The code generator reaches its session without a proxy: urllib honored
  the HTTP_PROXY an exec inherits, and a non-JSON error reply escaped as a
  traceback.
- The main object scan skips a file this Python can't parse, which the
  module may never import.
- experimentalPrivilegedNesting goes from the execs that reach the session:
  nesting is the default of the v1 API, which ignores the flag.
- The generated entrypoint's header names the beta.22 floor, and the README
  says which rules name a module's own declarations.
- A comment no longer points at the deleted engine refusal.

Signed-off-by: Alex Suraci <alex@dagger.io>
@vito
template: name the main class as the engine will
5044f68 
The template named a new module's class with formatIdentifiers, which
spells by the newest rules whatever the version. A new module has no
dang.toml, so the engine names it by v1.0.0-beta.20's legacy rules, and
takes HTTPServer, the class of `http-server`, for no main object: the module
did not load. Generation now asks the engine how it will name the module
(ModuleNaming.typeNames of the module name), of the manifest it writes,
before the template renders, and hands the name to render-template.

init-naming-check generates http-server without a version and pinned past
the cut, and loads it.

Signed-off-by: Alex Suraci <alex@dagger.io>
@vito
e2e: init http-server on every kind of entrypoint
4aa8a36 
The shared entrypoint and the runtime name a new module by its manifest
alone, a path of generation's main class naming that no other init check
took.

Signed-off-by: Alex Suraci <alex@dagger.io>
@vito
engine-e2e: build the floor at 9e25570
3d1c622 
The engine stack's new head names a module's main object pre-cut by what
typeNames answers, so init-naming-check now takes k8s-tools and e2e-test,
which strcase spells differently on a second read, on both sides of the cut.
Interface members are named once there, so the naming check's note about
the legacy double normalization goes.

Signed-off-by: Alex Suraci <alex@dagger.io>
@vito
sdk: move the cut to v1.0.0-beta.25
ddc8bef 
beta.22 to beta.24 ship without the engine stack that carries the new
naming rules and ModuleSource.naming/CurrentModule.naming, so the cut, and
this SDK's declared floor, is now v1.0.0-beta.25. engine-e2e builds the
floor from dagger/dagger c8f0ab8, the stack restacked at beta.25, until the
release exists.

Signed-off-by: Alex Suraci <alex@dagger.io>
@vito
codegen: format identifiers through Query.naming
c5d2178 
dagger#14996 moved formatIdentifiers under Query.naming(version:), so the
generator now asks naming(version: $v) { formatIdentifiers(...) }, or naming
without a version where it passed none. engine-e2e builds the floor from
dagger/dagger f921c4d, which serves the new API.

Signed-off-by: Alex Suraci <alex@dagger.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vito