Skip to content

fix: Reject deleted override entries, and key each override by the key it is listed under - #259

Open
kinyoklion wants to merge 2 commits into
rlamb/overrides-java-contract-testsfrom
rlamb/overrides-java-reject-deleted
Open

kinyoklion wants to merge 2 commits into
rlamb/overrides-java-contract-testsfrom
rlamb/overrides-java-reject-deleted

Conversation

@kinyoklion

@kinyoklion kinyoklion commented Nov 10, 2026 •

Copy link
Copy Markdown
Member

Update, 2026-11-13

  • Rebased onto feat/overrides, which now includes feat: Add the overrideAffected indicator to EvaluationReason #218 and feat: Count override-affected evaluations separately and send no individual events for them #219 and has main merged into it. From main it gets the 7.18.0 release, CI on the latest v3 contract tests, flag change listener support in the contract test service, and the java-sdk-internal 1.12.1 and 1.12.2 fixes. A common 2.6.0 and internal 1.13.0 release from this branch needs those fixes.
  • New commit, "fix: Key each override by the key it is listed under, and reject one listed under another key":
    • Evaluation reads an override entry by the key it is listed under, but the all-flags state and events report the definition's own key. The file source deserialized each flags and segments entry as it was, so an entry with no key property made allFlagsState() throw a NullPointerException to the application, and an entry pasted under a new key with its old key property was served under one key and reported under the other. I confirmed both with a probe against the stack.
    • Each entry now takes the key it's listed under, as the Node, Go and Python sources key it. A key property is optional, and FileOverrides.source() says so.
    • The override layer also rejects a snapshot that lists a flag or segment under a key other than its own, the same way as one with a deleted item: the sink logs it and the overrides in effect stay. Only a custom source can produce such a snapshot now. Python added the same check after keelerm84's review of its port (python-server-sdk#527). Go, Node, .NET and Ruby don't have it yet.
    • Tests: OverrideFileLoaderTest.entryIsKeyedByTheKeyItIsListedUnder (JSON and YAML, flags and segments), LDClientOverridesTest.fileEntryIsServedAndReportedUnderTheKeyItIsListedUnder, OverrideLayerTest.replacementThatListsAnItemUnderAnotherKeyIsRejectedAndTheLayerIsUnchanged (including a definition with no key), and OverrideSinkImplTest.snapshotThatListsAFlagUnderAnotherKeyIsRejected. Removing the keying fails the first two, and removing the layer check fails the other two.
  • Verified at the top of the stack, with common and internal published locally from feat/overrides:

Requirements

  • I have added test coverage for new or changed functionality
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions

Related issues

Implements the "reject deleted entries" rule of the flag overrides spec, launchdarkly/sdk-specs#288 (Requirements 1.1.1.1, 1.2.8.1 and 1.3.6.2).

This PR is stacked on #225 (rlamb/overrides-java-contract-tests) so that the PRs below it do not change. Retarget it when the stack merges.

Describe the solution you've provided

Before: a flag or segment entry with "deleted": true became a deleted placeholder. That was true both in an override file and in a snapshot from a custom source. The override layer stored it and the overlay served it, so the key looked absent: the LaunchDarkly definition was hidden from evaluation, from prerequisites and segment reads, and from the all-flags state, and the evaluation was not marked as override-affected.

After:

  • Sink (every source, including a custom one through OverrideSink). The override layer never holds a deleted item. If a snapshot contains one, the SDK rejects the whole snapshot. The layer stays unchanged, no flag change listener is notified, and the SDK logs an Error naming the kind and the key, for example Rejected an override snapshot because flag "f" is marked as deleted; the overrides in effect are unchanged. A deleted item is either the placeholder that DataKind.deserialize produces or a model object marked as deleted. The check runs in OverrideLayer.setAll before the contents are swapped, and OverrideSinkImpl logs the rejection. The OverrideSink signature is unchanged and nothing is thrown to the source.
  • File override source. OverrideFileLoader fails the load when a file has a flag or segment entry marked as deleted, the same way it fails for a file that cannot be parsed. The overrides in effect stay, and the existing retry applies. The failure is logged like other load failures and names the file and the key: Unable to load flags: file /path/overrides.json: flag "f" is marked as deleted; remove the entry to remove its override. The check also fires when the duplicate keys handling would drop the entry, because the file still needs correcting. The source detects this while loading, before it calls the sink.
  • "deleted": false is an ordinary definition and still loads.
  • Public docs. FileOverrides.source() gains one sentence saying that a file with an entry marked as deleted fails to load and that an override is removed by removing its entry or deleting the file. OverrideSink.setOverrides gains one sentence saying that a snapshot with a deleted flag or segment is rejected. The experimental notice is unchanged on both.

The ordinary file data source is unchanged on purpose. FileDataSourceBase and FileDataSourceParsing are not modified. Rejecting deleted entries there would be a breaking change, which is planned for a later major version. The check lives only in the override path (OverrideFileLoader and the override layer). A new test, DataLoaderTest.entriesMarkedAsDeletedStillLoadAsDeletedItems, pins the data source's current behavior: it still loads such entries as deleted items.

Tests

  • OverrideLayerTest: replacementThatContainsADeletedItemIsRejectedAndTheLayerIsUnchanged replaces deletedPlaceholderIsStoredAsIs, which pinned the old behavior. It covers the deserialized placeholder and a model object marked as deleted, for both flags and segments. definitionWithDeletedFalseIsStored is new.
  • OverrideSinkImplTest: snapshotWithADeletedFlagIsRejected, snapshotWithADeletedSegmentIsRejected and snapshotWithADeletedItemIsRejectedWhenNothingIsListening. Each checks that the layer is unchanged, that no notification is sent, and that the Error is logged.
  • OverrideFileLoaderTest: flagMarkedAsDeletedFailsTheLoad and segmentMarkedAsDeletedFailsTheLoad cover JSON and YAML, with the exact message naming the file and the key. Also entryMarkedAsDeletedFailsTheLoadEvenWhenItIsADroppedDuplicate and entriesWithDeletedFalseAreOrdinaryDefinitions.
  • FileOverrideSourceTest: flagMarkedAsDeletedFailsTheReloadAndKeepsTheOverrides and segmentMarkedAsDeletedFailsTheReloadAndKeepsTheOverrides. In each, an in-place edit adds the deleted entry and changes another override. Nothing is supplied, the logged Error names the file and the key, and removing the entry recovers. entriesWithDeletedFalseLoad is also new.
  • DataLoaderTest.entriesMarkedAsDeletedStillLoadAsDeletedItems: the ordinary file data source behavior is pinned as unchanged.

How I ran them, on JDK 17 (arm64):

  1. Published the unreleased shared libraries locally from the monorepo. Common came from the feat: Add the overrideAffected indicator to EvaluationReason #218 head (b5deba0) with ./gradlew publishToMavenLocal -Pversion=2.6.0 -PskipSigning=true in lib/shared/common. Internal came from the feat: Count override-affected evaluations separately and send no individual events for them #219 head (0e7744f) with -Pversion=1.13.0 in lib/shared/internal.
  2. In lib/sdk/server, ran the override and file data source classes with ./gradlew --max-workers=2 --no-daemon -Dorg.gradle.jvmargs=-Xmx2g test --tests '*Override*' --tests '*.integrations.File*' --tests '*.integrations.DataLoaderTest' --tests '*.integrations.FlagFileParser*' --tests '*.integrations.ClientWithFileDataSourceTest'. Result: 343 tests in 29 classes, 0 failures.
  3. Ran the full server test task (same Gradle options). Result: 2275 tests in 129 classes, 0 failures. The 72 skipped tests are the existing skips in PersistentDataStoreWrapperTest.
  4. Ran checkstyleMain checkstyleTest javadoc: passed, with no new Javadoc warnings.
  5. Checked that the new tests fail without the change. With the layer check and the loader check disabled, all 9 rejection tests fail, and the deleted: false tests and the data source pin still pass. With the sink's catch removed, the 3 sink tests fail.

Describe alternatives you've considered

  • Checking only in the sink, with a separate pass over the snapshot. That pass would iterate the source's Iterable twice. Checking in OverrideLayer.setAll keeps it to one pass and makes the layer enforce its own invariant.
  • Dropping only the deleted entry. The spec requires rejecting the whole snapshot, so the layer is always exactly a snapshot that the source supplied.

Additional context

CI stays red until the shared libraries are released: the stack pins launchdarkly-java-sdk-common 2.6.0 and launchdarkly-java-sdk-internal 1.13.0, which are not published yet.

Opened from termag

SDK-3578


Note

Overview
Override snapshots that include a deleted flag/segment placeholder, or a definition listed under a key that does not match its internal key, are now rejected as a whole: the override layer stays unchanged, no flag-change listeners fire, and OverrideSinkImpl logs an error (custom sources never see an exception).

The file override loader aligns with other SDKs by stamping each flags / segments entry with the key it is listed under (optional key in the document is overwritten), and treats "deleted": true as a load failure—operators remove overrides by deleting the entry or file, not via a deleted marker. Public docs on FileOverrides and OverrideSink describe both rules.

The ordinary file data source is unchanged; a test pins that it still loads deleted entries as placeholders.

Reviewed by Cursor Bugbot for commit 83375f8. Bugbot is set up for automated code reviews on this repo. Configure here.

@kinyoklion

Copy link
Copy Markdown
Member Author

bugbot review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit b3e3725. Configure here.

…or segment

The override layer never holds a deleted item (sdk-specs#288, Requirement
1.1.1.1). A snapshot from any source that contains one is rejected as a
whole: the layer is unchanged, no flag change listener is notified, and the
rejection is logged at Error level with the kind and the key (1.2.8.1).

The file override source fails the load of a file that has a flag or
segment entry marked as deleted, as for a file that cannot be parsed: the
overrides in effect stay, the failure names the file and the key, and the
retry applies (1.3.6.2). An entry with "deleted": false is an ordinary
definition. The file data source keeps accepting deleted entries.
…listed under another key

Evaluation reads an override entry by the key it is listed under, while
the all flags state and events report the definition's own key. The
file source deserialized each entry of "flags" and "segments" as it was,
so an entry with no key property had a null key, which made
allFlagsState throw a NullPointerException to the application, and an
entry pasted under a new key with its old key property was served under
one key and reported under the other. Each entry now takes the key it
is listed under, as the Node, Go and Python override sources key it; a
key property is optional.

The override layer now also rejects a snapshot that lists a flag or
segment under a key other than its own, as it rejects one with a deleted
item: the sink logs the rejection and the overrides in effect stay. Only
a custom override source can supply such a snapshot now. The Python SDK
makes the same check, from the review of its port.
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-reject-deleted branch from b3e3725 to 83375f8 Compare November 13, 2026 00:14
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-contract-tests branch from 67fff40 to 2a32216 Compare November 13, 2026 00:14
@kinyoklion kinyoklion changed the title fix: Reject override snapshots and files that contain a deleted flag or segment fix: Reject deleted override entries, and key each override by the key it is listed under Nov 13, 2026
@cursor cursor Bot added the risk:medium label Nov 13, 2026

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Automated risk review (dry run): Medium

This PR would not be automatically approved. Human approval required.

Why: OverrideLayer, OverrideSinkImpl, and OverrideFileLoader now reject deleted override entries and remap or reject mismatched keys, which changes SDK evaluation of overrides, all-flags state, and events. That is a bounded but non-trivial behavior change with several interacting pieces, so it is Medium.

If you disagree with this review, share this PR in #feedback-agent-approvals or submit a changes-requested review explaining why.

Open in Web View Automation 

Sent by Cursor Approval Agent: Dry-Run PR Approver

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Risk: medium.

Automated risk review (dry run): Medium

This PR would not be automatically approved. Human approval required.

Why: OverrideLayer, OverrideSinkImpl, and OverrideFileLoader change how flag and segment overrides are applied—rejecting deleted entries and mismatched keys, and re-keying file entries—which is SDK evaluation behavior with a bounded blast radius and tests covering the new paths.

If you disagree with this review, share this PR in #feedback-agent-approvals or submit a changes-requested review explaining why.

Open in Web View Automation 

Sent by Cursor Approval Agent: Dry-Run PR Approver

kinyoklion added a commit to launchdarkly/sdk-test-harness that referenced this pull request Nov 16, 2026
#599)

<!-- TERMAG_PR_BODY_BEGIN -->
## Summary

launchdarkly/sdk-specs#288 makes a flag or segment marked as deleted
invalid in an override snapshot and in an override file: Requirements
1.1.1.1, 1.2.8.1 and 1.3.6.2. A file that contains one fails the reload
in the same way as a malformed file, so the last good overrides stay in
effect. Before this change, the SDKs did five different things with such
an entry, and no contract test used one.

The hot reload group of the flag overrides suite gains four tests, which
run in both watching and polling mode:

- **file with a deleted flag retains last good overrides**
- **file with a deleted segment retains last good overrides**
- **file with a deleted flag at startup supplies no overrides until it
is corrected**
- **file with a deleted segment at startup supplies no overrides until
it is corrected**

Each file that holds a deleted entry also changes the override of the
evaluated flag. An SDK that dropped only the deleted entry, instead of
discarding the reload, would serve the changed value and fail the test.

The startup tests correct the file afterwards and wait for the
correction to take effect. A source that stops after a failed initial
load fails them.

An entry with `"deleted": false` is an ordinary definition. Every
full-flag override in the suite already carries that property, because
the harness model serializes it, so no new test covers it.

## Results

The `flag overrides` suite, run against each test service:

| Test service | Result |
|---|---|
| go-server-sdk `v7` | all 8 new tests fail: the SDK applies the file
and serves the changed value |
| go-server-sdk launchdarkly/go-server-sdk#510 | 88 ran, all passed |
| js-core launchdarkly/js-core#2187 | 73 ran, all passed |
| java-core launchdarkly/java-core#259 | 48 ran, all passed (the test
service has no listener capabilities) |
| dotnet-core launchdarkly/dotnet-core#404 | 88 ran, all passed |
| python-server-sdk launchdarkly/python-server-sdk#574, sync and async
services | 88 ran on each, all passed |
| ruby-server-sdk launchdarkly/ruby-server-sdk#493 | 88 ran, all passed
|

All 8 new tests ran on every service; none was skipped.

**Related issues**

- launchdarkly/sdk-specs#288
<!-- TERMAG_PR_BODY_END -->

<!-- termag: box=i-0575d1c54853058d7 project=Overrides -->
<sub>Opened from termag</sub>

[SDK-3578](https://launchdarkly.atlassian.net/browse/SDK-3578)
<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Overview**
> Adds **hot reload contract tests** so override files that include a
flag or segment with `"deleted": true` are treated like **malformed
reloads**: the SDK must keep the last good overrides (or, at startup,
serve **no** overrides until the file is fixed).
> 
> The suite covers **deleted flags and deleted segments** under both
**watching and polling** modes, using a new `docWithDeletedEntry` helper
that pairs a changed `flagValues` entry with a deleted definition so
partial application is detectable.
> 
> After an invalid reload, tests confirm a subsequent valid file still
applies normally.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
cfe421d. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

[SDK-3578]:
https://launchdarkly.atlassian.net/browse/SDK-3578?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant