Skip to content

LD cache bypass after data source 401 #49

Closed

Description

@fvigneault

Describe the bug
During the LaunchDarkly outage of July 10th, 2026, OpenFeature evaluations immediately started returning the call-site default value, even though the underlying LaunchDarkly SDK still had initialized/cached flag data available.

This appears to happen because the OpenFeature provider transitions to ProviderStatus.FATAL when the LaunchDarkly stream connection receives an HTTP 401 (invalid SDK key), and then short-circuits before calling the LaunchDarkly client for evaluation.

To reproduce

  1. Start with a valid SDK key and allow the provider to initialize and cache flag data.
  2. Verify evaluations are working:
    • Direct LaunchDarkly client evaluation returns expected targeted value.
    • OpenFeature evaluation returns the same value.
  3. Trigger stream failure with HTTP 401 (invalid SDK key) on the already-established stream connection.
  4. Observe:
    • LaunchDarkly SDK data source transitions to OFF and stops reconnecting (expected for unrecoverable 401).
    • LaunchDarkly SDK feature store remains initialized with cached data.
    • OpenFeature provider status becomes ProviderStatus.FATAL.
    • Subsequent OpenFeature evaluations return the default value immediately.

Observed behavior

  • Before OFF/401:
    • direct LD: True
    • OpenFeature: True
  • After OFF/401:
    • direct LD: True (from cached data)
    • OpenFeature: False (call-site default)
    • OpenFeature provider status: ProviderStatus.FATAL

From tracing, OpenFeature evaluation appears to return early on provider fatal status (e.g. _assert_provider_status()), so LaunchDarklyProvider.resolve_*() / LDClient.variation_detail() is not called, and LD cached data is never consulted.

Expected behavior
After stream 401 transitions the LD data source to OFF, OpenFeature should ideally still allow evaluations against cached LaunchDarkly data (or provide a configurable behavior), rather than always returning defaults due to provider fatal short-circuiting.

At minimum, users need a way to preserve “last known good” flag behavior in this scenario.

Logs
The stream reported:

Received HTTP error 401 (invalid SDK key) for stream connection - giving up permanently

SDK version

  • launchdarkly-openfeature-server==0.5.1
  • LaunchDarkly Python server SDK: 9.15.1

Language version, developer tools

  • Python (server-side, OpenFeature provider integration)

OS/platform

  • Reproduced in Linux containerized runtime (Kubernetes pods)

Additional context
This behavior caused immediate fallback to defaults across running clients during LaunchDarkly outage conditions, rather than only affecting newly started clients. The key point is that LD cached targeting data still existed, but was bypassed by the OpenFeature provider status handling.

Activity

  1. jsonbailey commented on Jul 17, 2026

    @jsonbailey
    Contributor

    @fvigneault thank you for reporting this. We are reviewing how our SDKs respond to 401s and will include how this cascades to OpenFeature in our discussions.

  2. devin-ai-integration commented on Nov 12, 2026

    @devin-ai-integration
    Contributor

    Hi, sorry for the long silence on this one. The problem you hit during the July outage is fixed. A 401 from the stream no longer puts the provider into a FATAL state, so evaluations keep using the cached flag values instead of returning defaults.

    The fix came in two parts. Version 0.7.0 of this provider changed how it handles the error, and version 9.18.0 of the LaunchDarkly Python SDK now treats a 401 as a temporary interruption and retries. We reproduced your report on provider 0.5.1 with SDK 9.15.1. On the current provider, 0.9.1 (which requires SDK 9.18.3 or later), the provider reports STALE and evaluations return the cached values.

    We recommend upgrading to 0.9.1. I'll close this issue for now. If you still see this behavior on 0.9.1, please reopen it or file a new issue and we'll take another look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions