Skip to content

Promoting experimental_requirement_cycles from experimental #1663

Open

Description

@Philip-Murzynowski

🚀 feature request

Relevant Rules

I have been making use of the experimental_requirement_cycles parameter for the pip_parse rule and greatly benefiting from it. I wanted to ask if there is a release plan or anything I could look into to help incorporate it into a release.

Description

The experimental feature is introduced here in the CHANGELOG. I saw that recommendation for requesting a feature to be taken out of experimental was to open an issue as noted in the support.md.

Describe the solution you'd like

As this is my first activity in this repository, please let me know if you would like more information.

Describe alternatives you've considered

Activity

  1. arrdem commented on Jan 23, 2024

    @arrdem
    Contributor

    As the primary author of that feature I'm 50/50 on promoting it to stable. It's a fine solution for making true dependency cycles work (eg. a <-> b in all cases of extras requirements). I'm less convinced that it's a good solution for making requirements using extras work since we have to convert what should be optional dependencies into strong dependencies.

    Consider a[] requiring b with b requiring a, and a[c] requiring c with c requiring a. py_requirement("a") is now ambiguous. Does the user intend a[c]? a[]? Do we intend the requirements.txt locks to function as virtualenvs from which everything* is installed or as a dependency solution set from which only the needed requirements should be installed?

    Personally I prefer the latter. I think having to merge all the Airflow optional dependencies into a single requirement group isn't a great solution because it means that any artifact which requires Airflow takes a dependency on the entire group of requirements. It would be better if say the mysql extra wasn't always implied because py_requirement("airflow") now means the entire group with all extras.

    There's plenty of room to critique setuptools/pip here for extras really behaving as just sneaky transitive deps not proper package configuration https://discuss.python.org/t/provide-information-on-installed-extras-in-package-metadata/15385.

    There's the related problem of needing to merge packaging cycles when there are multiple, again see the newer Airflow packaging. Is that really the right behavior? Or could we do something clever to break these cycles into separate components which would behave more reasonably as optional extras?

  2. aignas commented on Jul 1, 2024

    @aignas
    Collaborator

    Some thoughts about a better solution here: #1728 (comment)

    Not sure yet how to integrate the current experimental_index_url with the code snippet in there, but it would be an alternative that would not require the users to specify the experimental_requirement_cycles at all and it would be my preferred solution.

  3. gholms commented on Oct 21, 2024

    @gholms
    Contributor

    This came up as a potential solution for linking type stub wheels with their corresponding main wheels to ensure they are present for rules which run static type checkers like mypy. At present there is no clear way to do this; the community varies from requiring one to supply the transitive closure of stub wheels to individual test rules to automagically building a fake site-packages directory to throwing our hands up in the air. The most appropriate place to do this is in pip_parse, the code which actually provides wheels to bazel.

    While we could add a bespoke mechanism for metadata wheels like these, experimental_requirement_cycles is already a means of grouping wheels together which can handle this use case. Perhaps switching to a more generic name such as whl_groups as we stabilize this would be appropriate.

    This again raises the question of how to handle optional dependencies, but from a different perspective from extras. Type stubs are more or less useful in an "all-or-nothing" way, making build settings an attractive option. A solution which supports select statements would facilitate this quite well. That said, we don't necessarily have to block stabilization on solving this problem. It is enough to simply consider it in the design.

  4. aignas commented on Dec 10, 2024

    @aignas
    Collaborator

    Just to complete the option list for above stub proposal - #2425 has landed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions