custom sources for Python distribution binaries

[MeitarR]

Summary

Add an option to overwrite the list of available Python downloads from a local JSON file by using the environment variable UV_PYTHON_DOWNLOADS_JSON_URL

as an experimental support for providing custom sources for Python distribution binaries #8015

related #10203

I probably should make the JSON to be fetched from a remote URL instead of a local file.
please let me know what you think and I will modify the code accordingly.

Test Plan

normal run

root@75c66494ba8b:/# /code/target/release/uv python list
cpython-3.14.0a4+freethreaded-linux-x86_64-gnu    <download available>
cpython-3.14.0a4-linux-x86_64-gnu                 <download available>
cpython-3.13.1+freethreaded-linux-x86_64-gnu      <download available>
cpython-3.13.1-linux-x86_64-gnu                   <download available>
cpython-3.12.8-linux-x86_64-gnu                   <download available>
cpython-3.11.11-linux-x86_64-gnu                  <download available>
cpython-3.10.16-linux-x86_64-gnu                  <download available>
cpython-3.9.21-linux-x86_64-gnu                   <download available>
cpython-3.8.20-linux-x86_64-gnu                   <download available>
cpython-3.7.9-linux-x86_64-gnu                    <download available>
pypy-3.10.14-linux-x86_64-gnu                     <download available>
pypy-3.9.19-linux-x86_64-gnu                      <download available>
pypy-3.8.16-linux-x86_64-gnu                      <download available>
pypy-3.7.13-linux-x86_64-gnu                      <download available>

empty JSON file

root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=/code/crates/uv-python/my-download-metadata.json 
root@75c66494ba8b:/# cat $UV_PYTHON_DOWNLOADS_JSON_URL 
{}
root@75c66494ba8b:/# /code/target/release/uv python list
root@75c66494ba8b:/# 

JSON file with valid version

root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=/code/crates/uv-python/my-download-metadata.json 
root@75c66494ba8b:/# cat $UV_PYTHON_DOWNLOADS_JSON_URL 
{
  "cpython-3.11.9-linux-x86_64-gnu": {
    "name": "cpython",
    "arch": {
      "family": "x86_64",
      "variant": null
    },
    "os": "linux",
    "libc": "gnu",
    "major": 3,
    "minor": 11,
    "patch": 9,
    "prerelease": "",
    "url": "https://github.com/astral-sh/python-build-standalone/releases/download/20240814/cpython-3.11.9%2B20240814-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz",
    "sha256": "daa487c7e73005c4426ac393273117cf0e2dc4ab9b2eeda366e04cd00eea00c9",
    "variant": null
  }
}
root@75c66494ba8b:/# /code/target/release/uv python list
cpython-3.11.9-linux-x86_64-gnu    <download available>
root@75c66494ba8b:/# 

Remote Path

root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=http://a.com/file.json 
root@75c66494ba8b:/# /code/target/release/uv python list
error: Remote python downloads JSON is not yet supported, please use a local path (without `file://` prefix)

[zanieb]

Note this variable will need to be defined in crates/uv-static/src/env_vars.rs

I wonder if we should call it _URL or _URI? We can assume a local path and error with an informative message about the lack of remote file support for now, but we should have a forward-looking name.

We also need to be considerate about how the fetch is implemented, e.g., if we're making a network request it should probably be async which would mean this this function needs to be async (which I would be disappointed by). What kind of change would we need to make to avoid that?

[zanieb]

(I don't think we need remote file support to add this feature — but I would like to know how invasive it would be)

[zanieb]

A remote file also introduces the complexity of caching 🤔 though perhaps we could avoid that since installs don't happen that often.

[MeitarR]

looks like it won't be that bad to make it async.
we would also need to make the following async:

• iter_downloads
• from_request
• PytestRequest::new

But maybe we could use tokio::runtime::Handle::current().block_on or construct a new runtime for the request only.

[zanieb]

Maybe a bit of a silly question, but is there a reason we shouldn't deserialize directly into ManagedPythonDownload? Why the intermediate type?

[MeitarR]

TBH, that's what the AI suggested, but I had the same thought as you, so I tried to derive the Deserialize on ManagedPythonDownload and then on PythonInstallationKey. I understood that target_lexicon's enums do not implement the Deserialize trait, and even if I overcome this, the conversion from JSON will not be as is. So the only way to improve it (the way I see it) is to implement Deserialize on our own and define the JSON structs inside the implementation.

If you see a better way, please let me know

[zanieb]

A panic is probably not acceptable here. We'd need to handle this gracefully, i.e., by surfacing the error to the user.