Audit how we determine whether a file is a "Python source file"

[AlexWaygood]

I noticed in #13682 that there's some inconsistency regarding how we determine whether a file is a "Python source file" currently. In the code for ruff server (and the red-knot port of the server), we take care to do case-insensitive matching when figuring out whether something is a notebook file or not:

ruff/crates/ruff_server/src/session/index.rs

Lines 124 to 126 in 5b4afd3

	} else if Path::new(url.path())
	.extension()
	.map_or(false, |ext| ext.eq_ignore_ascii_case("ipynb"))

ruff/crates/red_knot_server/src/session/index.rs

Lines 75 to 79 in 5b4afd3

	} else if Path::new(url.path())
	.extension()
	.map_or(false, |ext| ext.eq_ignore_ascii_case("ipynb"))
	{
	DocumentKey::Notebook(url)

Elsewhere, however, we mostly use case-sensitive matching:

ruff/crates/ruff_python_ast/src/lib.rs

Lines 91 to 101 in 5b4afd3

	/// Infers the source type from the file extension.
	pub fn try_from_extension(extension: &str) -> Option<Self> {
	let ty = match extension {
	"py" => Self::Python,
	"pyi" => Self::Stub,
	"ipynb" => Self::Ipynb,
	_ => return None,
	};
	Some(ty)
	}

ruff/crates/red_knot_python_semantic/src/module_resolver/path.rs

Lines 41 to 60 in 5b4afd3

	pub(crate) fn push(&mut self, component: &str) {
	if let Some(component_extension) = camino::Utf8Path::new(component).extension() {
	assert!(
	self.relative_path.extension().is_none(),
	"Cannot push part {component} to {self:?}, which already has an extension"
	);
	if self.is_standard_library() {
	assert_eq!(
	component_extension, "pyi",
	"Extension must be `pyi`; got `{component_extension}`"
	);
	} else {
	assert!(
	matches!(component_extension, "pyi" | "py"),
	"Extension must be `py` or `pyi`; got `{component_extension}`"
	);
	}
	}
	self.relative_path.push(component);
	}

For places like the red-knot module resolver, it's likely correct to do case-sensitive matching (Python recognises foo.py as an importable module, but not foo.PY), but in other places it may not be. We should audit the code to make sure we're using case-sensitive matching and case-insensitive matching for file extensions in the correct places. We should also add comments to the places where there might be a subtle reason why case-(in)sensitive matching is required, rather than vice versa.

[MichaReiser]

For places like the red-knot module resolver, it's likely correct to do case-sensitive matching (Python recognises foo.py as an importable module, but not foo.PY

Note: I think that depends on the file system's case sensitivity.

Other than that. I think we should ignore casing when matching files because that's what most desktop environments to. They use the same application to open a test.jpg or test.JPG.

[AlexWaygood]

Here, for example, we might want to continue to do case-sensitive matching, since the rule only applies to the names of Python modules that are intended to be importable by other Python modules:

ruff/crates/ruff_linter/src/rules/pep8_naming/rules/invalid_module_name.rs

Lines 51 to 59 in 5b4afd3

	/// N999
	pub(crate) fn invalid_module_name(
	path: &Path,
	package: Option<&Path>,
	ignore_names: &IgnoreNames,
	) -> Option<Diagnostic> {
	if !PySourceType::try_from_path(path).is_some_and(PySourceType::is_py_file_or_stub) {
	return None;
	}

If a FOO.PY module can't be imported due to the fact that it uses a .PY extension on a case-sensitive file system, then the name of the file isn't really relevant to any PEP-8 concerns, as it's not a module (even if it might be a Python file). The rule is invalid-module-name, not invalid-python-file-name.

[dhruvmanila]

There's also this issue astral-sh/ruff-vscode#574 related to the server although not sure if this affects that.

[hauntsaninja]

Fwiw mypy has some special handling in a few places to be case sensitive even on case insensitive file systems