There is no documentation on what "executing a recipe as a script" means

[pfmoore]

In Writing Recipes in Other Languages, the documentation says

Recipes that start with a #! are executed as scripts, so you can write recipes in other languages

But it's not explained what "executing a recipe as a script" actually means. On Windows, if I have

set windows-shell := ["pwsh.exe", "-NoLogo", "-Command"]

I'm not at all clear what I should expect would happen with a recipe starting with #!. It would be great if this could be documented.

[pfmoore]

Having just looked a bit further, I found Shebang Recipe Execution on Windows.

I'm not clear precisely what this is saying, but I think that it implies that the process is something like the following:

1. The recipe is saved to a temporary file.
2. The contents of the shebang line is inspected. If it contains a slash it's processed by cygpath (giving an error if cygpath isn't available), otherwise it's left unchanged.
3. A command line is constructed using the shebang line and the temporary filename.
4. The command is run.

This appears to explain most of the weird behaviour I've been seeing. The one case I'm not able to explain is if I have a shebang of #!py. With a recipe:

test_py:
    #!py
    print(12)

I get the error:

Unable to create process using 'C:\Users\Gustav\AppData\Local\Programs\Python\Python311\py" C:\Users\Gustav\AppData\Local\Temp\justZ2tJ9P\test_py': The system cannot find the file specified.

error: Recipe `test_py` failed with exit code 101

I can't tell if that error is coming from just or from the executed py command (py.exe interprets an initial shebang line in a file). But I do know that if I save the script as a file, and run it using py, that works perfectly - so I suspect there's something that just is doing which is causing the problem. But I can't tell what. Adding -v doesn't help, and I can't see a way of getting debug output from just.

I'm particularly confused about what is constructing the path C:\Users\Gustav\AppData\Local\Programs\Python\Python311\py. That directory is where my Python installation is located, but there's no py executable in there.

Interestingly, I get the same error if I try to run just -c py. Which suggests that just is somehow special-casing the command py?

I have a recollection of having had a similar discussion preiously, but I can't find any record of it on the tracker 🙁

[pfmoore]

Ah, I found the old discussion, it's #650. But while that discussion is about py, it suggests that using a #!py will cause an infinite recursion¹. While the error is better than that, would it not be even better to just omit the shebang line from the temporary file so the problem couldn't arise? And even if we ignore that, I still don't see any reason why just test_py should work differently from py <a file containing the recipe>...

By the way, I apologise for the way I keep finding more information after posting my questions. I genuinely did try to research before posting - just not hard enough, it seems 🙁

Footnotes

1. Although the fact that #!py doesn't trigger the recursion in a standalone file suggests that since then, they have fixed that issue in the launcher. ↩

[pfmoore]

Actually, it looks like the issue might be std::process::Command. The following test program gives the same error:

use std::process::Command;

fn main() {
    let mut c = Command::new("py");
    c.arg("-V");
    println!("Command: {:#?}", c);
    let s = c.status();
    println!("Status: {:#?}", s);
}

It looks like there are various issues around this - rust-lang/rust#37519 is the first one I found, but that links to rust-lang/rust#87704 with a note that the original code was "replaced as a side effect" of that issue.

I've submitted rust-lang/rust#104358 with the problem, but I think (assuming it doesn't get fixed relatively quickly) the just documentation should include some advice about the issue - whether this is "always include the .exe extension in the program name in the shebang" or something else, I don't honestly know. But I wasted quite a lot of time digging into this, and I think there should be something documented¹ to save other people (or me again, 12 months from now!) having to repeat the process.

Footnotes

1. Even if it's only to note that shebangs don't always work on Windows, and you aren't able to fix that... ↩

[casey]

Thanks for digging into this! Executing shebang recipes is inconsistent between OSes, because Unix-like OSes parse shebang lines differently, and Windows doesn't support them at all, so just parses shebang lines on windows. I opened #1412 to document this.