The differences in paths for Windows `.msi` install and `janet --install`

[pepe]

Hello, I started to experiment with the spork/pm module on Windows and found one problem, which I think is more of an oversight than an error.

In the .msi configuration file on the path tools/msi/janet.wsx, there are two places I consider relevant:

1. https://github.com/janet-lang/janet/blob/master/tools/msi/janet.wxs#L78, where the directory structure of the installation is specified.
2. https://github.com/janet-lang/janet/blob/master/tools/msi/janet.wxs#L163, where the environmental variables are specified.

As you can see, JANET_PATH is set to <root>\Janet\Library and JANET_BINPATH is set to <root>\Janet\bin. So far, so good. JPM is respecting these variables, as can be seen at https://github.com/janet-lang/jpm/blob/d93b7c243645d31410a81fb9ab8f7a5e5608f0d0/jpm/make-config.janet#L39. But with Janet's install bundle/add-bin, the path is just the concatenation of JANET_PATH and the string "/bin", so it does not respect the JANET_BINPATH as can be seen at https://github.com/janet-lang/janet/blob/master/src/boot/boot.janet#L4501. In my opinion, we should also respect the environmental variable here.

I am not sure about the other operating systems. I am using Windows 11 with all the updates installed, and Janet 1.39.1 from the official release .msi installer.

[pepe]

Following further research and consultation, it was recommended that the old jpm method for handling environmental variables be phased out with the bundle/* toolkit. And after more thought, it is a good thing.

Yet, shouldn't we then set the PATH in the .msi installer to both paths? At least for the time being?

[sogaiu]

I'm wary of an additional modification to a user's PATH without the user being aware of it [1].

I think as mentioned in this commit:

Establishing a convention for scripts is beneficial for various tools.
However, we do not install scripts on anyones PATH - instead they go
to a self contained (dyn syspath) /bin folder which could be added to
path, or symlinks could be added.

However, I can see that it's more inconvenient on Windows to adjust PATH and symlinking is not really an option for many folks (still?).

As an alternative, may be it's possible to implement an option for the .msi built with WiX such that one could choose at install time (via a checkbox if via a GUI, or a switch if via the .msi) whether to have the modification be performed. I'm not sure what WiX can do and how much work this would be, but perhaps it is worth some investigation.

I think the default should be to not make an additional modification though to avoid surprises.

May be there is some sample here that would be easy to modify to get started [2].

---

[1] Coincidentally, I recently wrote in this comment, speculation about why the current approach is taken (tldr; mostly collision avoidance).
[2] Also of interest might be this post.

[sogaiu]

I've looked into this a bit more and IIUC the way the current installer works, PATH is appended to (so added to at the end).

Since this is an appending operation, it doesn't seem like it would lead to masking of existing programs (though after uninstallation, it could unintentionally "reveal" certain programs that had been "masked").

I think some of the code in question is

janet/tools/msi/janet.wxs

Line 174 in 8e7b1e9

<Environment Id="PATH_PERUSER" Name="PATH" Value="[BinDir]" Action="set" Permanent="no" System="no" Part="last"/>

Note the use of Part="last", which according to these docs means:

last
  This value is appended.

---

Since the installer appends (intead of replacing or prepends), may be instead of:

Value="[BinDir]"

the following sort of thing can be specified:

Value="[BinDir];[LibraryDir]bin"

---

On a side note, according to this announcement it appears that:

...there will be no future public releases of WiX v3. To be clear, that means there will be no future public releases of WiX v3, even if a security vulnerability is discovered. Apologies for the bold, but we don’t want anybody to miss it. (FireGiant’s Extended WiX Toolset support offers ongoing security support.)

They do seem to have newer versions including some kind of help in upgrading, though it seems they've moved to a "let's phase stuff out on a schedule and try to force people to upgrade" model.

[sogaiu]

I tried out the idea above (just modifying the appended value for PATH) and it seems to work.

If this is the kind of change we want to make, then it seems we could either:

• Change Value=... to Value=[BinDir];[LibraryDir]bin" in two places (here and here) -OR-
• Make another one of these with an appropriate name and associated value and apply that instead to fill out Value=... in the aforementioned two spots

I suppose the latter approach is more maintainable over time, but we would need an appropriate new name :)

[pepe]

After thinking it through, I don’t see a reason to bring JANET_BINPATH back into the code. A cleaner approach is to modify the Windows installer to optionally append both [BinDir] and [LibraryDir]\bin to the PATH. That makes Janet --install scripts work out of the box, while still maintaining the default conservative approach.

At the same time, the MSI should transition from WiX v3 (which is dead) to WiX v6 via v4. That gives us a maintained toolchain and a good place to add the PATH option without hacks. Keeps the bundle/* story intact, and makes Windows life easier.

I volunteer for the PR in this direction :-).

[sogaiu]

FWIW, what I did here is basically to arrange for both [BinDir] and [LibraryDir]bin (no backslash needed) to be added to PATH.

I've also gotten the WiX stuff to work with version 6.0.2.

Will put stuff in a branch if you want to have a look.

[sogaiu]

Ok, here is a commit that works locally for using WiX 6.0.2 to build an installer file (i.e. the .msi). I think there's a chance it won't work for CI. That may need to be figured out still.

Note that this commit does NOT do PATH manipulation -- for that see further down.

To get this to work my system had:

• wixtoolset 6.0.2 (NOT wxitoolset3) via scoop
• Visual Studio bits as described here -- though definitely don't follow the steps about Wix there because those mention version 3.1x.
• dotnet-sdk via scoop - WiX docs seemed to suggest Visual Studio might be enough, but I think dotnet-sdk was also needed in my case...who knows...

Some potentially helpful commands include:

• building janet.exe: build_win.bat
• building the .msi installer: build_win.bat dist
• cleaning up: build_win.bat clean
• installing from the .msi: msiexec /i janet-1.40.0-windows-x64-installer.msi
• uninstalling via the .msi: msiexec /x janet-1.40.0-windows-x64-installer.msi

It may be important to use the "x66 Native Tools Command Prompt" for the commands listed above. I got failures when using ordinary prompts.

---

For the PATH-related changes, please see this commit.

---

Possibly useful references:

• Quick Start | Create your first installation package - helped in making it clear that we now need some .wixproj file
• wix.exe command-line reference
  • build subcommand - instead of candle.exe and light.exe, wix.exe build ... seems to be used now
  • convert subcommand - used this initally on janet.wxs to "upgrade"
• WiX Toolset Issues - found an issue here that helped in coming up with the -ext portion of the wix.exe build ... invocation. not sure which issue it was though 😅

[pepe]

You are the best! I will definitely take a look at your codes. Thank you!

[sogaiu]

Hmm, I looked into janet's CI a bit -- that looks like it uses the windows-latest runner image. According to this section that seems to come with version 3.14 of WiX (may need to scroll down a bit).

May be it's better to hold off on upgrading the WiX parts?

[pepe]

Yep. Another hassle not needed. Thank you for your work. Yet, your WiX v6 works for me.

[sogaiu]

For future reference, the tools list of windows-latest currently has both InnoSetup and NSIS (both also available via scoop).

Here is a reddit discussion that touches on some alternatives and there is a bit of comparison too.

[sogaiu]

I started looking into what some other programming languages use.

So far I'm getting the impression that many seem to offer .exe as an obviously locatable option, though in some cases, there is a .msi available (or "embedded"?).

---

• Go - .msi (and .zip, but no .exe?)
• Haskell - .exe (ghcup.exe), according to this section
• Perl (StrawberryPerl) - .exe and .msi (traces of pre-4.x WiX - as I see mention of candle.exe and light.exe)
• Python - .exe, but it appears to contain one or more .msi files. WiX appears to be in use in some fashion and at version 3.14
• Ruby - .exe, according to text in this section - though .7z files appear to be available too
• Rust - .exe (rustup-init.exe) and .msi

[pepe]

As it was at the beginning, we were also provided only the .exe.

[sogaiu]

I guess NSIS used to be used...and it looks like there was at least one good reason to switch away.

Did not remember at all!