Bundle arduino-cli with extension

Author: benmcmorran

.gitignore

@@ -6,3 +6,4 @@ node_modules
 src/views/app/sprites-generated
 test/**/c_cpp_properties.json
 *.vsix
+assets/platform/*/arduino-cli

NOTICE_arduino-cli.txt

@@ -0,0 +1,16 @@
+NOTICES AND INFORMATION
+Do Not Translate or Localize
+
+Microsoft makes the arduino-cli open source code available at
+https://3rdpartysource.microsoft.com, or you may send a check or money order for
+US $5.00, including the product name, the open source component name, platform,
+and version number, to:
+
+Source Code Compliance Team
+Microsoft Corporation
+One Microsoft Way
+Redmond, WA 98052
+USA
+
+Notwithstanding any other terms, you may reverse engineer this software to the extent
+required to debug changes to any libraries licensed under the GNU Lesser General Public License.

README.md

@@ -15,7 +15,21 @@ Welcome to the Visual Studio Code extension for **Arduino** <sup>preview</sup> !
 * Integrated Arduino Debugging <sup>New</sup>
 
 ## Prerequisites
-Either the Arduino IDE or Arduino CLI are required.
+Either the Arduino IDE or Arduino CLI are required. The recommended approach is
+to use the version of Arduino CLI that comes bundled with the extension, which
+works out of the box.
+
+### Arduino CLI
+To use the bundled version of Arduino CLI, `arduino.useArduinoCli` should be `true`,
+and `arduino.path` and `arduino.commandPath` should be empty or unset.
+`arduino.useArduinoCli` defaults to `false` while we deprecate support for the
+Arduino IDE, but there will be a prompt when the extension first activates to
+switch to the Arduino CLI.
+
+If you want to use a custom version of Arduino CLI, it can be downloaded from
+the repository's [release page](https://github.com/arduino/arduino-cli/releases/).
+For custom versions, `arduino.path` must be set to the directory containing the
+Arduino CLI executable.
 
 ### Arduino IDE
 The Arduino IDE can be installed the Arduino [download page](https://www.arduino.cc/en/main/software#download).
@@ -24,11 +38,6 @@ The Arduino IDE can be installed the Arduino [download page](https://www.arduino
 - *Note:* Arduino IDE `1.8.7` had some breaking changes, causing board package and library installation failures.  These failures were corrected in `1.8.8` and later.
 - *Note:* Arduino IDE `2.X.Y` is not supported at this time [issue 1477](https://github.com/microsoft/vscode-arduino/issues/1477)
 
-### Arduino CLI
-The Arduino CLI can be downloaded from the repository's [release page](https://github.com/arduino/arduino-cli/releases/tag/0.13.0)
-- The extension has only been tested with v0.13.0.
-- If you use the CLI you will have to set `arduino.path` since the CLI does not have a default path.
-
 ## Installation
 Open VS Code and press <kbd>F1</kbd> or <kbd>Ctrl</kbd> + <kbd>Shift</kbd> + <kbd>P</kbd> *or* <kbd>Cmd</kbd> + <kbd>Shift</kbd> + <kbd>P</kbd> to open command palette, select **Install Extension** and type `vscode-arduino`.
 
@@ -71,9 +80,9 @@ This extension provides several commands in the Command Palette (<kbd>F1</kbd> o
 ## Options
 | Option | Description |
 | --- | --- |
-| `arduino.useArduinoCli` | Whether to use the Arduino CLI (`true`) or the Arduino IDE (`false`) -- defaults to `false`. If using `true`,  make sure to set the `arduino.path` and `arduino.commandPath` values correctly. |
-| `arduino.path`  | Path to the Arduino installation.  You can use a custom version of Arduino by modifying this setting to include the full path. Example: `C:\\Program Files\\Arduino` for Windows, `/Applications` for Mac, `/home/<username>/Downloads/arduino-1.8.1` for Linux. (Requires a restart after change). The default value is automatically detected from your Arduino IDE installation path. To use the Arduino CLI, use the path that contains the `arduino-cli` executable (e.g. `/usr/local/bin`). |
-| `arduino.commandPath` | Path to an executable (or script) relative to `arduino.path`. The default value is `arduino_debug.exe` for Windows, `Contents/MacOS/Arduino` for Mac and `arduino` for Linux, You also can use a custom launch script to run Arduino by modifying this setting. (Requires a restart after change) Example: `run-arduino.bat` for Windows, `Contents/MacOS/run-arduino.sh` for Mac and `bin/run-arduino.sh` for Linux. To use the Arduino CLI, use `arduino-cli`. |
+| `arduino.useArduinoCli` | Whether to use the Arduino CLI (`true`) or the Arduino IDE (`false`) -- defaults to `false`. If using `true`, either leave the `arduino.path` and `arduino.commandPath` values unset to use the bundled version of Arduino CLI, or point them at a custom version of Arduino CLI. Note that a future version of the extension will change this default to `true` and remove support for Arduino IDE. |
+| `arduino.path`  | Path to the Arduino installation.  You can use a custom version of Arduino by modifying this setting to include the full path. Example: `C:\\Program Files\\Arduino` for Windows, `/Applications` for Mac, `/home/<username>/Downloads/arduino-1.8.1` for Linux. (Requires a restart after change). The default value is automatically detected from your Arduino IDE installation path. To use the Arduino CLI, use the path that contains the `arduino-cli` executable (e.g. `/usr/local/bin`), or leave it unset to use the bundled version of Arduino CLI. |
+| `arduino.commandPath` | Path to an executable (or script) relative to `arduino.path`. The default value is `arduino_debug.exe` for Windows, `Contents/MacOS/Arduino` for Mac and `arduino` for Linux, You also can use a custom launch script to run Arduino by modifying this setting. (Requires a restart after change) Example: `run-arduino.bat` for Windows, `Contents/MacOS/run-arduino.sh` for Mac and `bin/run-arduino.sh` for Linux. To use the bundled version of Arduino CLI, leave this option unset. To use a custom version of Arduino CLI, use `arduino-cli`. |
 | `arduino.additionalUrls` | Additional Boards Manager URLs for 3rd party packages as a string array. The default value is empty. |
 | `arduino.logLevel` | CLI output log level. Could be info or verbose. The default value is `"info"`. |
 | `arduino.clearOutputOnBuild` | Clear the output logs before uploading or verifying. Default value is `false`. |
@@ -91,8 +100,7 @@ The following Visual Studio Code settings are available for the Arduino extensio
 
 ```json
 {
-    "arduino.path": "C:/Program Files (x86)/Arduino",
-    "arduino.commandPath": "arduino_debug.exe",
+    "arduino.useArduinoCli": true,
     "arduino.logLevel": "info",
     "arduino.allowPDEFiletype": false,
     "arduino.enableUSBDetection": true,
@@ -105,7 +113,6 @@ The following Visual Studio Code settings are available for the Arduino extensio
     "arduino.defaultBaudRate": 115200
 }
 ```
-*Note:* You only need to set `arduino.path` in Visual Studio Code settings, other options are not required.
 
 The following settings are as per sketch settings of the Arduino extension. You can find them in
 `.vscode/arduino.json` under the workspace.

build/assets.json

@@ -0,0 +1,33 @@
+{
+  "version": "0.29.0",
+  "assets": {
+    "arduino-cli_0.29.0_Linux_64bit.tar.gz": {
+      "hash": "febc558bb36a9aeb881ebcce0af6fff955234afda14435791ca41daf492c9cac",
+      "platforms": ["linux-x64"]
+    },
+    "arduino-cli_0.29.0_Linux_ARM64.tar.gz": {
+      "hash": "ce5cd44808462461c88ab3d2441363a7fa08035a1d0cc300823d357929223ae6",
+      "platforms": ["linux-arm64"]
+    },
+    "arduino-cli_0.29.0_Linux_ARMv7.tar.gz": {
+      "hash": "b2b98400a643324219f8fca1a4eeef3a129d6fe87356e4d126b1b56f88635c94",
+      "platforms": ["linux-armhf"]
+    },
+    "arduino-cli_0.29.0_macOS_64bit.tar.gz": {
+      "hash": "d2e5aa10c66440b27a55deca54d12fc6071bb4963423706aeae97bc68a5597de",
+      "platforms": ["darwin-x64"]
+    },
+    "arduino-cli_0.29.0_macOS_ARM64.tar.gz": {
+      "hash": "19da609c94383c98748ec79a751ac19dab38ce012880d9bb3e0e7e1d70128107",
+      "platforms": ["darwin-arm64"]
+    },
+    "arduino-cli_0.29.0_Windows_32bit.zip": {
+      "hash": "366a6d354083db5386ff5e0d92043d23940e11cba9c3a5cf46ccea5a849e21ea",
+      "platforms": ["win32-ia32"]
+    },
+    "arduino-cli_0.29.0_Windows_64bit.zip": {
+      "hash": "e7473a72e40c7d7a1b29c4a3ec4646f9b9163e46c35fd78e34f1e315c2dd1e38",
+      "platforms": ["win32-x64"]
+    }
+  }
+}

build/downloadAssets.js

@@ -0,0 +1,67 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT license.
+
+// This script downloads and verifies platform-specific arduino-cli binaries
+// from GitHub releases. The release is specified by the "version" field in
+// assets.json.
+
+const { execSync } = require("child_process");
+const { createHash } = require("crypto");
+const { readFileSync, mkdirSync, rmSync, renameSync } = require("fs");
+const { resolve } = require("path");
+
+function run(command) {
+  console.log(command);
+  execSync(command);
+}
+
+const config = require('./assets.json');
+
+for (const asset in config.assets) {
+  if (Object.hasOwnProperty.call(config.assets, asset)) {
+    const platforms = config.assets[asset].platforms;
+    const hash = config.assets[asset].hash;
+    for (const platform of platforms) {
+      const directory = resolve(__dirname, "..", "assets", "platform", platform);
+      const destination = resolve(directory, asset);
+
+      // Download the asset.
+      run([
+        "curl",
+        `https://github.com/arduino/arduino-cli/releases/download/${config.version}/${asset}`,
+        "--location",
+        `--output-dir ${directory}`,
+        `--remote-name`,
+        "--silent",
+        "--show-error",
+      ].join(" "));
+
+      // Verify the hash.
+      const actualHash = createHash("sha256")
+        .update(readFileSync(destination))
+        .digest("hex");
+      if (actualHash !== hash) {
+        throw new Error(
+          `Hash mismatch for ${asset} on ${platform}. Expected ${hash} but got ${actualHash}.`
+        );
+      }
+
+      // Extract to an "arduino-cli" directory.
+      const extractDirectory = resolve(directory, "arduino-cli");
+      mkdirSync(extractDirectory, { recursive: true });
+      run(`tar -xf ${destination} -C ${extractDirectory}`);
+
+      // Remove the downloaded archive. We don't need to ship it.
+      rmSync(destination);
+
+      // VSIX signing will silently strip any extensionless files. Artificially
+      // add a ".app" extension to extensionless executables.
+      const executable = resolve(extractDirectory, "arduino-cli");
+      try {
+        renameSync(executable, `${executable}.app`);
+      } catch {
+        // The file might not exist. This is expected for Windows.
+      }
+    }
+  }
+}

cgmanifest.json

@@ -0,0 +1,76 @@
+{
+  "$schema": "https://json.schemastore.org/component-detection-manifest.json",
+  "version": 1,
+  "registrations": [
+    {
+      "component": {
+        "type": "other",
+        "other": {
+          "name": "arduino-cli (Linux x64)",
+          "version": "0.29.0",
+          "downloadUrl": "https://github.com/arduino/arduino-cli/releases/download/0.29.0/arduino-cli_0.29.0_Linux_64bit.tar.gz",
+        }
+      }
+    },
+    {
+      "component": {
+        "type": "other",
+        "other": {
+          "name": "arduino-cli (Linux ARM64)",
+          "version": "0.29.0",
+          "downloadUrl": "https://github.com/arduino/arduino-cli/releases/download/0.29.0/arduino-cli_0.29.0_Linux_ARM64.tar.gz",
+        }
+      }
+    },
+    {
+      "component": {
+        "type": "other",
+        "other": {
+          "name": "arduino-cli (Linux ARMv7)",
+          "version": "0.29.0",
+          "downloadUrl": "https://github.com/arduino/arduino-cli/releases/download/0.29.0/arduino-cli_0.29.0_Linux_ARMv7.tar.gz",
+        }
+      }
+    },
+    {
+      "component": {
+        "type": "other",
+        "other": {
+          "name": "arduino-cli (macOS x64)",
+          "version": "0.29.0",
+          "downloadUrl": "https://github.com/arduino/arduino-cli/releases/download/0.29.0/arduino-cli_0.29.0_macOS_64bit.tar.gz",
+        }
+      }
+    },
+    {
+      "component": {
+        "type": "other",
+        "other": {
+          "name": "arduino-cli (macOS ARM64)",
+          "version": "0.29.0",
+          "downloadUrl": "https://github.com/arduino/arduino-cli/releases/download/0.29.0/arduino-cli_0.29.0_macOS_ARM64.tar.gz",
+        }
+      }
+    },
+    {
+      "component": {
+        "type": "other",
+        "other": {
+          "name": "arduino-cli (Windows x86)",
+          "version": "0.29.0",
+          "downloadUrl": "https://github.com/arduino/arduino-cli/releases/download/0.29.0/arduino-cli_0.29.0_Windows_32bit.zip",
+        }
+      }
+    },
+    {
+      "component": {
+        "type": "other",
+        "other": {
+          "name": "arduino-cli (Windows x64)",
+          "version": "0.29.0",
+          "downloadUrl": "https://github.com/arduino/arduino-cli/releases/download/0.29.0/arduino-cli_0.29.0_Windows_64bit.zip",
+        }
+      }
+    }
+  ]
+}

package-lock.json

@@ -473,7 +473,7 @@
         "arduino.useArduinoCli": {
           "type": "boolean",
           "default": false,
-          "markdownDescription": "Use Arduino CLI installed instead of Arduino IDE. `#arduino.path#` must be set, as there is no default path for 'arduino-cli'. (Requires a restart after change)"
+          "markdownDescription": "Use Arduino CLI installed instead of Arduino IDE. If `#arduino.path#` and `#arduino.commandPath#` are not set, the extension will use a version of Arduino CLI bundled with the extension. (Requires a restart after change)"
         },
         "arduino.path": {
           "type": "string",
@@ -596,7 +596,7 @@
   },
   "scripts": {
     "build": "gulp build --mode=production",
-    "postinstall": "cd ./src/views && npm install",
+    "postinstall": "node ./build/downloadAssets.js && cd ./src/views && npm install",
     "test": "gulp test"
   },
   "extensionDependencies": [

package.json

@@ -4,11 +4,14 @@
 import * as os from "os";
 import * as path from "path";
 import * as WinReg from "winreg";
+import * as vscode from 'vscode';
+import { chmod } from "fs/promises";
 import * as util from "../common/util";
 
 import { resolveArduinoPath } from "../common/platform";
 
 import { VscodeSettings } from "./vscodeSettings";
+import * as Logger from "../logger/logger";
 
 export interface IArduinoSettings {
     arduinoPath: string;
@@ -22,6 +25,7 @@ export interface IArduinoSettings {
     defaultBaudRate: number;
     preferences: Map<string, string>;
     useArduinoCli: boolean;
+    usingBundledArduinoCli: boolean;
     defaultTimestampFormat: string;
     analyzeOnSettingChange: boolean;
     reloadPreferences(): void;
@@ -44,7 +48,9 @@ export class ArduinoSettings implements IArduinoSettings {
 
     private _defaultTimestampFormat: string;
 
-    public constructor() {
+    private _usingBundledArduinoCli: boolean = false;
+
+    public constructor(private readonly _context: vscode.ExtensionContext) {
     }
 
     public async initialize() {
@@ -136,7 +142,7 @@ export class ArduinoSettings implements IArduinoSettings {
 
     public get commandPath(): string {
         const platform = os.platform();
-        if (platform === "darwin") {
+        if (platform === "darwin" && !this._usingBundledArduinoCli) {
             return path.join(util.resolveMacArduinoAppPath(this._arduinoPath, this._useArduinoCli), path.normalize(this._commandPath));
         } else {
             return path.join(this._arduinoPath, path.normalize(this._commandPath));
@@ -162,6 +168,10 @@ export class ArduinoSettings implements IArduinoSettings {
         return this._useArduinoCli;
     }
 
+    public get usingBundledArduinoCli() {
+        return this._usingBundledArduinoCli;
+    }
+
     public get defaultBaudRate() {
         return this._defaultBaudRate;
     }
@@ -225,14 +235,43 @@ export class ArduinoSettings implements IArduinoSettings {
         }
     }
 
+    private readonly bundledArduinoCliName: { [platform: string]: string } = {
+        'darwin-arm64': 'arduino-cli.app',
+        'darwin-x64': 'arduino-cli.app',
+        'linux-arm64': 'arduino-cli.app',
+        'linux-armhf': 'arduino-cli.app',
+        'linux-x64': 'arduino-cli.app',
+        'win32-ia32': 'arduino-cli.exe',
+        'win32-x64': 'arduino-cli.exe',
+    };
+
+    private async bundledArduinoCliPath(): Promise<string | undefined> {
+        const platform = await util.getPlatform();
+        const name = this.bundledArduinoCliName[platform];
+        if (!name) return undefined;
+        return this._context.asAbsolutePath(path.join('assets', 'platform', platform, 'arduino-cli', name));
+    }
+
     private async tryResolveArduinoPath(): Promise<void> {
         // Query arduino path sequentially from the following places such as "vscode user settings", "system environment variables",
         // "usual software installation directory for each os".
         // 1. Search vscode user settings first.
         const configValue = VscodeSettings.getInstance().arduinoPath;
         if (!configValue || !configValue.trim()) {
-            // 2 & 3. Resolve arduino path from system environment variables and usual software installation directory.
-            this._arduinoPath = await Promise.resolve(resolveArduinoPath());
+            // 2. Resolve arduino path from the bundled arduino-cli, if CLI support is enabled.
+            const bundledPath = await this.bundledArduinoCliPath();
+            if (bundledPath && this._useArduinoCli && !this._commandPath) {
+                // The extension VSIX stipped the executable bit, so we need to set it.
+                // 0x755 means rwxr-xr-x (read and execute for everyone, write for owner).
+                await chmod(bundledPath, 0o755);
+                this._usingBundledArduinoCli = true;
+                Logger.traceUserData("using-bundled-arduino-cli");
+                this._arduinoPath = path.dirname(bundledPath);
+                this._commandPath = path.basename(bundledPath);
+            } else {
+                // 3 & 4. Resolve arduino path from system environment variables and usual software installation directory.
+                this._arduinoPath = await Promise.resolve(resolveArduinoPath());
+            }
         } else {
             this._arduinoPath = configValue;
         }