Skip to content

Commit 25d7a3c

Browse files
PragmaTwicemakslevental
PragmaTwice
and
makslevental
authored
Add playground pages for WASM MLIR python bindings (#264)
This PR adds two playground pages for WASM-version MLIR python bindings. #### jupyterlite One is jupyterlite with the pyodide kernel. The mlir-python-bindings wheel package (latest daily package produced from workflows) is attached but will not be loaded automatically, you can type the following code in a code cell to load: ```python import piplite await piplite.install("mlir-python-bindings") # or piplite.install(["mlir-python-bindings", "numpy"]) ``` A notebook (mlir-python-starter.ipynb) is attached to demonstrate basic usage of MLIR python bindings (more examples can be added later?). #### console Another is a terminal emulator (console) with mlir-python-bindings (latest daily package) installed and automatically loaded, i.e. you can just type `import mlir...` to start coding. --- After a manual trigger of the page deploying workflow, we can now preview these changes: - jupyterlite: https://llvm.github.io/eudsl/jupyter/ - console: https://llvm.github.io/eudsl/console/ Try it! #### Screenshots <img width="2735" height="1363" alt="image" src="https://github.com/user-attachments/assets/4d670507-0943-41b7-83be-2c2c97a740c6" /> <img width="1703" height="952" alt="image" src="https://github.com/user-attachments/assets/1752731b-3f01-443a-b91c-4c96dc320c41" /> --- The index page remains unchanged: https://llvm.github.io/eudsl #### About eudsl-python-extras: I tried to also add the latest eudsl-python-extras package, but it seems that the sdist package (.tar.gz) doesn't work well with `pyodide.loadPackage` and jupyterlite pyodide kernel's piplite config. So I make it a TODO for now. --------- Co-authored-by: Maksim Levental <maksim.levental@gmail.com>
1 parent 47a30ff commit 25d7a3c

File tree

5 files changed

+453
-0
lines changed

5 files changed

+453
-0
lines changed

‎.github/workflows/deploy_pip_page.yml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,11 @@ jobs:
2929
- name: Setup Pages
3030
uses: actions/configure-pages@v5
3131

32+
- name: Check out repository
33+
uses: actions/checkout@v4
34+
with:
35+
submodules: false
36+
3237
- name: Download pip links
3338
shell: bash
3439
run: |
@@ -41,6 +46,37 @@ jobs:
4146
sed -i.bak 's/\/llvm\/eudsl/https:\/\/github.com\/llvm\/eudsl/g' index.html
4247
mkdir -p page && mv index.html page
4348
49+
- name: Fetch latest WASM wheel
50+
run: |
51+
pip download mlir-python-bindings --plat pyodide_2024_0_wasm32 --no-deps --python-version 3.12 -f https://llvm.github.io/eudsl
52+
echo "MLIR_PYTHON_WHEEL_NAME=$(ls mlir_python_bindings*)" >> $GITHUB_ENV
53+
54+
- name: Create WASM console page
55+
run: |
56+
mkdir -p page/console
57+
58+
cp pages/console/index.html page/console/index.html
59+
cp $MLIR_PYTHON_WHEEL_NAME page/console/mlir_python_bindings-0.0.1-cp312-cp312-pyodide_2024_0_wasm32.whl
60+
61+
- name: Setup Python
62+
uses: actions/setup-python@v5
63+
with:
64+
python-version: '3.12'
65+
66+
- name: Create WASM jupyterlite page
67+
run: |
68+
mkdir -p page/jupyter
69+
70+
python -m pip install -r pages/jupyter/requirements.txt
71+
jupyter lite build \
72+
--contents pages/jupyter/contents \
73+
--output-dir page/jupyter \
74+
--piplite-wheels $MLIR_PYTHON_WHEEL_NAME
75+
76+
- name: Show directory structure
77+
run: |
78+
tree page
79+
4480
- uses: geekyeggo/delete-artifact@v5
4581
with:
4682
name: github-pages

‎README.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,14 @@ Currently, there are five components:
3131
**When in doubt about this prefix**, it is everything up until `ir` when you import your bindings, e.g., in `import torch_mlir.ir`,
3232
`torch_mlir` is the `EUDSL_PYTHON_EXTRAS_HOST_PACKAGE_PREFIX` for the torch-mlir bindings.
3333

34+
## Wasm Playground
35+
36+
We currently provide two online playgrounds where you can try out the WebAssembly version of the MLIR Python bindings directly in your browser:
37+
38+
* https://llvm.github.io/eudsl/jupyter/ – A JupyterLite instance with a Pyodide kernel. You can install the MLIR Python bindings with: `await piplite.install("mlir-python-bindings")`.
39+
40+
* https://llvm.github.io/eudsl/console/ – A Pyodide-based REPL with `mlir-python-bindings` preloaded. Just run: `from mlir.ir import *` to start coding.
41+
3442
## Getting started
3543

3644
Python wheels of all the tools are available at the [`eudsl` release page](https://github.com/llvm/eudsl/releases/tag/eudsl).

‎pages/console/index.html‎

Lines changed: 304 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,304 @@
1+
<!doctype html>
2+
<!-- This page is modified from https://github.com/pyodide/pyodide/blob/a53c17fd8ef87e79c7d3bfbb1262ca7ff6718ef2/src/templates/console.html. -->
3+
<!-- It is licensed under Mozilla Public License 2.0. Check https://github.com/pyodide/pyodide/blob/main/LICENSE. -->
4+
<html>
5+
<head>
6+
<meta charset="UTF-8" />
7+
<script src="https://cdn.jsdelivr.net/npm/jquery"></script>
8+
<script src="https://cdn.jsdelivr.net/npm/jquery.terminal@2.35.2/js/jquery.terminal.min.js"></script>
9+
<script src="https://cdn.jsdelivr.net/npm/jquery.terminal@2.35.2/js/unix_formatting.min.js"></script>
10+
<link
11+
href="https://cdn.jsdelivr.net/npm/jquery.terminal@2.35.2/css/jquery.terminal.min.css"
12+
rel="stylesheet"
13+
/>
14+
<link
15+
href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 100 100'><text y='.9em' font-size='90'>🐍</text></svg>"
16+
rel="icon"
17+
/>
18+
<style>
19+
.terminal {
20+
--size: 1.5;
21+
--color: rgba(255, 255, 255, 0.8);
22+
}
23+
.noblink {
24+
--animation: terminal-none;
25+
}
26+
body {
27+
background-color: black;
28+
}
29+
#jquery-terminal-logo {
30+
color: white;
31+
border-color: white;
32+
position: absolute;
33+
top: 7px;
34+
right: 18px;
35+
z-index: 2;
36+
}
37+
#jquery-terminal-logo a {
38+
color: gray;
39+
text-decoration: none;
40+
font-size: 0.7em;
41+
}
42+
#loading {
43+
display: inline-block;
44+
width: 50px;
45+
height: 50px;
46+
position: fixed;
47+
top: 50%;
48+
left: 50%;
49+
border: 3px solid rgba(172, 237, 255, 0.5);
50+
border-radius: 50%;
51+
border-top-color: #fff;
52+
animation: spin 1s ease-in-out infinite;
53+
-webkit-animation: spin 1s ease-in-out infinite;
54+
}
55+
56+
@keyframes spin {
57+
to {
58+
-webkit-transform: rotate(360deg);
59+
}
60+
}
61+
@-webkit-keyframes spin {
62+
to {
63+
-webkit-transform: rotate(360deg);
64+
}
65+
}
66+
</style>
67+
</head>
68+
<body>
69+
<div id="jquery-terminal-logo">
70+
<a href="https://terminal.jcubic.pl/">jQuery Terminal</a>
71+
</div>
72+
<div id="loading"></div>
73+
<script>
74+
"use strict";
75+
76+
function sleep(s) {
77+
return new Promise((resolve) => setTimeout(resolve, s));
78+
}
79+
80+
async function main() {
81+
let indexURL = "https://cdn.jsdelivr.net/pyodide/v0.27.7/full/";
82+
const urlParams = new URLSearchParams(window.location.search);
83+
const buildParam = urlParams.get("build");
84+
if (buildParam) {
85+
if (["full", "debug", "pyc"].includes(buildParam)) {
86+
indexURL = indexURL.replace(
87+
"/full/",
88+
"/" + urlParams.get("build") + "/",
89+
);
90+
} else {
91+
console.warn(
92+
'Invalid URL parameter: build="' +
93+
buildParam +
94+
'". Using default "full".',
95+
);
96+
}
97+
}
98+
const { loadPyodide } = await import(indexURL + "pyodide.mjs");
99+
// to facilitate debugging
100+
globalThis.loadPyodide = loadPyodide;
101+
102+
let term;
103+
globalThis.pyodide = await loadPyodide({
104+
stdin: () => {
105+
let result = prompt();
106+
echo(result);
107+
return result;
108+
},
109+
});
110+
111+
// load mlir-python-bindings wheel package
112+
await pyodide.loadPackage("mlir_python_bindings-0.0.1-cp312-cp312-pyodide_2024_0_wasm32.whl");
113+
114+
let { repr_shorten, BANNER, PyodideConsole } =
115+
pyodide.pyimport("pyodide.console");
116+
BANNER =
117+
`Welcome to the Pyodide ${pyodide.version} terminal emulator 🐍\n` +
118+
`Try \`from mlir.ir import *\` to start your MLIR python journey.\n` +
119+
BANNER;
120+
const pyconsole = PyodideConsole(pyodide.globals);
121+
122+
const namespace = pyodide.globals.get("dict")();
123+
const await_fut = pyodide.runPython(
124+
`
125+
import builtins
126+
from pyodide.ffi import to_js
127+
128+
async def await_fut(fut):
129+
res = await fut
130+
if res is not None:
131+
builtins._ = res
132+
return to_js([res], depth=1)
133+
134+
await_fut
135+
`,
136+
{ globals: namespace },
137+
);
138+
namespace.destroy();
139+
140+
const echo = (msg, ...opts) =>
141+
term.echo(
142+
msg
143+
.replaceAll("]]", "&rsqb;&rsqb;")
144+
.replaceAll("[[", "&lsqb;&lsqb;"),
145+
...opts,
146+
);
147+
148+
const ps1 = ">>> ";
149+
const ps2 = "... ";
150+
151+
async function lock() {
152+
let resolve;
153+
const ready = term.ready;
154+
term.ready = new Promise((res) => (resolve = res));
155+
await ready;
156+
return resolve;
157+
}
158+
159+
async function interpreter(command) {
160+
const unlock = await lock();
161+
term.pause();
162+
// multiline should be split (useful when pasting)
163+
for (const c of command.split("\n")) {
164+
const escaped = c.replaceAll(/\u00a0/g, " ");
165+
const fut = pyconsole.push(escaped);
166+
term.set_prompt(fut.syntax_check === "incomplete" ? ps2 : ps1);
167+
switch (fut.syntax_check) {
168+
case "syntax-error":
169+
term.error(fut.formatted_error.trimEnd());
170+
continue;
171+
case "incomplete":
172+
continue;
173+
case "complete":
174+
break;
175+
default:
176+
throw new Error(`Unexpected type ${ty}`);
177+
}
178+
// In JavaScript, await automatically also awaits any results of
179+
// awaits, so if an async function returns a future, it will await
180+
// the inner future too. This is not what we want so we
181+
// temporarily put it into a list to protect it.
182+
const wrapped = await_fut(fut);
183+
// complete case, get result / error and print it.
184+
try {
185+
const [value] = await wrapped;
186+
if (value !== undefined) {
187+
echo(
188+
repr_shorten.callKwargs(value, {
189+
separator: "\n<long output truncated>\n",
190+
}),
191+
);
192+
}
193+
if (value instanceof pyodide.ffi.PyProxy) {
194+
value.destroy();
195+
}
196+
} catch (e) {
197+
if (e.constructor.name === "PythonError") {
198+
const message = fut.formatted_error || e.message;
199+
term.error(message.trimEnd());
200+
} else {
201+
throw e;
202+
}
203+
} finally {
204+
fut.destroy();
205+
wrapped.destroy();
206+
}
207+
}
208+
term.resume();
209+
await sleep(10);
210+
unlock();
211+
}
212+
213+
term = $("body").terminal(interpreter, {
214+
greetings: BANNER,
215+
prompt: ps1,
216+
completionEscape: false,
217+
completion: function (command, callback) {
218+
callback(pyconsole.complete(command).toJs()[0]);
219+
},
220+
keymap: {
221+
"CTRL+C": async function (event, original) {
222+
pyconsole.buffer.clear();
223+
term.enter();
224+
echo("KeyboardInterrupt");
225+
term.set_command("");
226+
term.set_prompt(ps1);
227+
},
228+
TAB: (event, original) => {
229+
const command = term.before_cursor();
230+
// Disable completion for whitespaces.
231+
if (command.trim() === "") {
232+
term.insert("\t");
233+
return false;
234+
}
235+
return original(event);
236+
},
237+
},
238+
});
239+
window.term = term;
240+
pyconsole.stdout_callback = (s) => echo(s, { newline: false });
241+
pyconsole.stderr_callback = (s) => {
242+
term.error(s.trimEnd());
243+
};
244+
term.ready = Promise.resolve();
245+
pyodide._api.on_fatal = async (e) => {
246+
if (e.name === "Exit") {
247+
term.error(e);
248+
term.error("Pyodide exited and can no longer be used.");
249+
} else {
250+
term.error(
251+
"Pyodide has suffered a fatal error. Please report this to the Pyodide maintainers.",
252+
);
253+
term.error("The cause of the fatal error was:");
254+
term.error(e);
255+
term.error("Look in the browser console for more details.");
256+
}
257+
await term.ready;
258+
term.pause();
259+
await sleep(15);
260+
term.pause();
261+
};
262+
263+
const searchParams = new URLSearchParams(window.location.search);
264+
if (searchParams.has("noblink")) {
265+
$(".cmd-cursor").addClass("noblink");
266+
}
267+
268+
let idbkvPromise;
269+
async function getIDBKV() {
270+
if (!idbkvPromise) {
271+
idbkvPromise = await import(
272+
"https://unpkg.com/idb-keyval@5.0.2/dist/esm/index.js"
273+
);
274+
}
275+
return idbkvPromise;
276+
}
277+
278+
async function mountDirectory(pyodideDirectory, directoryKey) {
279+
if (pyodide.FS.analyzePath(pyodideDirectory).exists) {
280+
return;
281+
}
282+
const { get, set } = await getIDBKV();
283+
const opts = {
284+
id: "mountdirid",
285+
mode: "readwrite",
286+
};
287+
let directoryHandle = await get(directoryKey);
288+
if (!directoryHandle) {
289+
directoryHandle = await showDirectoryPicker(opts);
290+
await set(directoryKey, directoryHandle);
291+
}
292+
const permissionStatus =
293+
await directoryHandle.requestPermission(opts);
294+
if (permissionStatus !== "granted") {
295+
throw new Error("readwrite access to directory not granted");
296+
}
297+
await pyodide.mountNativeFS(pyodideDirectory, directoryHandle);
298+
}
299+
globalThis.mountDirectory = mountDirectory;
300+
}
301+
window.console_ready = main();
302+
</script>
303+
</body>
304+
</html>

0 commit comments

Comments
 (0)