Follow
Share
to
Twitter
Discord
GitHubDeno is created by the original developer of Node.js, Ryan Dahl, to address what he called 10 things I regret about Node.js. It did away with the NPM and the infamous node_modules. Deno is a single binary executable to run applications written in TypeScript and JavaScript.
However, while TypeScript and JavaScript are suitable for the majority of web applications, they could be inadequate for computationally intensive tasks, such as neural network training and inference, machine learning, and cryptography. In fact, Node.js itself often needs to use native libraries for those tasks (e.g., to use openssl for cryptography).
Without an NPM-like system to incorporate native modules, how do we write server-side applications that require native performance on Deno? WebAssembly is here to help! In this article, we will write high performance functions in Rust, compile them into WebAssembly, and run them inside your Deno application.
WebAssembly support in Deno
WebAssembly is a lightweight virtual machine designed to execute portable bytecode at near native speed. You can compile Rust or C/C++ functions to WebAssembly bytecode, and access those functions from TypeScript. For some tasks, it could be much faster than executing equivalent functions authored in TypeScript itself. For example, this IBM study found that Rust and WebAssembly could improve Node.js execution speed by 1200% to 1500% for certain data processing algorithms.
Deno uses the Google V8 engine internally. V8 is not only a JavaScript runtime, but also a WebAssembly virtual machine. WebAssembly is supported in Deno out of the box. Deno provides an API for your TypeScript application to call functions in WebAssembly. The Deno WASI component enables WebAssembly applications to access the underlying operating system resources, such as the file system. In this article, I will teach you how to write high performance Deno applications in Rust and WebAssembly.
Set up
The first step of course is to install Deno! On most systems, it is just a single command.
$ curl -fsSL https://deno.land/x/install/install.sh | sh
Since we are writing functions in Rust, you also need to install Rust language compilers and tools.
$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Finally, the ssvmup tool automates the build process and generates all the artifacts to make it easy for your Deno applications to call Rust functions. Again, a single command installs the ssvmup dependency.
$ curl https://raw.githubusercontent.com/second-state/ssvmup/master/installer/init.sh -sSf | sh
The ssvmup uses wasm-bindgen to automatically generate the “glue” code between JavaScript and Rust source code so that they can communicate using their native data types. Without it, the function arguments and return values would be limited to very simple types (i.e., 32 bit integers) supported natively by WebAssembly. For example, strings or arrays would not be possible without ssvmup and wasm-bindgen.
Hello world
To get started, let's look into a hello world example adopted from the Deno hello world example. You can get the hello world source code and application template from GitHub.
The Rust function is in the src/lib.rs file and simply prepends “hello” to an input string. Notice that the say() function gets annotated with #[wasm_bindgen], allowing ssvmup to generate the necessary “plumbing” to call it from TypeScript.
#[wasm_bindgen]
pub fn say(s: &str) -> String {
let r = String::from("hello ");
return r + s;
}
The Deno application exists in the deno/server.ts file. The application imports the Rust say() function from the pkg/functions_lib.js file, which gets generated by the ssvmup tool. The functions_lib.js file name depends on the Rust project name defined in the Cargo.toml file.
import { serve } from "https://deno.land/std/http/server.ts";
import { say } from '../pkg/functions_lib.js';
type Resp = {
body: string;
}
const s = serve({ port: 8000 });
console.log("http://localhost:8000/");
for await (const req of s) {
let r = {} as Resp;
r.body = say (" World\n");
req.respond(r);
}
Now, let's run ssvmup to build the Rust function into a Deno WebAssembly function.
$ ssvmup build --target deno
After ssvmup successfully completes, you can inspect the pkg/functions_lib.js file to see how the Deno WebAssembly API gets used to execute the compiled WebAssembly file pkg/functions_lib.wasm.
Next, run the Deno application. Deno requires permissions to read the file system since it needs to load the WebAssembly file, and to access the network since it needs to receive and respond to HTTP requests.
$ deno run --allow-read --allow-net --allow-env --unstable deno/server.ts
In another terminal window, you can now access the Deno web application to make it say hello over an HTTP connection!
$ curl http://localhost:8000/
hello World
Calling Rust functions from TypeScript
The starter template project includes several more elaborate examples to show how to pass complex data between the Deno TypeScript and Rust functions. Here are some additional Rust functions in src/lib.rs. Notice that each of them is annotated with #[wasm_bindgen].
#[wasm_bindgen]
pub fn obfusticate(s: String) -> String {
(&s).chars().map(|c| {
match c {
'A' ..= 'M' | 'a' ..= 'm' => ((c as u8) + 13) as char,
'N' ..= 'Z' | 'n' ..= 'z' => ((c as u8) - 13) as char,
_ => c
}
}).collect()
}
#[wasm_bindgen]
pub fn lowest_common_denominator(a: i32, b: i32) -> i32 {
let r = lcm(a, b);
return r;
}
#[wasm_bindgen]
pub fn sha3_digest(v: Vec<u8>) -> Vec<u8> {
return Sha3_256::digest(&v).as_slice().to_vec();
}
#[wasm_bindgen]
pub fn keccak_digest(s: &[u8]) -> Vec<u8> {
return Keccak256::digest(s).as_slice().to_vec();
}
TypeScript program deno/test.ts shows how to call the Rust functions. As you can see String and &str are simply strings in TypeScript, i32 are numbers, and Vec<u8> or &[8] are JavaScript Uint8Array. You can pass complex objects between TypeScript and Rust using JSON strings.
import { say, obfusticate, lowest_common_denominator, sha3_digest, keccak_digest, create_line } from '../pkg/functions_lib.js';
const encoder = new TextEncoder();
console.log( say("SSVM") );
console.log( obfusticate("A quick brown fox jumps over the lazy dog") );
console.log( lowest_common_denominator(123, 2) );
console.log( sha3_digest(encoder.encode("This is an important message")) );
console.log( keccak_digest(encoder.encode("This is an important message")) );
After running ssvmup to build the Rust library, running deno/test.ts in the Deno runtime produces the following output:
$ ssvmup build --target deno
... Building the wasm file and JS shim file in pkg/ ...
$ deno run --allow-read --allow-env --unstable deno/test.ts
hello SSVM
N dhvpx oebja sbk whzcf bire gur ynml qbt
246
Uint8Array(32) [
87, 27, 231, 209, 189, 105, 251, 49,
... ...
]
Uint8Array(32) [
126, 194, 241, 200, 151, 116, 227,
... ...
]
Accessing system resources from Rust functions
The WebAssembly System Interface (WASI) enables WebAssembly applications to access the host operating system. With a capability-based security model, WASI allows WebAssembly application to access random numbers, clock, environment variables, and the file system. It is especially relevant for server-side applications. WASI is supported on Deno. You can check out this example project.
Rust library functions
You can use WASI functions to access the random number generator, standard output, and execution arguments in Rust library functions. As we described above, those library functions can be called from TypeScript and they can exchange data with TypeScript via wasm-bindgen. Below is a Rust library functions example. Those functions make Rust std API calls that are compiled into WASI.
use wasm_bindgen::prelude::*;
use rand::prelude::*;
use std::env;
#[wasm_bindgen]
pub fn get_random_i32() -> i32 {
let x: i32 = random();
return x;
}
#[wasm_bindgen]
pub fn echo(content: &str) -> String {
println!("Printed from Deno wasi: {}", content);
return content.to_string();
}
#[wasm_bindgen]
pub fn print_args() -> i32 {
println!("The args are as follows.");
for argument in env::args() {
println!("{}", argument);
}
return 0;
}
Due to the current limitations of the Rust compiler, Rust library functions cannot yet access the file system using the WASI API. In order to use full WASI in server-side Rust functions, you need to use Node.js and the SSVM. Checkout this WASI tutorial.
The TypeScript program to call these functions is as follows.
import { get_random_i32, echo, print_args } from '../pkg/wasi_example_lib.js';
console.log( "My random number is: ", get_random_i32() );
echo("Hello Deno");
print_args();
Building and executing give us the following results.
$ ssvmup build --target deno
... Building the wasm file and JS shim file in pkg/ ...
$ deno run --allow-read --allow-env --unstable deno/test.ts hello Deno
My random number is: -1283069866
Printed from Deno wasi: Hello Deno
The args are as follows.
hello
Deno
Standalone Rust programs
With WASI, you can also write standalone Rust applications, compile them into WebAssembly, and then execute the application from Deno command line. In this case, the Rust program does have access to file system through WASI. Here is an example Rust program.
use std::fs;
use std::fs::File;
use std::io::{Write, Read};
use std::env;
fn main() {
println!("The env vars are as follows.");
for (key, value) in env::vars() {
println!("{}: {}", key, value);
}
create_file("/hello.txt", "Hello WASI");
println!("{}", read_file("/hello.txt") );
del_file("/hello.txt");
}
fn create_file(path: &str, content: &str) {
let mut output = File::create(path).unwrap();
output.write_all(content.as_bytes()).unwrap();
}
fn read_file(path: &str) -> String {
let mut f = File::open(path).unwrap();
let mut s = String::new();
match f.read_to_string(&mut s) {
Ok(_) => s,
Err(e) => e.to_string(),
}
}
fn del_file(path: &str) {
fs::remove_file(path).expect("Unable to delete");
}
The ssvmup tool generates the JavaScript files to start and execute the WebAssembly program from Deno.
$ ssvmup build --target deno
... Building the wasm file and JS shim file in pkg/ ...
$ deno run --allow-read --allow-write --allow-env --unstable pkg/deno_wasi_example.js
The env vars are as follows.
XDG_SESSION_TYPE: tty
... ...
SHELL: /bin/bash
Hello WASI
Conclusion
In this article, I showed you how to run high performance WebAssembly programs in Deno. The most versatile approach is to compile Rust functions into WebAssembly library functions, and then call the library functions from TypeScript. That gives us the best of both worlds: the Deno ecosystem and the Rust ecosystem.
An alternative approach is to compile Rust programs into standalone WebAssembly applications, and then start the application using Deno command line tools inside Deno runtime host (i.e., the V8 runtime). The Deno runtime makes the Rust program portable and safe to run.
In both scenarios, WASI provides the WebAssembly functions and programs access to the host operating system. As WebAssembly evolves and gains use cases on the server-side, we believe that Deno, WebAssembly, and WASI will become an important software stack for future developers!