Kiesel Devlog #6: Catching up :^)
Published on 2024-01-29.Happy new year! Well, new is a stretch, but it's been a while since I last did one of these. This post is gonna be all over the place, but I want to get you up to speed before focusing on more specific things again :^)
TL;DR
The last (exactly!) three months can be summed up as follows:
Kiesel has three new contributors! Considering the total number is four this is a lot 🎉
Ali Mohammad Pur Fri Oct 27 02:53:42 2023 +0330 cli+language: Use zigline to provide a nice syntax-highlighted REPL Andrew Kaster Fri Dec 22 12:00:35 2023 -0500 builtins: Bump comptime limit in reg_exp to 20,000 Dominique Liberda Fri Dec 29 01:44:25 2023 +0100 builtins: Implement Promise.withResolvers()273 commits to
main, with tons of new features and bug fixestest262 went from 35.3% to 45.8%, almost half way there 📈
Let's take a closer look :^)
A Better REPL
I already hinted towards this in the last devlog:
CxByte managed to yakbait himself into writing a line editor in Zig and kindly integrated it into Kiesel's REPL!
It still had a bug or two early on but things seem stable now — if you happen to notice any issues, please report them upstream.
Additionally, I added a preamble to the REPL to make it more informative instead of dropping you into a prompt right away:
$ kiesel
Kiesel 0.1.0 [Zig 0.12.0-dev.2341+92211135f] on linux
Use Ctrl+D to exit.
>macOS CI Builds
Zig is excellent at cross compiling for other targets, but for the longest time I didn't add macOS CI builds because libgc includes mach-o/getsect.h from the macOS SDK, which needs to be installed on the build host. Vendoring the file didn't seem desirable, so I simply ignored the problem — macOS users could still build from source.
After finding and integrating this Zig package, the problem solved itself and there is now a kiesel-macos-aarch64 build on files.kiesel.dev!
Thanks to Andrew for fixing the build after I unknowingly broke it, which lead to me thinking about this again :^)
Promise.withResolvers()
This is a new function (stage 4 in November 2023) that implements the commonly used deferred promise pattern. The proposal was championed by a colleague of mine, and I ended up contributing the upstream TypeScript definitions to help getting it ready for production use.
The spec text is seven lines so I resisted writing an implementation myself, knowing it could be a good first task for someone wanting to contribute to Kiesel — and it was! At 37c3 Domi and I sat down and did some Zig hacking together :^)
Atomics, DataView, TypedArray
I finally got around to implementing these three:
TypedArray: still completely unoptimized (using the regular object get/set code paths), but at least memory-wise a raw blob of bytes is already an improvement over using a regular array (which, in Kiesel, is still backed by a hashmap of property keys to JS values)DataView: Really happy about this,ArrayBufferis largely useless if you can't read and write bytesAtomics: only vaguely useful as there are no shared array buffers yet — but hey, free test262 points
Internationalization
ECMA-402 (aka Intl) was never that high on my list of priorities for Kiesel (and still isn't), but it's nice to have those APIs I suppose!
Unlike LibJS I opted to not implement my own ICU replacement and chose ICU4X instead (using their C API as ICU4X is written in Rust 🦀).
Generally speaking you can get away with using C APIs in Zig directly, but in this case the code looked horrendous:
I decided to write a simple wrapper as a standalone library — and that's how icu4zig was born!
So far I've used it to implement locale handling, adding other objects will require more work on icu4zig first.
Expand to see the full list of newly implemented Intl builtins 📝
Intl.getCanonicalLocales()Intl.Locale.prototype.baseNameIntl.Locale.prototype.calendarIntl.Locale.prototype.caseFirstIntl.Locale.prototype.collationIntl.Locale.prototype.hourCycleIntl.Locale.prototype.languageIntl.Locale.prototype.maximize()Intl.Locale.prototype.minimize()Intl.Locale.prototype.numberingSystemIntl.Locale.prototype.numericIntl.Locale.prototype.regionIntl.Locale.prototype.scriptIntl.Locale.prototype.toString()Intl.Locale.prototype[@@toStringTag]Intl[@@toStringTag]
Annex B Has Joined the Chat
B Additional ECMAScript Features for Web Browsers
The ECMAScript language syntax and semantics defined in this annex are required when the ECMAScript host is a web browser. The content of this annex is normative but optional if the ECMAScript host is not a web browser.
Implementing it means:
- I get more test262 points
- You can now use Kiesel to implement a web browser. You're welcome.
Jokes aside, while these language features are hopefully not being relied upon in new code, there's existing JS out there that needs them. I've added a compile time flag to make them opt out (-Denable-annex-b=false), and given the implementation is not too invasive I'm happy to have it around :^)
So far this includes builtins and and the [[IsHTMLDDA]] internal slot (if you don't know what that means: good for you!), but none of the syntax changes.
![Demo of the [[IsHTMLDDA]] internal slot in the Kiesel REPL](/images/posts/kiesel-devlog-6/annex-b-ishtmldda.png)
Expand to see the full list of newly implemented Annex B builtins 📝
Date.prototype.getYear()Date.prototype.setYear()Date.prototype.toGMTString()RegExp.prototype.compile()String.prototype.substr()String.prototype.anchor()String.prototype.big()String.prototype.blink()String.prototype.bold()String.prototype.fixed()String.prototype.fontcolor()String.prototype.fontsize()String.prototype.italics()String.prototype.link()String.prototype.small()String.prototype.strike()String.prototype.sub()String.prototype.sup()String.prototype.trimLeft()String.prototype.trimRight()
Performance 🚀🚀🚀
While Kiesel remains largely unoptimized — make it work, make it right (we are here), make it fast — I've finally started profiling and implemented some quick wins:
Bytecode of functions is now cached after they are first called, no need to regenerate the same one on every call.
Determining the UTF-16 string length from the internal UTF-8 representation was incredibly heavy and is now done only once per string.
Asking a string for one of its code units is now implemented using an iterator that doesn't require allocating a list.
I copied an optimization for property lookups from LibJS, which in turn copied it from JavaScriptCore — when doing property lookups on primitives we don't need to create an object wrapper as per the spec, we can directly start looking on the respective prototype object, no own properties can exist on the wrapper object anyway!
The result can be seen in these flame graphs (before and after):
In general, optimizing is both necessary and heaps of fun, so I'll definitely do more of that! Special shout-out to CanadaHonk who was the source of yakbait for most of these while benchmarking against Porffor :^)
Modules
Not much happened here and you still can't fully import modules in Kiesel, but I got some basics up and running:
- Parsing of various (not all!) import/export declarations
- Parsing and evaluation of dynamic
import()calls - Module environments and namespace objects
- Hooking up module resolution in the CLI, i.e. turning module specifiers into paths and reading files from disk
Summary
A few other bits that I didn't mention yet:
- Added support for function parameter initializers (default values),
\xand\ustring escape sequences, and for-in/of statements (although the patch for that one is rather ugly) - I've done a decently large refactor to rip printing and bytecode generation out of the AST itself, those are separate now. Not only did it become unwieldy, but this also makes me more confident doing a rewrite or the bytecode VM (it's kinda bad lol), or adding other backends (e.g. outputting 3rd party IR, AOC, JIT, …)
Expand to see the full list of newly implemented builtins 📝
Atomics.add()Atomics.and()Atomics.exchange()Atomics.isLockFree()Atomics.load()Atomics.or()Atomics.store()Atomics.sub()Atomics.xor()Atomics[@@toStringTag]DataView.prototype.bufferDataView.prototype.byteLengthDataView.prototype.byteOffsetDataView.prototype.getFoo()DataView.prototype.setFoo()DataView.prototype[@@toStringTag]decodeURI()decodeURIComponent()encodeURI()encodeURIComponent()Function.prototype[@@hasInstance]()Map.groupBy()Object.fromEntries()Object.groupBy()Promise.all()Promise.allSettled()Promise.any()Promise.race()Promise.withResolvers()RegExp.prototype[@@match]()RegExp.prototype[@@split]()String.fromCharCode()String.fromCodePoint()String.prototype.codePointAt()String.prototype.includes()String.prototype.lastIndexOf()String.prototype.match()String.prototype.replace(), sans substitutionsString.prototype.replaceAll()String.prototype.split()String.prototype.substring()String.prototype.toLowerCase()String.prototype.toUpperCase()String.prototype.trim()String.prototype.trimEnd()String.prototype.trimStart()Symbol.prototype.descriptionTypedArray.BYTES_PER_ELEMENTTypedArray.prototype.BYTES_PER_ELEMENT%TypedArray%.from()%TypedArray%.of()%TypedArray%.prototype.at()%TypedArray%.prototype.buffer%TypedArray%.prototype.byteLength%TypedArray%.prototype.byteOffset%TypedArray%.prototype.copyWithin()%TypedArray%.prototype.entries()%TypedArray%.prototype.every()%TypedArray%.prototype.fill()%TypedArray%.prototype.filter()%TypedArray%.prototype.find()%TypedArray%.prototype.findIndex()%TypedArray%.prototype.findLast()%TypedArray%.prototype.findLastIndex()%TypedArray%.prototype.forEach()%TypedArray%.prototype.includes()%TypedArray%.prototype.indexOf()%TypedArray%.prototype.join()%TypedArray%.prototype.keys()%TypedArray%.prototype.lastIndexOf()%TypedArray%.prototype.length%TypedArray%.prototype.map()%TypedArray%.prototype.reduce()%TypedArray%.prototype.reduceRight()%TypedArray%.prototype.reverse()%TypedArray%.prototype.slice()%TypedArray%.prototype.some()%TypedArray%.prototype.sort()%TypedArray%.prototype.subarray()%TypedArray%.prototype.toLocaleString()%TypedArray%.prototype.toReversed()%TypedArray%.prototype.toSorted()%TypedArray%.prototype.toString()%TypedArray%.prototype.values()%TypedArray%.prototype.with()%TypedArray%.prototype[@@iterator]()%TypedArray%.prototype[@@toStringTag]%TypedArray%[@@species]
Thanks for reading, until next time! :^)
