A Practical Edge.js Example

(posted in tech)

Out of Date Notice (Nov. 2019): The Ficdown web server no longer uses Edge.js to compile stories. Instead it spins up the Ficdown compiler as an async out-of-band process. We now return you to your regularly scheduled programming.

A side project I’ve been working on during my free time for a while now is Ficdown, a system for creating choice-based interactive fiction in Markdown. The meat of the project is in the form of a “compiler” that will take a Ficdown source file and generate an epub that can be read on any hyperlink-capable e-reader.

I wrote the compiler in C#, mainly because we coders are nothing if not creatures of habit, and that’s what I’ve spent the majority of my professional career coding in. At work, we’ve recently made the switch from writing and running our .NET stuff on Windows machines and using Visual Studio to doing everything in Mono on Linux machines. Freeing myself from the bloat of Visual Studio in a Windows VM and switching to coding entirely in Vim directly on my Linux host was like a massive breath of fresh air—so much so that I decided to make the same switch at home.

I could (and may at some point) fill a whole post or two about the joys of coding C# on Linux in Vim, but for now suffice it to say it’s everything I love about C#, .NET, and (obviously) Vim, minus everything I hate about Windows and Visual Studio. The best of both worlds. Cake and eating it too. Yadda yadda yadda.

The only thing I was unsure about, as far as Ficdown was concerned, was how I would integrate my compiler into a website without a Windows server to host it on (my current server is a Centos box on DigitalOcean). At work we’re using ServiceStack which has a self-hosting mode, meaning your web app lives inside a console application that does its own connection handling, request threading, and so on. I could have gone that way with the older free version of ServiceStack, or I could try to figure out how to get XSP working with a standard MVC project…

But come on! This is Linux! There are so many cool ways to create websites on Linux that don’t involve maintaining cumbersome csproj and sln files and having to compile and deploy crap. I just want to spin up a light-weight site in something like Node or even a static generator like Jekyll. The only fancy part would be where it needs to post uploaded source files to my C# compiler and then give a link to the generated output.

Enter Edge.js

A static site generator was probably taking it a step too far—I can’t imagine any feasible way to get that going. There’s no client-side .NET runtime written in Javascript that could execute my Ficdown compiler that I’m aware of. Node, however, was a definite possibility. We use Node at work for a bunch of stuff. I’ve worked on some of those projects. I sorta like Node.

Then I remembered about Edge.js.

Now I freakin’ love Node.

Edge.js is basically an interop provider that allows you to run .NET code natively within a Node application (and vice versa). Using that, I could create my website as a simple Node/Express app, and make calls directly to my Ficdown compiler wherever I needed to. So deliciously brilliant! Here’s how I did it:

How I Did It

I’ll skip over the prerequisites since those steps would be ultra boring, change depending on which distro you’re on, and you should be able to work that out for yourself anyway. Basically you need Node, npm, Mono, and (in my case) CoffeeScript installed globally. That’s right, I prefer CoffeeScript over raw JavaScript. Deal with it. Unless you also prefer CoffeeScript, in which case have a high-five!

The web app itself started off about as simply as a Node/Express app can get, which is a thing of beauty.

express = require 'express'
app = express()

app.set 'views', __dirname + '/views'
app.set 'view engine', 'jade'
app.use express.static __dirname + '/public'

app.get '/', (req, res) -> res.render 'index', active: index: true
app.get '/learn', (req, res) -> res.render 'learn', active: learn: true
app.get '/tutorial', (req, res) -> res.render 'tutorial', active: learn: true, tutorial: true
app.get '/reference', (req, res) -> res.render 'reference', active: learn: true, reference: true
app.get '/write', (req, res) -> res.render 'write', active: write: true
app.get '/compile', (req, res) -> res.render 'compile', active: write: true, compile: true
app.get '/playground', (req, res) -> res.render 'playground', active: write: true, playground: true
app.get '/play', (req, res) -> res.render 'play', active: play: true
app.get '/source', (req, res) -> res.render 'source', active: source: true

All of the actions just render their respective views, passing some information that the header uses to highlight menus and stuff. Pretty much a static site. It could probably be done more efficiently but it hasn’t gotten cumbersome enough yet for me to work out how.

The only really interesting view from a back-end standpoint is the compile page, which contains a form that allows someone to upload a Ficdown source file. Here’s essentially what the form looks like (I’m using the Jade templating engine):

form(method='POST' enctype='multipart/form-data' action='/compile')

    label(for='source') Ficdown Source File:
    input#source(type='file' name='source')

    label(for='author') Author Name:
    input#author(type='text' name='author')

    input#debug(type='checkbox' name='debug')
    label(for='debug') Include player state debug information.

  p: input(type='submit' value='Compile')

To wire it up, I created a new file called compiler.coffee and required it at the top of my app with compiler = require './compiler', then added this line to the bottom:

app.post '/compile', compiler

Then I started working on the new handler for posts to the /compile endpoint:

edge = require 'edge'
formidable = require 'formidable'

compile = (req, res) ->
  form = new formidable.IncomingForm()
  form.parse req, (err, fields, files) ->
    if fields.author == '' or files.source.size == 0
      res.render 'compile'

        # user didn't fully fill out the form
        # insert logic to render error on the view here

      data =
        author: fields.author
        debug: if fields.debug == 'on' then true else false
        source: files.source.path

      # need to actually do the compiling here

module.exports = compile

Simple enough so far, especially using Formidable to handle the form parsing for me.

Although, this is where I hit my first snag.

On my machine running Arch Linux, just the mere act of requiring the edge module causes mono to segfault. I’m not sure if it’s because the version of Node or Mono in the Arch repos are greater than what edge has been tested against, but that’s my best guess. I opened an issue about it.

At any rate, to continue I ended up firing up a Centos Docker container with all of the prereqs loaded up in order to continue. The app ran fine in the Centos environment.

Making It Edgier

The Edge docs cover how to integrate .NET code into Node pretty well. Basically for Node to invoke a method in .NET, the .NET function must be of the form Func<object, Task<object>>. In simpler speak, it must be a function that takes an object as its parameter, and returns a Task<object> value (which is basically a function that runs asynchronously and returns an object when it’s done). My first thought was ‘oh great, do I have to add a method with that signature to my Ficdown library now?’ Then I realized how silly that was—Edge supports writing .NET functions and classes inline. I could write the function with that signature right in my compiler.coffee file and have that function invoke the methods already in my Ficdown library as they were intended to be called.

It took a few iterations and different attempts to get my DLL referenced correctly, but here’s what I eventually landed on in my compiler.coffee file:

do_compile = edge.func
  source: ->
    using System;
    using System.Linq;
    using System.IO;
    using System.Collections.Generic;
    using System.Threading.Tasks;
    using Ficdown.Parser;
    using Ficdown.Parser.Model.Story;
    using Ficdown.Parser.Model.Parser;
    using Ficdown.Parser.Render;

    public class Startup {
      public async Task<object> Invoke(dynamic input) {
        var response = new Dictionary<string, object>();

        // getting the form input from node here:

        var author = (string) input.author;
        var debug = (bool) input.debug;
        var source = (string) input.source;

        // using .NET to read the uploaded file content:

        var storyText = File.ReadAllText(source);

        var parser = new FicdownParser();
        ResolvedStory story;
          story = parser.ParseStory(storyText);
          response.Add("success", true);
          response.Add("warnings", story.Orphans.Select(o => string.Format("Warning (line {0}): {1} {2} is unreachable", o.LineNumber, o.Type, o.Name)).ToArray());
        catch(FicdownException ex)
          story = null;
          response.Add("success", false);
          response.Add("error", ex.ToString());

        // generating the epub to a temp file:

        if(story != null)
          var rend = new EpubRenderer(author);
          var output = Guid.NewGuid();
          response.Add("output", output);
          rend.Render(story, string.Format("/tmp/{0}.epub", output), debug);

        return response;
  references: [ __dirname + '/lib/Ficdown.Parser.dll' ]

compile = (req, res) ->
  form = new formidable.IncomingForm()
  form.parse req, (err, fields, files) ->
    if fields.author == '' or files.source.size == 0
      res.render 'compile'
      # snipped out logic for error rendering
      data =
        author: fields.author
        debug: if fields.debug == 'on' then true else false
        source: files.source.path

      do_compile data, (error, result) ->
        if error?
          res.render 'compile'
          # snipped out logic for error rendering
          res.render 'compile'
          # snipped out logic for success rendering

I removed some of the boring boilerplate code that passes error and success messages to the views for the sake of brevity. Here’s a breakdown of some of the cooler things that are going on here:

  • You write your inline .NET code inside of comments. When the app is run, the code gets compiled. If there are any compile-time errors in your code you’ll see them right away when you try to run your Node app.

  • I add all of the form input values to a Javascript object called data, and then pass that as the parameter to my do_compile function. Edge then marshalls that object into a .NET dynamic object c that I call input.

  • Edge does have a way to marshall binary data between Node and .NET, so in theory I could have read the uploaded file using Node and passed its contents to my function, but I decided it would be easier (and more efficient) to just read the file from the .NET code instead.

  • In the .NET code I construct a response dictionary of values that I want to pass back to Node (was the compilation successful? Were any warnings or errors generated? Where was the epub file written to? etc.) In node that object is marshalled back into a normal JSON object.

Pretty effing sweet.


The final site is a little more complex than what I’ve shared. I still had to add handlers in the Node app for downloading and removing the compiled epub files, and I added in some rate-limiting middleware to prevent abuse of the compile form, but I shared all of the Edge-related stuff here.

Easily spinning up Node-based server apps that can natively hook into powerful compiled .NET libraries is essentially the best thing that has ever happened anywhere ever. I had to pinch myself several times while setting this stuff up to make sure I wasn’t dreaming. I love writing my hard-core libraries in C#, but I hate writing websites in C# (and hate having to host websites on Windows servers). Problems solved.

It’s projects like Edge.js that restore my faith in humanity.

Short Permalink for Attribution: rdsm.ca/g20fg