When people share content on Facebook, they expect that the content will show up a certain way in the news feed: with their own title, description, and most importantly a relevant picture. Of course, RunKit is largely about text, whether in the form of source code or the properties in an object viewer, so we had to find a way to generate pictures that made sense. Building a prototype to do that with RunKit turned out to be so simple that we wanted to share the process.
What are we doing?
The first thing to figure out was what should be in the picture? We could generate an image that
contained the source code easily enough, but in many instances the results are more interesting
than the code itself. And sometimes the opposite is true: an interesting chunk of code might just
output something true, which is hardly worth turning into a picture.
We settled pretty quickly on the idea of turning the entire document into an image, and then taking it from there. There are a number of tools out there for turning HTML into an image, and after checking out a few, we decided to try moving forward with wkthmltoimage. It has the nice feature of being a self-contained binary, making it simple to run and deploy. And it can take HTML as a string, or a URL to a webpage that it will render directly.
Getting the image
wkhtmltoimage is cross platform, making it pretty easy to download to an OS X machine and see what an image of one of our notebooks would look like. Things worked ok out of the gate, but there was room for improvement. In order to actually make improvements, though, and to eventually deploy this as a production tool, we needed a way to generate these images programatically. We needed an API. What a great excuse to play around with Endpoint!
Creating an API with Endpoint is incredibly simple: implement one function in your notebook. The only thing our API had to do was take in a URL and then return an image of that URL. Part one was a straightforward "hello world" type Node.js problem, but part two required creating an interface to the tool running in another process. Fortunately, someone has already done that work on npm. Unfortunately, the library expected wkhtmltoimage to already be installed on the machine.
Endpoint requests, and RunKit notebooks generally, are running in generic linux sandboxes somewhere
"in the cloud," so it isn't super simple to just install a new binary. You can always script the
install process as a series of exec calls, but that's slow and cumbersome. But if you recall
that wkhtmltoimage is a self contained binary, you might reach a much easier conclusion: ship
a version of the library that vendors in the binary directly.
So that's what we did.
Forking the library on Github, downloading the dependency, and publishing a new package to npm took a few minutes. And less than a minute after that, the new package was available for use in RunKit. After that, writing the code to respond to incoming requests with an image was just a few lines.
const toImage = require("wkhtmltoimage-linux");
const { parse } = require("url");
exports.endpoint = function(request, response)
{
const query = parse(request.url, true).query;
response.writeHead(200, { "Content-Type": "image/png" });
toImage.generate(query.url, { format: "png" }).pipe(response);
}
And since we specified a custom example for our package, you can play around with your own copy of this server by just visiting the package page.
"Interestingness"
Facebook wants a specific aspect ratio for the images it displays, but notebooks will be many different sizes, so the next task was to figure out how to choose the right section of the page to show in a preview. Again we ended up turning to an existing solution, with an existing package already up on npm: [SmartCrop.js].
Just as before, we wanted our image to be generated with an API, and so again we created a new Endpoint. With the same basic skeleton, this API was only a bit more complicated. First, we actually get the screenshot as an image from our existing endpoint. Then we load it into [node-canvas], and pass the image to SmartCrop. Finally, we stream the resulting image to the response.
const Canvas = require("canvas");
const SmartCrop = require("smartcrop");
const { parse } = require("url");
const got = require("got");
const BaseURL = "https://runkit.io/boucher/screenshot-page/1.0.0"
const DefaultWidth = 640;
const DefaultHeight = 480;
exports.endpoint = async function(request, response)
{
const { query } = parse(request.url, true);
const width = parseInt(query.w, 10) || DefaultWidth;
const height = parseInt(query.h, 10) || DefaultHeight;
const options = { width, height, canvasFactory: (w, h) => new Canvas(w, h) };
const imageURL = `${BaseURL}?url=${encodeURIComponent(query.url)}`
const imageData = await got(imageURL, { encoding: null });
const image = new Canvas.Image();
image.src = imageData.body
SmartCrop.crop(image, options, function(result)
{
const canvas = new Canvas(width, height);
const context2D = canvas.getContext("2d");
const crop = result.topCrop;
context2D.patternQuality = 'best';
context2D.filter = 'best';
context2D.drawImage(image,
crop.x, crop.y, crop.width, crop.height,
0, 0, canvas.width, canvas.height);
response.writeHead(200, { "Content-Type": "image/png" });
canvas.pngStream().pipe(response);
});
}
There are a lot of parameters you can tune for your own use (adding those to your API is left as an exercise for the reader).
Deploying to production
In just a few hours, we had a working API for generating "interesting" previews for our Notebooks. There were a few more steps though. First, we spent some time optimizing the visual display of our notebook previews. For example, there was no need to show navigational elements. We created a simpler, cleaner page, and eliminated the need to load any javascript at all to make things faster.
Next was generating open graph meta tags in order to have Facebook see our previews. There's plenty of documentation on how to generate these in Facebook's developer center. One thing you may want to pay attention to if you're doing this yourself is that Facebook won't refetch an image it has already fetched, so if your image may change over time (as ours does), you'll need some kind of query paramter to force the image to update.
Facebook also expects images to load in just a couple of seconds. Unfortunately, this doesn't always work for us when generating a preview of a larger notebook. We ended up building a caching layer for our previews, generating them upfront and storing them for a period of time in s3.
The last step we did was move our code off Endpoint and into our own deployment environment. There are a few reasons this made sense for us, and probably will for you too if you're moving beyond a prototype. The most obvious is that Endpoints have limits. They only allow a fixed number of requests per day, and have a memory use cap as well. Deploying to our own infrastructure lets us put our own restrictions in place, like keeping our traffic with our own private network. And making the transition was as simple as downloading the code from our Notebook and adding it to our git repo (with a few modifications).
Wrapping Up
This is just a small sampling of the ways we're using RunKit ourselves to continue building RunKit. It's become an invaluable tool for us when experimenting with new features and ideas, or when just trying to simplify a problem down to its smallest reproducible example.
As always, we love to hear what our users are doing with RunKit, and what they'd like to see us work on next, so please get in touch!
What is RunKit?
- RunKit is an interactive playground for running Node.js in the cloud.
- RunKit offers serverless functions with zero deploy time — prototype code changes in real time!
- RunKit can be embedded in your tutorials and docs to make them interactive as seen on lodash.com, expressjs.com, and stripe.com.
Follow @runkitdev on Twitter for the latest updates from RunKit
© 2015–2018 Playground Theory, Inc.