Designers are blind without solid usability research, and solid usability research for 10-foot experiences is difficult to conduct due to the use of a remote. While there are some prototyping tools that do support sending commands from one prototype to another, they are not free and come with their own steep learning curve. So I set out to build a solution for our team that was free, didn't require any code, and could imitate our entire mobile app TV remote.
Sketch
Origami Studio
JavaScript ES6
React.js
CSS 3
HTML 5
Git
While working on the user experience for Smart Room and other mobile projects, I did extensive protoyping for native apps. I love prototyping because it's useful at nearly every stage of the design process, for everything from stakeholder feedback, to just getting a sense of a design direction, all the way to detailed usability testing, C-Suite demos, and building excitement around a product.
By prototyping at every fidelity, I'd learned many tools, but none of them are a silver bullet. Principle is great for rapid, and more realistic protoypes, but it falls flat when complexity grows due to the screen-to-screen transition workflow, and creating multiple changes in response to one event. Sketch or XD is great for lower-fidelity rapid prototypes for demoing a flow, or IA testing. But Origami Studio is one of the most powerful ones that does not require any programming knowledge.
Origami Studio is amazing because it uses a module-based approach to enable to designer to impart many changes to various parts of the prototype from a single event. In order to create an app remote for Origami Studio TV prototypes, I knew I'd need some way of sending and recieving data over the network. Luckily Origami has a module called Network Request. It simply calls a URL and recieves the response (as JSON, in my case).
Ok, realistically I didn’t hack anything, I just applied a little creativity to how the built-in networking module could be used. Once I realized that Origami could interact with the network, it opened up the possibility to spin up a local Node.js server (one that simply runs right from the computer running origami studio, and on the same network as the computer). From there, both Origami Studio and my phone could both talk to this Node server, and exchange some messages about what buttons were pressed. Below is a high-level overview of the communication logic.
The only caveat to this approach was that Origami Studio would need to ‘poll’ the node server with multiple requests per second to achieve the illusion that the remote is sending the command directly to Origami. The reality is that Origami Studio is actually GET-ting the key press from the server, and the mobile web-app is POST-ing.
Luckily, Origami Studio has another module called Repeating Pulse which could be connected to Network Request to make it send many requests per second. Since the server is running directly on the computer that Origami Studio is also running on, there is virtually no latency with this polling scheme.
Once the basic logic of how these three applications would interact was decided, I jumped into sketch and simply exported the UI comp of our mobile app TV remote as an SVG file. If this were a ‘real’ project, it would have benefited from coding that interface from scratch, however, for protoyping purposes, it was much faster to simply port over the svg file into a react component, and then hook event listeners for taps directly into various areas of the svg code.
Once I ran the raw svg file through SVGR.com to generate equivalent react svg code, I added a stylesheet and an HTML file which would bundle up the react code and css. I assigned classes to each button in the remote, and toggled css classes on them when pressed to add animations and on/off states to the remote prototype.
After the remote ‘client’ was done, I started working on the server itself. I wanted to make the utility as simple as possible to run so I decided to create an actual Mac app bundle that would spin up the Node.js server, and provide instructions on how to open the app remote.
Just like you typed in a URL to get to this page, the user would need to type in the URL of the server on the local network (usually 192.168.###.###/remote.html). This would require the user to get the IP of the computer they’re running the server on, and then type it in the address bar on their iphone. Ew. How can I make this less painful to start up?
I dug around a bit online, and it turns out, someone wrote a Node package to send iMessages from a node process. So instead of the user having to type in the IP of the server (which is just the IP of the computer), I could just prompt the user to put in their imessage email and the server would send its own IP in the URL of the app remote as an imessage. It could then simply be opened on the phone, and the remote web app would magically appear. Sweet!
At its most basic, a mac app is really just a folder structure with a few key files that describe the app, its capabilities, the entry-points that get run, and so on. So it was pretty simple to replicate the necessary folder structure, specify the node.js server script as the entry point, slap an icon on it and call it done.
With the hard technical bits done, I jumped back over to Origami Studio to create a reusable component that allowed me to listen to remote events from anywhere in the prototype. All the component does is take the “next button state” as an input, and then sends the corresponding buttons’ pulse as an output.
With all the machinery in place, I took a stab at creating a TV prototype of a ‘Home’ screen, and bundled this demo prototype with the Origami Server Git repository. Being able to put an app remote in a stakeholder’s hand and make stuff move on the TV made it much easier to collaborate with product on various aspects of the 10ft experiences we were developing. Check out the (rough) demo of what this tool is capable of!