Astromech - The Road to Beta2

Going quiet means I've been getting things done on Astromech. I have a set of specific features I want in the next beta, and in the last week nearly all of them have reached a semi-stable point.

Probably the best single demo of this is my little homage to the Caffeine molecule:

But first, the setbacks. The big one is; I've had to seriously question my use of Google Drive.

Anyone who saw the original video noticed how heavily I relied on Google Drive for the task of storing the bulk assets in each 'level' as well as building a collaborative editor (using the Docs realtime API) to set up the scene / load script.

Here's the stage I got to with that, before some rather bad news broke:

A mobile-friendly way to edit assets and  DSP 'circuits', backed by the Google real-time collaborative API.
Shame it will probably never see the light of day.

And that's just part of it. There's also the Panel designer - a hierarchical 'box layout' editor for all those cool LCARS-like consoles that litter the Astromech levels.

But unfortunately, Google has announced that they will disable file hosting from Google Drive shortly. I ranted a little about that in my previous post.

That has two very specific impacts on the Astromech 'GUI' editor. It means the files that it creates can't be read anonymously anymore. So. any Astromech levels based on a script stored in Google Docs will not be accessible to everyone. That's bad enough on it's own to kill that part of the project dead.

What's the point of collaboratively editing a "public world" file if the world's public can't read it?

And where do you put all the resources files it references? On another service? Then what's the point of using all the Google Docs API's if the 'real' data is elsewhere?

Shoving everything into one directory made things nice and manageable, using relative links. Once you've got your resources scattered across half the internet using absolute URI's, it makes things so much harder.

It's not just that I'd have to add a "Save As..." button to the Google Drive app, I have to re-think the entire premise of how users collaboratively store and work with terabytes of data. Instead of a central dumb (but reliable) fileserver and peer-to-peer clients, I'll probably need a peer-to-peer _server_ layer as well. ie: I need to replace Google by Christmas.

The old levels still work for now, but the access they depend on is deprecated and goes away next year. But it's the wasted effort in that direction that really hurts. Hopefully I can salvage most of the UI and editors, while backing them with a different datastore.

Then, there was the whole getUserMedia http:// deprecation thing I had to deal with. Within months, "powerful browser features" (which is basically everything I use.) will not work from http:// servers. Only https://.

This really broke me for a couple of days (I even got into a discussion with the security@chromium.org list) because it implied that running Astromech would only be possible from the Internet by paying money to Versign-derivatives. Not, for example, on your own goddamn own computer.

I settled down a bit when it was pointed out that localhost: is supposed to be considered "secure", (even if it only uses http:// without the SSL) So you will still be able to download and run Astromech on your own machine and use all the features. You can imagine how the alternative would have been maddening.

However, this still leaves the localnet in limbo. It's no longer clear how you'd run the software on your own desktop and access it, for example, from your own iPad over your own WiFi network. (Why, it's just as simple as creating your own SSL certificate authority for signing local machines, and then installing the certificates on each device, of course... why are you complaining?)

It was always in the back of my mind that a piece of Astronomy software that only really worked indoors while connected to high-speed internet might not be as useful as it could be. (instead of, say, in a dark paddock filled with amateur astronomers bristling with advanced imaging equipment and local bandwidth but poor global internet connectivity.)

I really don't want Astromech to be a "local webserver" install, for every individual user/machine. It should be more like running a minecraft server. If you need to install local servers everywhere to get a browser app to work, then what's the point of doing it in a browser? Why not just write a full application?

And besides, it seems really counter-intuitive that the only way to work with/around the 'increased browser security' is to start installing local code (eg, a node.js micro-server) with full binary access to the machine. That's just security whack-a-mole. If the machine gets boned through an exploit in my code, then it's not their fault for leading me down that path, obviously.

But the browser makers are determined to deprecate http:// and that's that. It doesn't matter that https:// is flawed, costly, inefficient, and creates barriers to entry.

OK, so now the good stuff that's made it into Astromech in the last few weeks:


iOS + Edge support
Astromech now 'works' on iOS9, to the extent it will load and render the scene using WebGL. What it doesn't do very well (or at all) is replace the keyboard/mouse control scheme with something that functions equivalently using touch.  I'm probably going the little "thumbsticks in the screen corners" route there, as soon as I get the time.

Microsoft Edge is running Astromech fairly well, better than IE did, but it also has some feature gaps (like getUserMedia / WebRTC) that effectively disable some of the more advanced features.

Chrome and Mozilla are still the 'preferred' browser, but all roads are slowly leading to HTML5 compatibility across all devices.


Improved Blender/WebGL shaders
The first-gen model loader did well with geometry, but badly with surfaces. For a start, only the first texture worked, and there was no real lighting model. So, scenes looked very different in Astromech from how they originally looked in Blender, even if the geometry was correct.

The 'shader compiler' I wrote has been extended with a full multi-source specular lighting model, with 'sun' and 'point' lights. Technically it does a Lambert/Blinn-Phong pass with fixed lights.

So, now you can export a fairly generic existing Blender model (instead of carefully building one specifically for Astromech) and it will mostly work as you expected. Common surface materials work. Multiple scene textures work. Bump maps sort-of (they have the common view-independence problem because of the lack of tangent vectors in the collada export, so the bumps always point 'up' instead of 'out', though there's probably a way to solve that. Good for floors though.)

I still don't have a great solution for transparent surfaces, but then, neither does anyone else.


Multiple Scene Models
Version 1 only properly loaded a single 3D model as the 'primary scene'. That's been fixed, so you can load an arbitrary number of collada files, and instance them multiple times within the scene at multiple locations.

eg: In the "Atomic Caffeine" demo, each of the four atoms was modelled/coloured in Blender, and then instanced into the scene as many times as the entire molecule needed.

New features sometimes magnify minor old problems; in this case the lack of a global lighting model. Since each 'scene' model carries its own lights in its own reference frame, obvious visual inconsistencies occur when you put several models together and rotate some of them. (Although, less jarring than I'd have thought.)

Fully dynamic lighting is a major overhead with diminishing returns. So, I'll probably go for a compromise, with only a few global dynamic lights.


Cannon.js Physics Engine
The other side of the multiple-model system is the ability to define a 'physics proxy' (usually a box or sphere with properties of mass and friction) to which the position of the 3D models is attached.

I've chosen the cannon.js physics system to do the heavy lifting. It can connect the proxy objects together with 'hinges', 'springs', and other physical constraints like gravity, and then model the physics over time and update the objects.

It's extremely efficient (the solver it uses is very advanced) although there are severe practical limits to just how much you can do in real-time. But a little physics is a great way to add some life to an otherwise static scene, and give the user the sense that they're there, and bumping into things.


Scriptable UI
I've just about finished exposing all the things that Astromech can do as scriptable elements - as opposed to my early examples that used lots of  hard-coded javascipt.

It's slightly less flexible than the raw javascript - at least until I create a 'module' system capable of safely loading arbitrary code. It's still just a set of pre-approved LEGO blocks you can arrange in various ways, but at least the set of blocks is getting bigger.

Scripts don't all have to run on load. The script can define UI "command buttons" which run parts of the script later... which might load more resources and create new buttons. A common use of command buttons is to provide "teleport" options which can jump you around the map.

In practice, you can already build a 'conversation tree' system which offers choices dependent on previous choices. (All the buttons would be pre-defined, just shown and hidden using commands activated by other buttons.)


Social 'presence' and messaging.
The chat system has been functioning for a couple of versions now, based on a websocket 'pub-sub' server that I'm running on an OpenShift cartridge. (Thanks, RedHat!) I've gone through a couple of revisions of this system, and it's been stable and reliable for months.

Previously, you'd get a 'chat message' when someone connected to the channel ("hailing frequencies open") but in the background the networking code always had the full list of the other participants this entire time, you just couldn't see them. Now, the right-hand of the screen is one long column of everyone else in the level with you.

This makes everything feel a lot more MMORPG, and future extensions will be things like "friends lists" and private instances that build on this social side, since there's going to be obvious problems if a 'level' gets too popular.


File Transfer & Videoconferencing
The first features the social list made possible was inter-user private chat (easy) followed by file transfer (not so easy, but mostly working) and video conferencing. (just got the prototype working)

It's a core idea of astromech that you should be able to exchange data with other people. This is an essential part of that plan.

The file transfer I'm particularly proud of. To 'transmit', the sender just drags a file out of the File Manager and drops it on the button for the intended recipient.  An 'offer' is then sent to the receiver, and turns up in their corresponding list of options for the sender.

If the receiver clicks on this offer button, the file is downloaded to the browser's "Temporary FileSystem" (you get a little progress message while the transfer is in progress) and then the recipient can either click on the button a second time which open the (now local) file in the browser, or they can drag the file icon back out of the browser to the filesystem again.

In summary, one user drags the file into their browser. The other user accepts the offer, and then can drag the file back out of their browser. (Well, Chrome) I don't think I can make it simpler.

Remember, this is peer-to-peer. Although right now all the comms goes through the chat relay, (as private messages) but I have the RTC channels working, so I intend to make that the preferred transport to make it truly peer-to-peer, and reserve the relay server as the 'fallback'.


Voice Recognition
This was nearly a 'freebie', in that I went - in one morning - from not knowing that browsers offer a full voice-recognition engine javascript API, to having it working by lunchtime.

Any 'command button' can be given an array of "speech" strings that, if heard by the engine, activates that command button. It's that easy.

It's good to have an optional prefix word that wakes up the engine but can be missed, because that happens a lot. Originally I used "computer" (duh) but soon changed it to "Scottie!" after shouting at my machine for a little while to transport me to new locations and switch on parts of the engine. Feels much more natural, somehow.



I could go on for pages about all the things I want to do next, the improvement and changes, but I think we'll just stick with what I've actually done so far.

The major features are now mostly in. There's a ton of clean-up work and major bugfixes that need doing before release, but no more super-bleeding-edge experimental features. I did the hard stuff first.

A 'Beta2' release isn't far off now. I'm trying to be quick, before the ground shifts under my feet again. It's not easy doing all this single-handed, but I'll get there.