tag:blogger.com,1999:blog-72713153675583005692024-03-13T05:27:33.931-07:00The Unorthodox EngineersA one-man development teams' cries of warning from the cutting edge.Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.comBlogger100125tag:blogger.com,1999:blog-7271315367558300569.post-4362936161359142522016-11-13T23:05:00.001-08:002016-11-14T10:21:16.856-08:00Frank, Breathe!<span style="font-family: "verdana" , sans-serif; font-size: x-small;">// breathing LED, for @frankippolito</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">// simplest possible sketch, using only loops and IO, no PWM hardware or libraries.</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">// copy and paste this into a new Arduino sketch, modify the indicated parameters and upload.</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">// (</span><span style="font-family: "verdana" , sans-serif; font-size: x-small;">v2 </span><span style="font-family: "verdana" , sans-serif; font-size: x-small;">some corrections and comment mods)</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"><br /></span>
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">#define LED_PIN A0 // which pin is the LED on</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">// CHANGE THIS ^^ FOR DIFFERENT PIN</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">#define LED_GAMMA 4.4 // gamma correct the brightness curve</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">// CHANGE THIS ^^^ FOR DIFFERENT BREATH CURVE</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">// larger values make it "pop" more at the end</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"><br /></span>
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">#define LED_OFF_STATE LOW // in theory the arduino can sink more current than it can source, </span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">#define LED_ON_STATE HIGH // so swap these if you're wiring the other side of the LED to VCC rather than GND</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"><br /></span>
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">#define PWM_MIN 0 // LED brightness minimum (0..255)</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">#define PWM_MAX 255 // LED brightness maximum (0..255)</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">// there is no need to use a series resistor for the LED as long as you manage the maximum on time. PWM is </span><span style="font-family: "verdana" , sans-serif; font-size: x-small;">More efficient</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"><br /></span>
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">#define DELTA_RATE 0.001 // LED brightness change per delta step (rate of change)</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">// CHANGE THIS ^^^ TO TUNE RATE</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">#define DELTA_LOOPS 2 // how long to spend in the PWM loop per delta step (major divider)</span><br />
<br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">// the rate of the "breathing" is going to be a slightly complication multiplication of</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">// the processor clock speed, some overheads, and the </span><span style="font-family: "verdana" , sans-serif; font-size: x-small;">DELTA_RATE</span><span style="font-family: "verdana" , sans-serif; font-size: x-small;"> divided by </span><span style="font-family: "verdana" , sans-serif; font-size: x-small;">DELTA_LOOPS</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">//</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">// Code is written so the MIN and MAX values can be tweaked later without altering timings.</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">// There is also potential for "gamma correction" if the log brightness curve isn't sophisticated </span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">// enough</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">// </span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">// update: Changing the DELTA_RATE is now the best </span><span style="font-family: "verdana" , sans-serif; font-size: x-small;">way to fine-tune the breathing rate. </span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">// It scales better and with finer control than the LOOPS, since it's a floating point number.</span><br />
<br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">void setup() {</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> Serial.begin(57600); // needed on some boards</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> pinMode(LED_PIN, OUTPUT);</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> digitalWrite(LED_PIN, LED_OFF_STATE);</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">}</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"><br /></span>
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">int direction_up = true;</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">float linear = 0.0;</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"><br /></span>
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">void loop() {</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> // update our linear value</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> if(direction_up) {</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> // going up</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> if( (linear += DELTA_RATE) >= 1.0 ) {</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> // we hit max</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> linear = 1.0; direction_up = false;</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> }</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> } else {</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> // going down</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> if( (linear -= DELTA_RATE) <= 0.0 ) {</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> // we hit max</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> linear = 0.0; direction_up = true;</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> }</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> }</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> // turn the linear value into an absolute PWM brightness via the gamma curve</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> float gamma = pow(linear, LED_GAMMA);</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> gamma = gamma * (float)(PWM_MAX - PWM_MIN) + (float)PWM_MIN;</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> int pwm = gamma;</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> //Serial.println(pwm);</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> // if we have PWM hardware available, this next bit would be just a call to set it's value and a wait(),</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> // but we're not going to assume any is available on the pin you want, so we just use raw loops to do </span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> // rough-and-ready PWM to the LED for a few turns.</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> for(int j = DELTA_LOOPS; j>0; j--) {</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> int i;</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> // spend some time on</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> for(i = pwm; i>0; i--) {</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> digitalWrite(LED_PIN, LED_ON_STATE);</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> }</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> // spend some time off</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> for(i = (256-pwm); i>0; i--) {</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> digitalWrite(LED_PIN, LED_OFF_STATE);</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> }</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"> }</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;">}</span><br />
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i9.ytimg.com/vi/cw-Q_zHUohM/default.jpg?sqp=CJzEpcEF&rs=AOn4CLB26j8rjg9TP21vsgz4FWx8uzfNVA" frameborder="0" height="266" src="https://www.youtube.com/embed/cw-Q_zHUohM?feature=player_embedded" width="320"></iframe></div>
<span style="font-family: "verdana" , sans-serif; font-size: x-small;"><br /></span>Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com0tag:blogger.com,1999:blog-7271315367558300569.post-58383865140528661982016-06-25T11:07:00.000-07:002016-06-25T11:07:24.539-07:00Pi Manipulator Build - Part 1Here's the latest thing I've built: (screen not entirely tucked in yet...)<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-szIOwiZX3Kw/V26WeXBLKEI/AAAAAAAACkg/mEhEma1FfSk9CAo7u_uNzgIU-Pnqnf9GQCK4B/s1600/IMG_0769.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://1.bp.blogspot.com/-szIOwiZX3Kw/V26WeXBLKEI/AAAAAAAACkg/mEhEma1FfSk9CAo7u_uNzgIU-Pnqnf9GQCK4B/s320/IMG_0769.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">I'm into wearable technology, now.</td></tr>
</tbody></table>
<br />
Base specs for the machine are:<br />
<br />
<ul>
<li>Raspberry Pi Zero - 1Ghz ARM processor, 512Mb of RAM</li>
<li>8Gb SD card with Raspbian Jessie</li>
<li>WiFi 802.11n adapter</li>
<li>MPU6040 accellerometer/gryo inertial sensor</li>
<li>1/5" PAL/NTSC LCD television, 240x180 pixels</li>
<li>Turnigy Nanotech 950mAh 1-cell LiPo</li>
<li>Polulu 5v step-up voltage converter/regulator.</li>
<li>2 old mouse buttons</li>
</ul>
<div>
The MPU6040 sensor is used by my 'gyromouse' driver, which means the manipulator has a fully functional X11 GUI interface. I can run most linux apps, (that don't require large screens or shift-click combos) and even use the 'Florence' on-screen keyboard if I need to tap out a few characters. (yes, it's painfully slow.)</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-LRioXcjNZqE/V27B7bDsA7I/AAAAAAAACoU/9foK2DsRSz8QL6s-qt4BG5wnRRhGUbLyQCK4B/s1600/IMG_0770.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://2.bp.blogspot.com/-LRioXcjNZqE/V27B7bDsA7I/AAAAAAAACoU/9foK2DsRSz8QL6s-qt4BG5wnRRhGUbLyQCK4B/s320/IMG_0770.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The screen is just resting on top. <br />Once I push it in, I'm not sure it's coming back out.</td></tr>
</tbody></table>
<div>
<br /></div>
<br />
What makes the device actually useful is that it's a full Linux machine, with a WiFi connection. That makes any nearby PC that can run SSH or VNC into a handy keyboard, mouse and 'big' screen. Or even a remote PC - once the manipulator is on the network, it doesn't matter where it is, or you are. It acts very much like a 'cloud server' which just happens to be located on your wrist.<br />
<br />
What I _can_ easily do is pre-create the equivalent of desktop shortcuts that run shell scripts or any arbitrary action you can do on a linux machine. The ultimate programmable remote control - that can also show you progress and logfiles.<br />
<br />
Why do I need such a thing? Well, personally, I've found that doing digital astronomy is a pain in the ass if you have to keep running between the telescope and PC. Especially if one is inside the house. Or you're in the field, literally.<br />
<br />
The main person who will get to use it is a unix security expert, for whom having a wifi-scanning wrist strap (running Kismet) is a useful work tool.<br />
<br />
Neither application requires a lot of user interaction - just enough to navigate a few menus and click a 'start' button or two - but they're apps which have a lot of status displays it's handy to keep an eye on.<br />
<br />
The build proceeded in very careful steps. I'd actually developed the software in advance on my Pi A+, so when the Zero and other hardware pieces arrived, I was basically ready to go. It only took a couple of hours to hook up the primary components and give everything a whirl.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-X8g-zSnAfFQ/V26gbRZ0LvI/AAAAAAAACk4/IoS_X4fOPZM_qwLj3ANb1HoEvCwD_Jv-gCK4B/s1600/IMG_0558.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://2.bp.blogspot.com/-X8g-zSnAfFQ/V26gbRZ0LvI/AAAAAAAACk4/IoS_X4fOPZM_qwLj3ANb1HoEvCwD_Jv-gCK4B/s320/IMG_0558.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The parts on arrival.</td></tr>
</tbody></table>
<br />
It's always good to check your parts individually if you can, so I test-fired up the Pi with my prepared OS on a standard monitor/keyboard setup, despite the ridiculous relative size of the connectors and cables. All fine.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-qJAoxf4_fgA/V26iUTrp3gI/AAAAAAAAClM/Lg-HNBKn2UAYs0k9NxeSiCm7gX1D9K_OQCK4B/s1600/IMG_0596.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://3.bp.blogspot.com/-qJAoxf4_fgA/V26iUTrp3gI/AAAAAAAAClM/Lg-HNBKn2UAYs0k9NxeSiCm7gX1D9K_OQCK4B/s320/IMG_0596.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">HDMI cable with an end-cap made of computer</td></tr>
</tbody></table>
<br />
After that, I spent a good hour or two just staring at all the pieces, physically and mentally stacking them in various ways, and trying to plan how they were all going to fit together.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-g4liVa11ZkI/V26qcN_BB_I/AAAAAAAACmo/KbfmOoSpTfE7MfVsyDGimkxesS1hjdq2wCK4B/s1600/IMG_0572.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://2.bp.blogspot.com/-g4liVa11ZkI/V26qcN_BB_I/AAAAAAAACmo/KbfmOoSpTfE7MfVsyDGimkxesS1hjdq2wCK4B/s320/IMG_0572.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Jenga!</td></tr>
</tbody></table>
<br />
The RCA connector for the LCD module's video input was far too big, and even the 4-pin "low profile" socket is comparatively large.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-BcOa4rV3M4I/V26g1qt1snI/AAAAAAAAClA/of7BD76kPJE8T1KI_IbRcEtZkgze0iewQCK4B/s1600/IMG_0580.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://4.bp.blogspot.com/-BcOa4rV3M4I/V26g1qt1snI/AAAAAAAAClA/of7BD76kPJE8T1KI_IbRcEtZkgze0iewQCK4B/s320/IMG_0580.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">80-year old socket technology.</td></tr>
</tbody></table>
<br />
So the first thing was to pop off that connector and direct-solder to the Pi's video output. I also de-cased the WiFi adaptor and direct soldered that to the Zero as well.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-MSrmsn2G4IM/V26kmVCWiiI/AAAAAAAACl0/tp4nqSzxPGoD0ULPkD-Z_6Z4rWDPXYI7QCK4B/s1600/IMG_0606.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://4.bp.blogspot.com/-MSrmsn2G4IM/V26kmVCWiiI/AAAAAAAACl0/tp4nqSzxPGoD0ULPkD-Z_6Z4rWDPXYI7QCK4B/s320/IMG_0606.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Naked WiFi dongle.</td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-T2NXKWNHrns/V26kVTUPSxI/AAAAAAAACls/4016O-5L8Mg3SNDaQOjhHyFYLRk8Q18SgCK4B/s1600/IMG_0605.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://1.bp.blogspot.com/-T2NXKWNHrns/V26kVTUPSxI/AAAAAAAACls/4016O-5L8Mg3SNDaQOjhHyFYLRk8Q18SgCK4B/s320/IMG_0605.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Almost like they expected us to do this kind of thing.</td></tr>
</tbody></table>
<br />
<br />
Then time for another test, to make sure the LCD module was working.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-ACiCMnYOmxw/V26jThK5cuI/AAAAAAAAClY/gk3Q9MLG650L_CCGPN9sJzSI0DBZX4zLACK4B/s1600/IMG_0610.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://4.bp.blogspot.com/-ACiCMnYOmxw/V26jThK5cuI/AAAAAAAAClY/gk3Q9MLG650L_CCGPN9sJzSI0DBZX4zLACK4B/s320/IMG_0610.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">First LCD test</td></tr>
</tbody></table>
<br />
Perfect. By this time it was getting late and so I wound down by wiring up the Polulu voltage converter to a handy LiPo and testing that out. I hadn't used a Polulu before, but I'm impressed. Quality little component. Get some.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-Jci4vlPYiek/V26mBU7l64I/AAAAAAAACmA/3TcLauZ74JEX6rZDZpj0HeGT6o0wk6KhACK4B/s1600/IMG_0615.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://3.bp.blogspot.com/-Jci4vlPYiek/V26mBU7l64I/AAAAAAAACmA/3TcLauZ74JEX6rZDZpj0HeGT6o0wk6KhACK4B/s320/IMG_0615.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Not actually sure of the capacity of this battery. About 600mAh?</td></tr>
</tbody></table>
<br />
I didn't want to plug an untested power converter it into my shiny new machine, so I tried it first on an old Arduino board that's seen it's share of action.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-eXpwRzSrhrY/V26mTuXpxNI/AAAAAAAACmM/wnvfnywb5P4WNNZqm8ENwdy4BizB30KdwCK4B/s1600/IMG_0616.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://4.bp.blogspot.com/-eXpwRzSrhrY/V26mTuXpxNI/AAAAAAAACmM/wnvfnywb5P4WNNZqm8ENwdy4BizB30KdwCK4B/s320/IMG_0616.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Your bravery is appreciated.</td></tr>
</tbody></table>
<br />
And to finish the night, the first un-tethered boot of the machine:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-hLm5yhU4b0I/V26oYHLJUxI/AAAAAAAACmc/vmjGlDtasQMVZpGnl_H8cmMF-u1WEqtrgCK4B/s1600/vlcsnap-2016-06-26-01h48m36s145.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="180" src="https://2.bp.blogspot.com/-hLm5yhU4b0I/V26oYHLJUxI/AAAAAAAACmc/vmjGlDtasQMVZpGnl_H8cmMF-u1WEqtrgCK4B/s320/vlcsnap-2016-06-26-01h48m36s145.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">First boot from battery. All systems go. Power stable for 63 minutes.</td></tr>
</tbody></table>
A good first day. I slept well.<br />
<br />
The next day, I added the MPU6040 inertial sensor and a single membrane button I pulled from an old remote control. I put some heatshrink tubing around various easy-to-short components like the WiFi adapter, and then very carefully folded up all the boards into a neat little package.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-wrhhZjT2c1U/V26r7b4DfRI/AAAAAAAACm0/kL1Gv6Uz2WYDl4HDsO9QcPQN4e7RiqDSQCK4B/s1600/IMG_0630.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://1.bp.blogspot.com/-wrhhZjT2c1U/V26r7b4DfRI/AAAAAAAACm0/kL1Gv6Uz2WYDl4HDsO9QcPQN4e7RiqDSQCK4B/s320/IMG_0630.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Now it's looking more like a... um... thing.</td></tr>
</tbody></table>
<br />
Another boot to make sure it's all still working, and if gyromouse has started up properly. Which it had. It took longer to find an on-screen keyboard that worked properly. "matchbox-keyboard" is the one you'll see mentioned, but it was chewing up my CPU because of a known bug, and I switched to "Florence" instead. Which has different issues, but nevermind.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td><a href="http://1.bp.blogspot.com/-YHYkwLNxnkU/V26aOTY7Q8I/AAAAAAAACks/IZYBqSMMFJkZeDQzl9vPYiYzTbzGovpjACK4B/s1600/IMG_0665.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://1.bp.blogspot.com/-YHYkwLNxnkU/V26aOTY7Q8I/AAAAAAAACks/IZYBqSMMFJkZeDQzl9vPYiYzTbzGovpjACK4B/s320/IMG_0665.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="font-size: 12.8px;">Taking "hunt and peck" to a new level, but the GUI is useable.<br /></td></tr>
</tbody></table>
Now that I had a mouse-equivalent and an on-screen keyboard, I could finally run a few apps. Like the web browser.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-lpmUxz4-t_w/V26tATnRSVI/AAAAAAAACnA/rd5msjiTRi0n-Kb2gnyInMul-evgSr4yACK4B/s1600/IMG_0668.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://3.bp.blogspot.com/-lpmUxz4-t_w/V26tATnRSVI/AAAAAAAACnA/rd5msjiTRi0n-Kb2gnyInMul-evgSr4yACK4B/s320/IMG_0668.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Doing a google search</td></tr>
</tbody></table>
<br />
Or kismet:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-lfqzkm1e0WU/V26tUXWUgzI/AAAAAAAACnI/V3qIDjldG9kgz6aF-m9CoYuik5KWI_wgACK4B/s1600/IMG_0675.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://2.bp.blogspot.com/-lfqzkm1e0WU/V26tUXWUgzI/AAAAAAAACnI/V3qIDjldG9kgz6aF-m9CoYuik5KWI_wgACK4B/s320/IMG_0675.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Detecting nearby Wifi devices...</td></tr>
</tbody></table>
<br />
Or even Minecraft.<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/fJKJoDHDHiA/0.jpg" src="https://www.youtube.com/embed/fJKJoDHDHiA?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<br />
<br />
At this point I was feeling pretty satisfied with myself. I spent the next day or two tuning gyromouse, setting up the desktop and generally getting everything nice.<br />
<br />
It's this point where I started to realize just how much power the LCD module was consuming. Not actually the screen, so much as the video decoder chip that drives it. The module specs say "120mA at 5V" and really means it. I suspect most of that goes through the decoder, which in normal operation gets almost too hot to touch.<br />
<br />
The LCD power consumption will be important later, because addressing it has become the one remaining issue in the build, though not for the reason you probably think.<br />
<br />
Apart from that, everything was going well. Really well. (too well?) All the computer hardware and software was working, and all I needed to do now was fit it into some kind of case. I had just the thing in mind, and was checking the mailbox every day in anticipation.<br />
<br />
Then it arrived! My Official Doctor Who Vortex Manipulator! Batteries Included! I've just got to scoop out the old insides, and put my hardware in its place.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-R4xNY8Wt9qc/V26xEtjz37I/AAAAAAAACnY/O-S79GyL63kmXM4eYQPLRPbOZhsv0ErQwCK4B/s1600/IMG_0684.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://2.bp.blogspot.com/-R4xNY8Wt9qc/V26xEtjz37I/AAAAAAAACnY/O-S79GyL63kmXM4eYQPLRPbOZhsv0ErQwCK4B/s320/IMG_0684.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">'cause if you're gonna build a computer into a wrist-strap, <br />why not do it with some style?</td></tr>
</tbody></table>
<br />
Yes, I'm a fan, but I also appreciate the thought that went into this particular prop: Capn' Jacks' personal device. Half wallet, phone, keys, and watch. So sensible an idea that no-one gives it a second thought. Well <i>of course</i> future advanced humans strap their mobiles to their wrists on cool beat-up leather bands. Why wouldn't you?<br />
<br />
Why don't we?<br />
<br />
But then it became obvious - it wasn't going to fit. At least, not the toy version. The real prop is about 50% bigger and probably would have worked nicely. But an accurate replica is also about twenty times more expensive, and I'd never have been able to bring myself to damage it.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-6Hsjrfqxd14/V26yErzbEdI/AAAAAAAACns/JYMvYp5tltsZBNIiD3v77dVGAdBG3X6zQCK4B/s1600/IMG_0695.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://2.bp.blogspot.com/-6Hsjrfqxd14/V26yErzbEdI/AAAAAAAACns/JYMvYp5tltsZBNIiD3v77dVGAdBG3X6zQCK4B/s320/IMG_0695.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The original toy also has one working button, <br />which activates a blue light! Wow!<br /></td></tr>
</tbody></table>
At this point, I had a long, long think. And a talk to my friends about leather-work. Basically, I had two choices: try to fit the manipulator into the toy, or get a new custom wrist-strap specially for the project.<br />
<br />
I chose both.<br />
<br />
Eventually I'll get a custom wrist-strap built - already made inquiries. I'll have to build a hard case to hold the components too. (Or use an Altoids gum tin, perhaps, since that seems the right size.) But in order to figure out the issues, I decided to go ahead and try to stuff everything into the toy anyway. I didn't have any real use for it otherwise, and I figured I'd learn a few things that would be useful in building the "real" manipulator.<br />
<br />
And I expected to fail. The Pi Zero is slightly bigger than the strap is wide. I didn't expect it to fit. It shouldn't have fit. But somehow it did.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-AzvtKf87KOs/V263GSdy81I/AAAAAAAACoI/i2LJsXz7NJYAU0iWmKF7ieGpKoA6fRoHwCK4B/s1600/IMG_0741.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://4.bp.blogspot.com/-AzvtKf87KOs/V263GSdy81I/AAAAAAAACoI/i2LJsXz7NJYAU0iWmKF7ieGpKoA6fRoHwCK4B/s320/IMG_0741.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Bigger on the inside?</td></tr>
</tbody></table>
<br />
Next time, I'll show you how I managed it. Also, how I snapped the SD card containing the entire OS in the process, I'll sum up where I've got to, and the few problems left, and what's next, It's a rollercoaster.<br />
<br />
<br />
<br />
<br />Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com13tag:blogger.com,1999:blog-7271315367558300569.post-28442850896985418942016-06-13T22:11:00.002-07:002016-06-13T22:11:51.014-07:00SnailMail - Batteries Not IncludedSo I had a weird call today from one of my electronics parts suppliers, and apparently they "can't ship LiPos anymore", so that part of my order has been refunded.<br />
<br />
Given this particular LiPo is about the size of a credit card and I've ordered enormous batteries from HobbyKing in the past - this sounded strange to me! So I looked into it and yes, the internet boards are newly filled with hate for Australia Post and IATA over the reclassification of LiPos as "class 9" dangerous goods.<br />
<br />
Or something. No-one's really sure, except AusPo won't take their batteries anymore.<br />
<br />
What I think that means is that a whole new documentation path needs to be applied for transporting LiPos via air, and most of Australian Posts' inter-city mail goes that way. And AusPo either doesn't have the ability, or doesn't want to, supply all the new paperwork.<br />
<br />
Sounds annoying, until you realize that it's not just big RC batteries, it's apparently anything with lithium. Installed or not. Coin cells included. Which means people are being told they can no longer send ordinary things like <i>watches </i>and <i>mobile phones</i> through the post that, perhaps they already sold to someone on eBay. It's not even just a matter of "you have to remove the battery before posting" and expecting the receiver to somehow find a matching replacement - look at a modern iPhone or iPad. Remove the battery - <i>how</i>?!?!<br />
<br />
Couriers are apparently still able to get LiPos through, at increased cost of course. But there's vast amounts of mail in international transit that might end up being returned because it doesn't have the required documentation. Maybe.<br />
<br />
So, the nations primary package carrier now no longer will carry the majority of the things that we use and sell to each other, due to an air transportation safety rule change. Apparently they hate electronic mail so much they're withholding our electrons.<br />
<br />
Makers are particularly hard hit, because we can't get parts, and we can't ship our creations. Well, not using our "national infrastructure" - in which case, what's it for?<br />
<br />
The rule change may even affect carry-on luggage for plane passengers. Or it might not. It might all just be temporary, while they sort out systems. Or it might be a federal crime to post an iPhone from now on. No-one is completely sure right now, because inconsistencies abound, straight answers are hard to come by, and official documents keep going up and down on the web like well-formatted yoyos..<br />
<br />
Hmm. "Well formatted yoyos." I think that's my new name for Australia Post.Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com1tag:blogger.com,1999:blog-7271315367558300569.post-81753864067199526402016-06-08T22:45:00.000-07:002016-06-08T22:45:08.907-07:00GyroMouse PreviewSo, I'm putting together another "Device". I make them occasionally. Some of them even work.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://4.bp.blogspot.com/-zHzCa2r4-d4/V1j5edZ1kuI/AAAAAAAACjE/Kq9scWZrvn48lxO9gsupzfjfLuAmDqn6QCLcB/s1600/IMG_0535.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://4.bp.blogspot.com/-zHzCa2r4-d4/V1j5edZ1kuI/AAAAAAAACjE/Kq9scWZrvn48lxO9gsupzfjfLuAmDqn6QCLcB/s320/IMG_0535.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">GyroMouse Prototype. <br />MPU6040 IMU Attached to Raspberry Pi Model A+</td></tr>
</tbody></table>
<br />
While I'm waiting for most of the specific hardware pieces to turn up from all over the internet, I've been slapping together prototype parts and getting the software at least vaguely right. I find that helps.<br />
<br />
One goal for this new device is to be compact, so I'm going to use a Raspberry Pi Zero (which I <i>believe </i>is in the post. They're a little hard to get.) So compact that the entire machine can be wrist-mounted, smart-watch style.<br />
<br />
And in fact, that's working out fairly well. The Pi Zero is only 70x30mm, 1.5" screens are about 30mm vertical. I have a very specific 80x40x20mm lozenge-shaped space that I need to fit, for reasons which will become obvious in later posts.<br />
<br />
You can fit a whole 512mb / 1Ghz processor with GPU hardware in that space, with enough battery to run for hours. WiFi dongles are mostly just connector now.<br />
<br />
What's hard to fit is any kind of human-friendly input device. The screen is so small that "touchscreen" tech would be ludicrous. It might be nice to turn the entire surfacebeyond the screen into a capacitive touchpad, but there's no ready sources for such a module, and I'm trying to use Adafruit-level commodity pieces.<br />
<br />
Assuming I've got a small linux machine with X11 on my wrist, what's the best way of using that device? Well, mouse is probably still it, in terns of compatibility with existing software. With a mouse you can run an on-screen-keyboard if you have to. It will do everything, if badly.<br />
<br />
So to fit something like a mouse in the space available, I've written a little program which listens to a MPU6040 Intertial Management Unit and interprets it's movements as mouse commands. Gyro rotations in X and Y move the mouse cursor, and "taps" on the device (detected by the accellerometer) are interpreted as mouse clicks. At least for today.<br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://3.bp.blogspot.com/-ruXHRInFEtU/V1j5fy8YLFI/AAAAAAAACjM/NgkDb73nSFco1PTGPa8K-aeEUHaGwpj2gCLcB/s1600/IMG_0537.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://3.bp.blogspot.com/-ruXHRInFEtU/V1j5fy8YLFI/AAAAAAAACjM/NgkDb73nSFco1PTGPa8K-aeEUHaGwpj2gCLcB/s320/IMG_0537.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">To control a tiny computer you need a tiny input device!</td></tr>
</tbody></table>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
As of an hour ago, it works. I had to go through four "i2c" libraries to find one that's stable under node.js, ('i2c-bus" is the one you want.) I'm not happy with the CPU usage to be honest, but it does actually work. I can push the mouse cursor around the desktop by tilting the sensor various ways, and even click on things with considerable difficulty.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://2.bp.blogspot.com/-mVHKSAD2nmw/V1j5p4JQcxI/AAAAAAAACjc/f0-MUcigx2MQII-Rui6z59edNtixn2jcgCLcB/s1600/IMG_0539.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://2.bp.blogspot.com/-mVHKSAD2nmw/V1j5p4JQcxI/AAAAAAAACjc/f0-MUcigx2MQII-Rui6z59edNtixn2jcgCLcB/s320/IMG_0539.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Size comparison against my usual mouse.</td></tr>
</tbody></table>
<br />
I'm using Gyros for movement and Acceleration for mouse 'clicks', which is quite different from the "accellerometer" mice I've seen before. The difference there is, accelerators measure "tilt" due to gravity. So in those cases, the mouse cursor acts a lot like the little steel ball in those closed maze puzzle games, rolling around on a flat surface.<br />
<br />
Sounds like a good idea at first, but you're asking people to keep the device perfectly flat in order to keep the mouse still. Most people don't intuitively know where "perfectly flat" is.<br />
<br />
Gyros sense <i>relative</i> rotational motion. The turning to and fro of an object. Gyros are the most important part of a Quadcopter's IMU, which I know well.<br />
<br />
Funny thing, I did a search and I don't think anyone else has built a gyromouse. They all go for accellerometer-mice. Well; technically the Wii-mote, but it also uses the IR lights, so it's a hybrid.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://4.bp.blogspot.com/-8zr3NerPhow/V1j5XzWblbI/AAAAAAAACi8/8F21qcqfUuQ8oVluvMxgA-fwU9BCkqIbQCLcB/s1600/IMG_0534.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://4.bp.blogspot.com/-8zr3NerPhow/V1j5XzWblbI/AAAAAAAACi8/8F21qcqfUuQ8oVluvMxgA-fwU9BCkqIbQCLcB/s320/IMG_0534.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Is this the world's only Gyromouse? <br />To be honest, I haven't really looked that hard.</td></tr>
</tbody></table>
<br />
GyroMouse works off the relative turning motion of the chip to simulate the movement of a mouse. And it detects the "freefall" moment of when you tap the device downwards as a mouse click. It feels horrible, needs extensive tuning, but it functions. I can click on menus. I might be able to extend it to detects 'taps' in various axes, so whacking this particular device from the sides might end up working as other buttons, or maybe the scroll wheel.<br />
<div>
<br />
I think that's called a "kinetic interface". Similar, although slightly different to "percussive maintenance". But that's kind of the point here... to experiment with what happens when you make a computer aware of it's physical state, and smart enough to respond and adapt.<br />
<br />
If you want to be mean, you can denigrate the whole thing as an over-complicated Wii-Mote. And you'd be right. It's a Wii-Mote that doesn't need the Wii. It does everything all by itself.<br />
<br />
Right now this is still at proof of concept stage, though later on I'll release the code when it's not quite as embarrassing, and the real hardware has arrived. Leave a comment if you're interested.</div>
Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com7tag:blogger.com,1999:blog-7271315367558300569.post-73780987670384588322016-06-03T23:58:00.000-07:002016-06-03T23:58:08.868-07:00Curiosity Physics DemoAnother video; made while testing out the physics engine scripting. This is my most complicated physics demo to date: a fairly complete mechanical model of the Curiosity Mars Rover:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/aLBVs-d-hBo/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/aLBVs-d-hBo?feature=player_embedded" width="320"></iframe></div>
<br />
Unlike the '<a href="https://www.youtube.com/watch?v=zwQLJwbQedc">Atomic Caffeine</a>' demo where you don't really notice that the behavior is a little un-physical, the rover is technically a real vehicle. And you can <i>feel</i> it go over every bump and step. There are moments I forget it's a simulation, and I built it.<br />
<br />
Actually, this is what the rover would drive like if it lost independent steering on all four corner wheels (they're not supposed to just spin freely like a broken shopping-trolley wheel like that, they're supposed to be precisely controlled) and was being driven purely on differential speeds, like a treadless tank. You can see why NASA likes this configuration, it's very robust.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-VRXBB8YUPZs/V1J0fMflCyI/AAAAAAAAChk/WVT7ibmblR4TuhZhyFK0z9rQAU1kQosgwCKgB/s1600/astromech%2B-%2Bcuriosity%2Bbuild%2B1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="225" src="https://1.bp.blogspot.com/-VRXBB8YUPZs/V1J0fMflCyI/AAAAAAAAChk/WVT7ibmblR4TuhZhyFK0z9rQAU1kQosgwCKgB/s400/astromech%2B-%2Bcuriosity%2Bbuild%2B1.png" width="400" /></a></div>
<br />
The 'hexcode' visual script editor has also come a long way in a short time. This was the initial build script, with a set of COLLADA model files containing the parts like wheels, chassis, camera mast, etc. arranged into 'reference frames' to check that everything was where it should be.<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-rmhjmHPfUOs/V1J0kLjLOAI/AAAAAAAAChs/5M9hrjrKGRYZIzHhkNfCNcjS9C6I0KERgCKgB/s1600/astromech%2B-%2Bcuriosity%2Bbuild%2B2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="225" src="https://1.bp.blogspot.com/-rmhjmHPfUOs/V1J0kLjLOAI/AAAAAAAAChs/5M9hrjrKGRYZIzHhkNfCNcjS9C6I0KERgCKgB/s400/astromech%2B-%2Bcuriosity%2Bbuild%2B2.png" width="400" /></a></div>
<br />
The new item/inventory system is working out well. Most of these designs are edited directly inside the browser from local storage. (No more google docs!) When I'm happy, a copy is uploaded to the web server for everyone to see.<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-YcdXDjyLsBE/V1J0mH4ldVI/AAAAAAAACh0/OVP780v5OPsp1-aZc_lZW97_rfGzI3b0QCKgB/s1600/astromech%2B-%2Bcuriosity%2Bbuild%2B3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="225" src="https://2.bp.blogspot.com/-YcdXDjyLsBE/V1J0mH4ldVI/AAAAAAAACh0/OVP780v5OPsp1-aZc_lZW97_rfGzI3b0QCKgB/s400/astromech%2B-%2Bcuriosity%2Bbuild%2B3.png" width="400" /></a></div>
<br />
Looks nice with an appropriate 'SkyShader' background, although Simulated Mars really hits the FPS, let me tell you.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-M1BBndm8Vj0/V1J0stbLgRI/AAAAAAAACh8/haK4XBbQo6MZkwi8xloZNFzK2vOuIrHgwCKgB/s1600/astromech%2B-%2Bcuriosity%2Bbuild%2B4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="216" src="https://3.bp.blogspot.com/-M1BBndm8Vj0/V1J0stbLgRI/AAAAAAAACh8/haK4XBbQo6MZkwi8xloZNFzK2vOuIrHgwCKgB/s400/astromech%2B-%2Bcuriosity%2Bbuild%2B4.png" width="400" /></a></div>
<br />
The fully completed 'program' for the rover removes the heirarchy, (all frames are attached to the root) and then physical constraints are created between them.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://3.bp.blogspot.com/-b9fTCDrWruw/V1J0ysCyViI/AAAAAAAACiE/8uiL4gZ8PjgiwURrvzKvgv1G5kIhpzRpACKgB/s1600/astromech%2B-%2Bcuriosity%2Bbuild%2B5.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="216" src="https://3.bp.blogspot.com/-b9fTCDrWruw/V1J0ysCyViI/AAAAAAAACiE/8uiL4gZ8PjgiwURrvzKvgv1G5kIhpzRpACKgB/s400/astromech%2B-%2Bcuriosity%2Bbuild%2B5.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Is it a bird? Is it a plane? No, it's a visual programming language!</td></tr>
</tbody></table>
<br />
Quite a lot of progress. Only a day before, the first attempts to import the rover looked like this:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://3.bp.blogspot.com/-EqRlMuU1eoY/V1J0zkZHKKI/AAAAAAAACiM/LZs7icjKYT8GDigVuJIKMgPQfIaEtImhgCKgB/s1600/astromech%2B-%2Bcuriosity%2Bfirst%2Bimport.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="216" src="https://3.bp.blogspot.com/-EqRlMuU1eoY/V1J0zkZHKKI/AAAAAAAACiM/LZs7icjKYT8GDigVuJIKMgPQfIaEtImhgCKgB/s400/astromech%2B-%2Bcuriosity%2Bfirst%2Bimport.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Explody</td></tr>
</tbody></table>
<br />
Other NASA models are also making an appearance in Astromech. The corner of the Deep Space Network 70m dish can be seen above, and here's the Z2 spacesuit standing on the bridge of my (fake) Starship Tyson over Simulated Mars.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://3.bp.blogspot.com/-xGVNvYOwfRs/V1J1KAHuhGI/AAAAAAAACiU/nRzN_6snIvQ3qd0e9Q5NxG_e9LixW4cZwCLcB/s1600/astromech%2B-%2Bfirst%2Bavatar.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="216" src="https://3.bp.blogspot.com/-xGVNvYOwfRs/V1J1KAHuhGI/AAAAAAAACiU/nRzN_6snIvQ3qd0e9Q5NxG_e9LixW4cZwCLcB/s400/astromech%2B-%2Bfirst%2Bavatar.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Hi!</td></tr>
</tbody></table>
<br />
That's actually the first "Avatar" in Astromech. The first humanoid figure in the system. And likely to become the default choice for user representation. Although in deference to the sensibilities of a friend of mine, I'm probably going to make a version in a gold lamé material.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://4.bp.blogspot.com/-MUCvw-2Cqbs/V1J54mvXFfI/AAAAAAAACig/KNrs9hHE99U7IvJ3OMnkxRVHrXWBCru_ACLcB/s1600/icon%2B-%2Bz2%2Bspacesuit.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="247" src="https://4.bp.blogspot.com/-MUCvw-2Cqbs/V1J54mvXFfI/AAAAAAAACig/KNrs9hHE99U7IvJ3OMnkxRVHrXWBCru_ACLcB/s320/icon%2B-%2Bz2%2Bspacesuit.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Mentally queue track "Journey of the Sorcerer" now.</td></tr>
</tbody></table>
<br />Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com2tag:blogger.com,1999:blog-7271315367558300569.post-57324104544775218432016-04-23T00:12:00.000-07:002016-04-23T00:12:05.750-07:00Why AMD/Radeon is Increasingly Dead To MeHistorically, I've been a fan of AMD. They make good, cheap processors. Their motherboard chipsets are pretty solid. In 2006, AMD bought out ATI technologies, and started selling graphics cards.<br />
<br />
When I built my latest system, I dropped a nice efficient Radeon R7 in there. I've had persistent problems with the machine, even after changing the motherboard, PSU, hard drives, OS, and cooling. The machine stability seems to wax and wane with the release of the AMD Catalyst drivers.<br />
<br />
I thought it was just me, but a friend recently put together a machine with the latest and greatest in AMD/Radeon R9 tech to drive a pair of 4K monitors, and her problems are even worse. There was a time it was stable, but drivers changed, and now the machine won't stay up for more than 20 minutes. Trying to run Elite Dangerous is an instant hard crash now. Maximizing the browser window will crash the machine 50% of the time.<br />
<br />
And it's not just us. The anger-boards are filled with similar stories. There are rumors several Games companies threatened to write their own Radeon driver set, because their customers were having so much trouble.<br />
<br />
The core problem has been ATI's historic secrecy, born from a time when graphics cards could be nice and proprietary, so long as you'd done the driver deal with Microsoft. Patent litigation abounded, as everyone tried to produce a 'better' anti-aliasing or MIP-mapping algorithm and sued everyone else for pushing pixels in near-identical ways. It was toxic.<br />
<br />
Since then, graphics cards have become commodity items. No-one cares, or even wants, non-standard bells and whistles. What we really, really need is direct access to the inside guts of the graphics card so we can use it for 'GPU compute' operations, to blat textured polygons onto the screen as fast as possible, and to write "shader programs" in a generic way that works on all computers.<br />
<br />
No-one wants "Tesselation" or "PhysX" because those are over-specific features that only do one thing, aren't standard, and can be replicated (often better) with general compute.<br />
<br />
NVIDIA has been a leader here with their 'CUDA' architecture. For years, they've allowed people to get up all inside their GPU and know exactly how all the blocks fit together, with no surprises. They figured out that in order for people to get best use out of your product, they have to know how it works.<br />
<br />
AMD Graphics also seems to use the necessity to install their OS drivers as a way to get "shovelware" onto the users machine. That's a dangerous trend, because it means they're now also in the business of selling their customers. Not just selling _to_ them. That creates a tension, that leads to 300-400 megabyte "driver downloads" filled with all kinds of crap.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/t0hK1wyrrAU/0.jpg" src="https://www.youtube.com/embed/t0hK1wyrrAU?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<div style="text-align: center;">
300Mb for a graphics card driver? </div>
<div style="text-align: center;">
You cannot be serious.</div>
<div style="text-align: center;">
<br /></div>
Not only that, it's pretty obvious that AMD Graphics are doing deals with the publishers of software and games to "tune" the performance of their applications.<br />
<br />
How do I know this? My graphics card crashed again this morning, (not the whole machine, just the card dropped and re-installed) and because I hunt down the causes of these things, I noticed the system log entry<br />
<br />
Process C:\Windows\System32\SET645.tmp (process ID:3660) reset policy scheme from [hex block]<br />
<br />
Since I'm not fond of anonymous ".tmp" executables changing my system settings, I had a look at the files. (There were several) They contain strings like:<br />
<br />
"http://www.amd.com/us/driverxml ReleaseVersion Catalyst_Version"<br />
"AMD PMP-PE CB Code Signer v201504130"<br />
<br />
...so I'm guessing they're automatic updates downloaded by Catalyst. If you look carefully at some of the files, you notice lists of executable names for games and other programs. Some games I've never had installed on my system. Some apps (like DirectX and Chrome) we all have.<br />
<br />
I'll include just a small sample of that list: (sorry about the UTF16 spaces)<br />
<br />
5 C O D - B l a c k O p s B l a c k O p s . e x e B l a c k O p s M P . e x e S k y r i m T E S V . e x e S t a r C r a f t 2 S C 2 * . e x e 3 D M a r k N E X T 3 D M a r k I C F D e m o . e x e 3 D M a r k I C F W o r k l o a d . e x e 3 D M a r k . e x e 3 D M a r k C m d . e x e M a s s E f f e c t 3 M a s s E f f e c t 3 * . e x e W h i t e L i s t A p p U n i g i n e . e x e S a n c t u a r y . e x e L e o _ D 3 D 1 1 . e x e M e c h a _ D 3 D 1 1 . e x e L a d y b u g _ D 3 D 1 1 . e x e C o r e T e c h 2 _ X 6 4 _ 1 0 . e x e C o r e T e c h 2 _ X 6 4 _ 1 1 . e x e C o r e T e c h 2 _ X 8 6 _ 1 0 . e x e C o r e T e c h 2 _ X 8 6 _ 1 1 . e x e B a t m a n A C . e x e h n g . e x e R e n e g a d e O p s . e x e S a i n t s R o w T h e T h i r d _ D X 1 1 . e x e G a m e C l i e n t . e x e A n n o 4 . e x e f c 3 _ b l o o d d r a g o n _ d 3 d 1 1 . e x e f c 3 _ b l o o d d r a g o n _ p _ d 3 d 1 1 . e x e f c 3 _ b l o o d d r a g o n _ r _ d 3 d 1 1 . e x e f c 3 _ b l o o d d r a g o n _ r t _ d 3 d 1 1 . e x e N Z A . e x e a r m a 3 . e x e z a t . e x e M a d M a x . e x e 3 D M a r k S k y D i v e r 3 D M a r k S k y D i v e r . e x e T a l o s . e x e T a l o s _ U n r e s t r i c t e d . e x e T a l o _ D e m o . e x e M u r d e r e d S o u l S u s p e c t F a t e G a m e - W i n 6 4 - T e s t . e x e M u r d e r e d . e x e H i t m a n A b s o l u t i o n H M A . e x e F i f a O n l i n e 2 F F 2 C l i e n t . e x e D i a b l o I I I D i a b l o I I I . e x e D i r t S h o w d o w n s h o w d o w n . e x e s h o w d o w n _ a v x . e x e s h o w d o w n _ d e m o . e x e s h o w d o w n _ d e m o _ a v x . e x e K r a t e r K r a t e r . e x e M a x P a y n e 3 M a x P a y n e 3 . e x e T J 3 T J 3 . e x e<br />
<br />
You'll note they have special detection of the 3DMark _benchmarks_. That's a worry by itself. How do you trust a video card driver that auto-detects benchmark software? (and presumably switches on specific settings just for that case)<br />
<br />
What's more concerning - if getting an entry like this is how you gain 'top performance' for your software (because apparently the basic video card drivers aren't up to the job by themselves) then how exactly does your company gets its optimal settings into this list? Does monetary compensation change hands? How much? Can Indie developers afford it?<br />
<br />
I'm sure there are nice excuses for this behavior, (like enabling new graphics modes for legacy games) but it also enables a whole class of dodgy practices for AMD and un-obvious behavior for the user. The AMD graphics drivers don't really do what you ask... they do it "better!", according to the latest update.<br />
<br />
What's amusing is - if the auto-update system didn't keep crashing and properly cleaned up it's temp files, I might never have seen this list in such obvious plain-text.<br />
<br />
So, I'm sick of this crap. For my next graphics cards, I'm going back to NVIDIA, to get me some of that sweet CUDA lovin' and stability. If that means changing the rest of my machine to Intel, well, so be it.<br />
<br />
<br />Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com2tag:blogger.com,1999:blog-7271315367558300569.post-80901958894757778742016-04-14T08:50:00.001-07:002016-04-14T08:50:23.032-07:00Astromech - Hex Editor PreviewHere's tonight's screenshot from Astromech: A very early preview of the new "Hex" Editor.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-l6bOnos5qBw/Vw-ypIsgRzI/AAAAAAAACgs/kTz7jMERdSo2ffgL48INwT87fzCsmmrpgCLcB/s1600/astromech%2B-%2Bfirst%2Bhexcode.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="308" src="https://4.bp.blogspot.com/-l6bOnos5qBw/Vw-ypIsgRzI/AAAAAAAACgs/kTz7jMERdSo2ffgL48INwT87fzCsmmrpgCLcB/s640/astromech%2B-%2Bfirst%2Bhexcode.png" width="640" /></a></div>
<br />
<br />
<div>
It doesn't edit hexadecimal code -it edits dependency trees. I'm sure at first the whole thing looks a little hand-drawn, a bit <i>too</i> good, but in fact all of the connection lines are auto-routed by Dijkstra's algorithm on a hexagonal grid - which I find gives more pleasing results than square grids, assuming you can handle the math.</div>
<div>
<br /></div>
<div>
This is a "first preview" of what's going to be the cap-stone of Astromech before I finally push it out the door - the integrated script editor needed to tie together all the other parts.</div>
<div>
<br /></div>
<div>
I'm trying to do an end-run around all the problems of text, keyboards, languages and localization, and comp.sci jargon in general - by having another tilt at one of the big windmills of computer science - Visual Code Editing. Representing code graphically, rather than as text.</div>
<div>
<br /></div>
<div>
Many before have tried and failed. I'd probably be doomed to fail if I also tried to create a 'generic' programming language, but I have a very specific set of needs that isn't fully Turing complete - mostly I just need to connect up predefined modules into processing chains which have well-behaved startup and shutdown semantics.</div>
<div>
<br /></div>
<div>
These scripts will be hidden inside every Astromech item and level, responding to clicks and collisions and requests to add new 3D models into the visible scene. They're already there, but large chunks of JSON are a pain to edit. I need something better.</div>
<div>
<br /></div>
<div>
This might be it.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com1tag:blogger.com,1999:blog-7271315367558300569.post-14918360655329195082016-04-10T18:32:00.002-07:002016-04-10T18:50:29.268-07:00I just got waved to from VRThis is one of those "the future is here" moments. The first 'tv show' transmitted from VR.<br />
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/PT3jZyOXqzU/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/PT3jZyOXqzU?feature=player_embedded" width="320"></iframe></div>
<div>
<div style="text-align: center;">
<span style="font-size: x-small;">The Foo Show - Firewatch Tower Tour Episode</span></div>
<br />
It's clunky, their arms don't move properly because the inverse kinematics forgot to include shoulders, and in terms of content - it's a bunch of people wandering around a small room looking intensely at every-day objects like teenagers on their first acid trip.<br />
<div>
<br /></div>
<div>
But it's also groundbreaking. Will's completely right about how little it takes before you anthropomorphise their chunky avatars into "real people". Ten minutes in and I forgot they were polygons.</div>
<div>
<br /></div>
<div>
And at the end, they wave goodbye. Such a simple, utterly human gesture. The first time anyone has waved to me from VR.</div>
<div>
<br /></div>
<div>
<a href="http://foovr.com/">FooVR.com</a> is in early days still. The resolution will improve, structured motion systems will replace the polygons with photogrammetry, the inverse kinematics will get better, and the "virtual sets" will evolve and explore the limits of what's visually possible.<br />
<br />
But they got first post. And that's what matters.</div>
</div>
<div>
<br /></div>
<div>
I've been expectantly waiting for this day for over 20 years, and pushing the technology and art behind it. This concept is as old as Gibson's "Neuromancer", or Simmon's "Hyperion". Now it's real. We finally get to explore not just the idea, but it's consequences.</div>
<div>
<br /></div>
<div>
Well done, Will Smith. I tip my hat to you, sir. </div>
Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com1tag:blogger.com,1999:blog-7271315367558300569.post-89397014605042104902016-04-06T19:52:00.001-07:002016-04-06T19:52:36.722-07:00Inventory Management in Online GamesInventories! They've been in every RPG and MMO for the entire history of computer games. Type 'invent' into 'advent' if you don't believe me.<br />
<br />
So, you'd think the finer points of "inventory management theory" would have been long hashed out by the games development community, and you'd be wrong.<br />
<br />
Gamasutra has two articles; one in 2010 and then another in 2015. That's the opposite of a hot topic.<br />
<br />"<a href="http://www.gamasutra.com/blogs/JohnMawhorter/20100904/87970/Inventory_Management_Sucks.php">Inventory Management Sucks!</a>" is a short reminder on what the player probably expects out of their inventory system. Too short.<br /><br />"<a href="http://www.gamasutra.com/blogs/ElliotPinkus/20150120/234511/Loot_Quest_From_Ruminations_to_Release.php">Loot Quest: From Ruminations to Release</a>" gets much more into the issue, from the point of view of the choices one makes building the inventory system, and how it affects the game mechanics. Those guys make explicit and careful choices, because "a streamlined inventory was seen as one of the highest priorities" for their "heroes-with-equipment premise".<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://www.gamasutra.com/db_area/images/blog/234511/IMG_0064.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://www.gamasutra.com/db_area/images/blog/234511/IMG_0064.jpg" height="300" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Elegant Simplicity</td></tr>
</tbody></table>
<div>
<br /><div>
<br />
There's also "<a href="http://gamedevelopment.tutsplus.com/articles/designing-an-rpg-inventory-system-that-fits-preliminary-steps--gamedev-14725">Designing an RPG Inventory System That Fits: Preliminary Steps</a>" which is a good overview of the historical 'categories' of inventory management, and reviews the major systems seen in games over the last decade.<br />
<br />
Of course, you can't discuss inventory systems without referring to the grand-daddy of the genre, World of Warcraft. Although more as a cautionary tale, many would say.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://i136.photobucket.com/albums/q195/Chatmay/WOW%20Pics/WoWScrnShot_020712_161153.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://i136.photobucket.com/albums/q195/Chatmay/WOW%20Pics/WoWScrnShot_020712_161153.jpg" height="303" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Loot Porn. </td></tr>
</tbody></table>
<br />
Something that's taken for granted with all these 'Inventory' systems is that items are <b>scarce</b><i style="font-weight: bold;">.</i> They are games, after all. Items in games are intended to be rewards, and often consumable. This can lead to entire 'virtual economies' for 'goods' that are really just one entry in a database table somewhere, linking a pre-defined 'item' object into your inventory space.<br />
<br />
When you 'craft' an item in these MMOs, what you're doing is collecting a bunch of predefined database tokens which you exchange (via server calls) for another predefined database token. You're not making anything <b>new</b><i style="font-weight: bold;">.</i> Even though it can sometimes feel 'unique' because stats are randomly rolled or cute names are chosen from a big list.<br />
<b><i><br /></i></b>
Remember how innovative it seemed when, in Diablo II, you got access to the ability to <b><i>name </i></b>one item? With your own name, but still. That mechanic alone resulted in entire generations of characters being created with names that looked good on a sword.<br />
<br />
And yet, the amount of time and effort trading these game items has been incredible. Big companies like Blizzard have had to face the question of whether allowing characters to transfer items in-game will create an entire secondary 'trading markets' and whether they allow that. (Or, since you can't stop it, whether they banhammer the players they catch, or try to set up their own market and get a slice of that pie.)<br />
<br />
As you might know, I'm writing a game-ish VR environment thingy called "Astromech". Think of it as a level designer. You place geometry, define what the sky looks like, etc.<br />
<br />
I found I was building an "Asset Manager" for all the content that goes into a level. And because Astromech is a 'world builder', I like the concept of just picking up 'items' in one level, putting them in your 'inventory', and then dropping them into others.<br />
<br />
That means that, in astromech, Items don't come out of a predefined set. They start as directories of asset files, from which visible things in the level are instanced. They are 'program scripts'.<br />
<br />
That makes "item crafting" in Astromech a whole new thing. For a start, items can be created by just dragging a filesystem directory into the inventory screen. The directory is rifled through for metadata and assets, and you get a new inventory item! Want a different item, then edit the files and go again! Want to change the icon or title or comment, then do!<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://4.bp.blogspot.com/-3Nj8cJv0Jp8/VwW9qCTNSEI/AAAAAAAACgc/wXN6-P5aWxMqxX35FY0uh7_LggE9DcdRQ/s1600/astromech%2B-%2Binventory%2B1.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="241" src="https://4.bp.blogspot.com/-3Nj8cJv0Jp8/VwW9qCTNSEI/AAAAAAAACgc/wXN6-P5aWxMqxX35FY0uh7_LggE9DcdRQ/s400/astromech%2B-%2Binventory%2B1.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The beginnings of Inventory Management in Astromech. <br />Damn, where did I leave that dataset of all known asteroids in the solar system?</td></tr>
</tbody></table>
<br />
That's a concept most games go to extreme lengths to <i>avoid</i>. Because in a competitive system, you don't want other people arbitrarily increasing their power by modifying their tools. But in a co-operative system about building "virtual machines" to solve real-world problems, you actually <b>want </b>the other "players" to be all that they can be.<br />
<br />
Because if they build some incredible item that detects bird-song in audio streams, or reconstructs 3D scenes from multiple photos, (using the DSP 'components' that Astromech provides) then that becomes something they can share. Something not originally programmed into the 'game' by me.<br />
<br />
How? Just walk up to their avatar in VR and open their 'shop' of public items. All the tropes of inventory trading in MMOs are just the accountancy of database management dressed up in a pikachu suit.<br />
<br />
This allows other niceties, such as users storing the items in their own local machines, and doing direct peer-to-peer sharing without central servers. "Duping items" is <i>assumed</i>. Some are just URLs.<br />
<br />
There's a LOT of little details to work out, but the paradigm of item sharing in MMOs is far more friendly than file sharing systems like dropbox. I've prototyped the issues now, and I can see where it's all going.<br />
<br />
Minecraft introduced "item transforming items" like furnaces and workbenches. I'm going a step further, to "item compilers" which effectively craft unique pieces of functional 'software' from source components. That's what an Astromech item <i>is<b>.</b></i><br />
<br /> Astromech's real job is to create a space where all these "virtual devices" can play together in a shared 3D environment, and connect with each other, while protecting the user's browser from malicious code, and devices from each other. (Because, this is all in a web page!)<br />
<div>
<br /></div>
<div>
<div>
Remember, the secret is to bang the rocks together, guys.</div>
</div>
<div>
<br /></div>
Possibly the closest equivalent available today is in Steam's VR demo "The Lab". </div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/GiDBp6OnsKY/0.jpg" src="https://www.youtube.com/embed/GiDBp6OnsKY?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
Towards the end, after the Tested team has played most of the mini-games, the table in the hub room fills up with devices they've 'collected' from the various games. Balloon inflaters, bows and arrows... and they begin to delight in the way these tools can be played with and combined, just for their own sakes. And unlike most games, the tools aren't purely destructive.</div>
<div>
<br /></div>
<div>
<br /><br /> </div>
</div>
Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com2tag:blogger.com,1999:blog-7271315367558300569.post-10725345005615914802016-03-03T18:24:00.002-08:002016-03-03T18:24:40.690-08:00Real-Time 3D Fractals videoSo I procrastinated from other things and made a video showing off the new Skyshaders in Astromech, set to some of the music I listened to while developing it.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/CMTEZxYayLQ/0.jpg" src="https://www.youtube.com/embed/CMTEZxYayLQ?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<br />Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com0tag:blogger.com,1999:blog-7271315367558300569.post-19335849804029625402016-02-26T00:56:00.001-08:002016-02-26T00:56:22.173-08:00Procedural SkyshadersInstead of spending ages building geometry for your 3D worlds, why not just define a mathematical 'field' function that uses fractals and folded space and pseudo-random noise and all kinds of other tricks? It certainly saves on the disk space.<br />
<br />
Think of it as a kind of disguised Mandelbrot set. You can generate infinite variation from a carefully iterated function, and often you can zoom all the way down and variation just keeps appearing.<br />
<br />
The hard part is combining these 'shader' backgrounds with actual geometry. Until recently, this wasn't possible to do in the browser, but a new WebGL extension (EXT_frag_depth) allows you to control the 'fragment depth' of individual pixels, (ie: skyboxes can now push parts of themselves into the foreground.)<br />
<br />
Fortunately, it's a trick I've managed to implement in Astromech.<br />
<br />
So now, I can do this kind of thing: take a classic 3D model (saved to COLLADA format from blender) and compose it with a procedurally generated 'skybox' that I call a 'skyshader'.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-qd54f2C4dAg/VtAO3ArOLQI/AAAAAAAACe4/WyaOfYk-RvA/s1600/skyshaders-hills.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="252" src="https://3.bp.blogspot.com/-qd54f2C4dAg/VtAO3ArOLQI/AAAAAAAACe4/WyaOfYk-RvA/s400/skyshaders-hills.png" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-F57BcLSG9Uc/VtAO0etrh9I/AAAAAAAACew/EHePzfcdwnY/s1600/skyshaders-hyperlepsy.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="255" src="https://3.bp.blogspot.com/-F57BcLSG9Uc/VtAO0etrh9I/AAAAAAAACew/EHePzfcdwnY/s400/skyshaders-hyperlepsy.png" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-GPjKvlFejjs/VtAO2vU0TuI/AAAAAAAACe0/VnCO2lgPJdY/s1600/skyshaders-mountains.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="222" src="https://2.bp.blogspot.com/-GPjKvlFejjs/VtAO2vU0TuI/AAAAAAAACe0/VnCO2lgPJdY/s400/skyshaders-mountains.png" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-bSWdjaXnQdY/VtAO5HDlFXI/AAAAAAAACe8/9QYMmNe4Es4/s1600/skyshaders-stormoid.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="250" src="https://1.bp.blogspot.com/-bSWdjaXnQdY/VtAO5HDlFXI/AAAAAAAACe8/9QYMmNe4Es4/s400/skyshaders-stormoid.png" width="400" /></a></div>
<br />
I hope to do a video shortly, showing off these fully animated 3D environments.Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com1tag:blogger.com,1999:blog-7271315367558300569.post-15414083263672697752016-01-03T19:35:00.000-08:002016-01-03T19:35:19.254-08:00Web Crypto API - First ImpressionsSo, who knew that the W3C was, among other things, defining a <a href="http://www.w3.org/TR/WebCryptoAPI/">Javascript API to do hard crypto</a>?<div>
<br /></div>
<div>
I've had weeks messing around with the API, listening to the lectures, and reading the other blogs. First thing you notice... there's not a lot of detail, and not a lot of stories of people using it. I assume they're all quietly working hard.</div>
<div>
<br /></div>
<div>
The best I've found is probably <a href="https://github.com/diafygi/webcrypto-examples">Daniel Roesler's webcrypto-examples</a> - an essential resource for anyone writing real code</div>
<div>
<br /></div>
<div>
And actually, there's a lot of blustery crap in blogs about how you can never possibly do <i>real</i> crypto in the browser. Because why? Um, because you can't trust the browser, that's why. I mean, sure, otherwise you have to trust the operating system, but that's <i>completely different</i>.</div>
<div>
<br /></div>
<div>
And besides, Javascript isn't a real language anyway, they'll grumble. Because nobody likes losing the language wars.</div>
<div>
<br /></div>
<div>
So first, the good. Here's what it looks like when you've got a modern (Chrome / Mozilla) browser and you can use all the HTML5 :</div>
<div>
<br /></div>
<div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;">var crypto_subtle = window.crypto.subtle || window.crypto.webkitSubtle;</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"><br /></span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;">function create_identity(meta,algo) {</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> meta = meta || {</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> name: "anonymous",</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> created: (+new Date),</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> };</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> algo = algo || {</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> name: "ECDH",</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> namedCurve: "P-521", //can be "P-256", "P-384", or "P-521"</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> };</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> var ident = { algo:algo, meta:meta };</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> return crypto_subtle.generateKey(</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> algo,</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> true, //whether the key is extractable (i.e. can be used in exportKey)</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> ["deriveKey", "deriveBits"] //can be any combination of "deriveKey" and "deriveBits"</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> )</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> .then( function (key) {</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> console.log(key);</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> return Promise.all([</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> crypto_subtle.exportKey(</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> "jwk", //can be "jwk" (public or private), "raw" (public only), "spki" (public only), or "pkcs8" (private only)</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> keys.privateKey //can be a publicKey or privateKey, as long as extractable was true</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> )</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> .then(function(keydata){</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> ident['private'] = keydata;</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> }),</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> crypto_subtle.exportKey(</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> "jwk", //can be "jwk" (public or private), "raw" (public only), "spki" (public only), or "pkcs8" (private only)</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> keys.publicKey //can be a publicKey or privateKey, as long as extractable was true</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> )</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> .then(function(keydata){</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> ident['public'] = keydata;</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> }),</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> ]).then(function() {</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> return ident;</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> });</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;"> });</span></div>
<div>
<span style="font-family: Verdana, sans-serif; font-size: xx-small;">}</span></div>
</div>
</div>
<div>
<br /></div>
<div>
That code is complete, no library dependencies. You could cut-and-paste it into any script. What does it do? It generates an Elliptic Curve public/private keypair, serializes the keys, and returns the whole chunk of JSON-y goodness in a Promise. </div>
<div>
<br /></div>
<div>
<div>
It's the core operation to create a "cryptographic identity" for future operations like signatures or encryption or link security.</div>
</div>
<div>
<br /></div>
<div>
If you don't know about EC6 Promises that are now standard in all browsers, go read about that instead. Go, go! That's even more important!</div>
<div>
<br /></div>
<div>
So, what's the bad news? Well, Apple and Microsoft are being predictably slow in implementing the good and useful (the less kind would say the "not horribly broken and unsafe") algorithms that we badly need, such as the Elliptic Curve Diffie Hellman (ECDH, use above) or Elliptic Curve Digital Signature Algorithm (ECDSA) that are about 100 times faster, and yet 10 times more secure, than the previous generation of RSA-based algorithms.</div>
<div>
<br /></div>
<div>
You may have heard that everyone wants to deprecate SHA-1 as being utterly broken now. Guess which algorithm has near-universal support across all browsers?</div>
<div>
<br /></div>
<div>
There's a reason for this, and that reason is at the center of a large fight. The W3C crypto API does what the W3C usually does - it standardizes, but without mandating a standard. If that sounds odd, it's basically the process of writing down what everyone currently does (the "state of the art") and saying "It's ALL legal!".</div>
<div>
<br /></div>
<div>
I personally don't mind this approach. It recognizes the fact that you can't <i>force</i> the browser makers to do squat. No matter how many "thou shalls" you put in the spec, they won't if they don't want to. So they best you can do is get all the documentation out into the open.</div>
<div>
<br /></div>
<div>
After a couple of years, all the crap shakes itself off, and we get left with a minimal core of useful tools that actually achieve a purpose, and work in the real world. Then there's usually a V2 of the specification which normalizes <i>that</i>.</div>
<div>
<br /></div>
<div>
The browsers already have extensive crypto stacks, SSL layers, key vaults, etc. So the API makes it easy for the browsers to expose a lot of that existing machinery, even though most of it isn't that useful inside the sandbox. It's what they've got for now, and they get to show it off. That's why all the browsers offer SHA-1 despite it being horribly broken - 'cause it was already there.</div>
<div>
<br /></div>
<div>
With the Web Crypto API, that means 80% of the spec is dead and pointless on arrival, Old crap that should have been left to rot, but got dragged along because some company wanted the backwards compatibility with some obscure password system, and didn't want to spend time and money working on writing the good stuff. </div>
<div>
<br /></div>
<div>
The spec is almost unreadable, even for a W3C recommendation, and I've been doing this a while. hundreds of pages long, and it still excludes a lot of the important technical detail via references to IEEE documents.</div>
<div>
<br /></div>
<div>
But that's OK. This is Javascript. Just dance around the landmines, and you'll be fine.</div>
<br />(For a full map of the landmines, try <a href="https://diafygi.github.io/webcrypto-examples/">https://diafygi.github.io/webcrypto-examples/</a> or the spreadsheet of <a href="https://docs.google.com/spreadsheets/d/14oTKnccypDRieszGLV7GbZXcIai0qLYOwgk_ELIj5A0/pubhtml">Browser Support for the Web Cryptography API</a> )<div>
<br /></div>
<div>
The browser makers that don't come up to snuff fast enough, they'll just get a polyfill. Sure, their browsers still won't be 'secure', but they'll be able to talk to our browsers, which suddenly are. The weak links will get pressure applied until they crack.</div>
<div>
<br /></div>
<div>
To be honest. Microsoft Edge is looking pretty good here. For most of the new HTML5 features, I'm pretty impressed with The Edge, and I'm happy to put it close 3rd after tied Chrome and Mozilla, but clearly closing the gap. They don't have ECDH yet, but it smells like they're close... certain error messages have the feeling of "not implemented yet" rather than "Not a feature!", if you know what I mean. They're even talking about protecting CryptoKeys stored in indexedDB on Edge at the OS level, which is the sort of thing that shows real thoughtfulness. </div>
<div>
<br /></div>
<div>
Safari is becoming the great standards holdout, especially so on iOS. Which is a big shame, because most of these new technologies are especially useful on mobiles. The fact that you can generate a secure Elliptic Curve key in milliseconds, even on low-CPU mobiles, (rather than seconds for RSA) means they're more useful and save battery life. (And that's the last platform you want to polyfill math routines on.)</div>
<div>
<br /></div>
<div>
The lack of ECDH/DSA on iOS is probably the biggest thing holding back HTML5 apps from communicating securely with each other, as general practice. But Apple's walking a tightrope there... if HTML5 pages become too capable, (with offline storage and crypto and media systems) who will visit the App Store?</div>
<div>
<br /></div>
<div>
But I'm not worried. This standard is being implemented surprisingly quickly, and as I said, there's a core of about 20% of the API which does some wonderful, fantastic, critically important stuff that the web has been waiting for, for <i>decades</i>. </div>
<div>
<br /></div>
<div>
That's the reward for anyone willing to dive deep into the API. I'm sure over time, there'll be some handy jQuery functions that make it all easy. But remember, they named it "subtle" for a <i>reason</i>. </div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com0tag:blogger.com,1999:blog-7271315367558300569.post-78241868155003608562015-12-20T18:06:00.003-08:002015-12-20T20:31:53.651-08:00You can't take the sky from meOh, but the FAA is trying hard!<br />
<br />
So full disclosure first - I'm not a US citizen, and I really, really like flying robots. (I've built four of them.) I'm also a big fan of logic and consistency, which used to be a popular band back in the day but not so much anymore.<br />
<br />
In the US, the Federal Aviation Administration has announced it will require all "drone" operators to register with the agency. Their definition of "drone" includes any remote controlled flying device over .55lb. (250 grams) That means quads, multis, foam planes, helicopters, blimps, balloons, and possibly Dune Buggies if they have too much 'hang time after sweet jumps. (it's unclear)<br />
<br />
Paper planes are still fine. Any uncontrolled thrown thing is fine. 50-foot Frisbees are allowed. Rockets are cool. If you put foam wings on your iPhone, you're in a grey area deeper than shadow.<br />
<br />
I'm not in-principle against enhanced safety, but this doesn't do that. The word "Overreach" is being used a lot. Congress explicitly said they <i>couldn't</i> do this. This moves the FAA from regulating a few dozen major airlines, to regulating the behavior of millions of private US citizens.<br />
<br />
Quick review of what the FAA is: It got it's major "powers" in the 1960s, at a time when passenger planes were colliding over New York and dropping flaming wreckage on sleeping people in their apartments. People didn't like that. So the response was to invent the Air Traffic Control system and give the FAA powers over civil aviation, instead of letting the Airlines make up their own rules.<br />
<br />
This has done a great deal to improve air safety. But it should be noted that planes still crash on New York quite a lot. There were the famous 11th September 2001 incidents, but who remembers Flight 587? which two months later crashed onto Queens because Air Traffic Control had told them to take off into the backwash of another plane, and some "aggressive piloting" caused the tail stabilizer to snap off. At a time when you'd think they'd be paying attention.<br />
<br />
In fact, if you look at the big accidents (rarely deliberate), they're all caused either by pilots crashing into things they couldn't see, (like mountains) or Air Traffic Control directing them to crash into things they didn't know were there (like other planes).<br />
<br />
Not a single aviation fatality has actually occurred because of RC hobby planes. Which have also been flying since the 60's, long before modern brushless motors and batteries. (The 'gasser' era.)<br />
<br />
Military drones have caused crashes, it's true... but not Hobbyists. In fact, there have been 400 major accidents caused by US military drones. (which are the size of a car, and often armed) They once hit a C-130 Hercules. (literally, the broad side of a barn) But the FAA doesn't regulate military air traffic. And it likes to <a href="https://www.modelaircraft.org/gov/docs/AMAAnalysis-Closer-Look-at-FAA-Drone-Data_091415.pdf">exaggerate the civilian threat</a>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://ecx.images-amazon.com/images/I/61yKGx1QoxL._SL1500_.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://ecx.images-amazon.com/images/I/61yKGx1QoxL._SL1500_.jpg" height="300" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">One of these things is not like the other.</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://asset1.djicdn.com/uploads/brand_site_product/cover/6/small_1@2x.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://asset1.djicdn.com/uploads/brand_site_product/cover/6/small_1@2x.png" height="320" width="320" /></a></div>
<br />
So, the FAA has announced it will create a "Drone Registry", so that anyone who intends to do bad things with a drone will helpfully write their details on the device, and this will help police track them down and arrest them for bad behavior.<br />
<br />
No, really! That's their cunning plan. Some cynical observers suggest this is just stage 1, and future stages will require anyone buying an RC device to provide registration at Point Of Sale, otherwise the whole concept is utterly useless. And then they'll have to regulate batteries, motors, and computers, because otherwise you just buy the parts off eBay and build it yourself.<br />
<br />
Or alternately, if a Policeman sees you flying in a park, they can ask for your registration and thereby keep the sky safe from bad people.<br />
<br />
So, all we have to do to eliminate the "drone threat" is to put millions of US citizens (many of them children) into a huge database that will be <a href="http://www.forbes.com/sites/johngoglia/2015/12/18/faa-finally-admits-names-and-home-addresses-in-drone-registry-will-be-publicly-available/">publicly accessible</a> by anyone who wants their phone number and home address. The FAA will have enforcement powers over every family in the country.<br />
<br />
One of my favorite things is the $5 registration fee. That doesn't sound like much, true, but that's also the same cost to register a full size Boeing 747 Jumbo Jet. Another sign that the FAA doesn't really distinguish between a hundred tonnes of flying metal and a piece of motorized foam-board.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://upload.wikimedia.org/wikipedia/commons/0/0a/Boeing_787_first_flight.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="212" src="https://upload.wikimedia.org/wikipedia/commons/0/0a/Boeing_787_first_flight.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">This also costs $5 to register with the FAA.<br />
It's a real one.</td></tr>
</tbody></table>
<br />
<br />
Amazingly, the US congress told the FAA they couldn't do this. The FAA went and did it anyway. Despite long-standing legislation that reads:<br />
<blockquote class="tr_bq">
<br />
Notwithstanding any other provision of law relating to the incorporation of unmanned aircraft systems into Federal Aviation Administration plans and policies, including this subtitle, the Administrator of the Federal Aviation Administration may not promulgate any rule or regulation regarding a model aircraft, or an aircraft being developed as a model aircraft, if—<br />
(1) the aircraft is flown strictly for hobby or recreational use;<br />
(2) the aircraft is operated in accordance with a community-based set of safety guidelines and within the programming of a nationwide community-based organization;</blockquote>
Meanwhile the AMA (The Academy of Model Aeronautics, one of those "nationwide community-based organizations" the legislation mentions) has <a href="http://amablog.modelaircraft.org/amagov/2015/12/17/hold-off-on-registering-model-aircraft/">told all it's members to hold off on drone registration</a> while they try to sort through all the conflicting reports. Latest news is that they intend to take legal action to fight the new rules.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/Supreme_Court_US_2010.jpg/350px-Supreme_Court_US_2010.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="213" src="https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/Supreme_Court_US_2010.jpg/350px-Supreme_Court_US_2010.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">And now, these people might get to weigh in. <br />
Including the Notorious RBG!</td></tr>
</tbody></table>
<br />
So, in summary: the FAA wants every hobbyist over 13yo to put their details in a public database, (because, y'know, privacy of the general public is important...) contrary to existing law, and the leading community organisation wants to take it to court. Hobbyists are furious. None of the new rules will make the skies any safer.<br />
<br />
It's a path that treats RC craft purely as a threat to "real airspace users", and ignores the immense opportunities. And it also puts the FAA on a collision course with civil liberties for the American public, and <i>that's </i>the kind of thing that gets them hauled before the Supreme Court which might strip them of their powers as unconstitutional overreach, (you can't even force Americans to register their guns!) and we'll have <i>no</i> oversight, which is even worse than bad oversight.<br />
<div>
<br /></div>
It's a shambles. A hypocritical, pointless, mess. Years will be lost fighting the "freedom vs order" civil war, instead of just pushing for technological solutions to what are essentially technological problems. (hint: GPS broadcast beacons & official listed "crashing zones" for RC craft that need to get out of the way of emergency crews. So models can automatically go "If I sense a medivac chopper nearby, I'll crash myself in the nearest zone".)<br />
<br />
Instead, I'm sure everyone is busy stripping off their backups, flight loggers, and safety gear - to fit under the 0.55lb weight limit. Those parachute systems are heavy, y'know.<br />
<br />
<br />
<br />Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com0tag:blogger.com,1999:blog-7271315367558300569.post-57759648962036882812015-12-03T18:36:00.001-08:002015-12-03T18:36:06.098-08:00Don't Do This #102 - GPS and raspividIf you're like me, you've often thought, "I really need GPS on a high-resolution camera, and probably accelerometers too, so I can do photogrammetry."<br />
<br />
OK, maybe you don't. Even my spellchecker doesn't like the word "photogrammetry", which is when you take a whole bunch of photos of something with the intention of creating a 3D model (or other measurement) from the imagery.<br />
<br />
Like what land surveyors do when they fly over with cameras to create topological maps. And like that, it really helps to know <i>exactly</i> where you were, and how the camera was positioned. A lot of the new algorithms can get by without it, but there's a time cost, and a lot of pathologies can be avoided if we start with a good bundle estimate.<br />
<br />
Here's what I did as a first go:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-lHgIR4qBE0k/VmDxWP4QjNI/AAAAAAAACcs/3Bhmp7vf5RM/s1600/IMG_0446.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="297" src="http://3.bp.blogspot.com/-lHgIR4qBE0k/VmDxWP4QjNI/AAAAAAAACcs/3Bhmp7vf5RM/s400/IMG_0446.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">An Aerial Photograph of my Aerial Photography Machine</td></tr>
</tbody></table>
<br />
That's a Raspberry Pi model A, with the 2k camera and WiFi modules, connected to a UBlox Neo6 GPS I got from Hobbyking last year. Less than $90 of stuff, most of which has been used in other projects. (And will again)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-Ns5wwIiOaQU/VmDyHG40fzI/AAAAAAAACdQ/XrEeKfFnn84/s1600/IMG_0442.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="298" src="http://3.bp.blogspot.com/-Ns5wwIiOaQU/VmDyHG40fzI/AAAAAAAACdQ/XrEeKfFnn84/s400/IMG_0442.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Techno-periscopes Up!</td></tr>
</tbody></table>
<br />
<br />
So here's what you need to know first: <b>It doesn't work.</b><br />
<b><br /></b>
Well, I mean all the independent bits work fine, but not all together. That's the point. To spoil the ending: When the camera is operating, so much multi-megahertz digital interference is generated by the <i>flat cable </i>connecting the camera module, that the GPS loses signal lock.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/--CrevDfw6EI/VmDyen_afOI/AAAAAAAACdc/vImYuVjYRlY/s1600/IMG_0447.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="298" src="http://3.bp.blogspot.com/--CrevDfw6EI/VmDyen_afOI/AAAAAAAACdc/vImYuVjYRlY/s400/IMG_0447.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Thar's yer problem right thar, boyo! The big flat white thing <br />what's right near the little square doodad. And all bendy, too!</td></tr>
</tbody></table>
<br />
I'm sure I could also make a gripping yarn about how I bravely tracked down and cornered the bug, and how developing many features at once (streamed low-latency WiFi video, plus GPS) is a great way to <i>find</i> the problems, but leave yourself very confused about what <i>causes</i> them.<br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-W_jjRvcBhXo/VmDxgPYhwYI/AAAAAAAACc4/AqrR7OmJpkM/s1600/IMG_0444.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="298" src="http://4.bp.blogspot.com/-W_jjRvcBhXo/VmDxgPYhwYI/AAAAAAAACc4/AqrR7OmJpkM/s400/IMG_0444.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Close-up of the connections, showing how easy it is to wire a 3.3v GPS to the Pi.<br />Standard linux 'gpsd' is used to decode the signals.<br />The plastic cap of the left is just to protect that end of the connector from physical damage/shorts.</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
For a long time I assumed it was the WiFi streaming part, since it's an RF transmitter, and the GPS is an RF receiver, and all of it is cheap as beans, so logically... but no! Those parts are well engineered and stay out of each others bandwidth. You can WiFi and GPS just great. But the moment you start recording video to /dev/null, the GPS lights go out. That was the clincher.</div>
<br />
If you're taking still photos, it's mostly fine. The GPS can stay locked on, and the brief static bursts during the camera shots are ignorable.<br />
<br />
But I wanted <i>video</i>. And the moment you open up the throttle, it all fails.<br />
<br />
Now, the obvious potential solution is to wrap a foil shield around the flat ribbon cable, especially where it bends, but that's something I'll need to do with great care, otherwise I've potentially got bits of foil flapping against the main electronic components and that's when the magic smoke comes out. There's also the question of how much of the digital path is exposed on the board. That would be harder to fix.<br />
<br />
Perhaps a ground plane to shield them from each other; but shoving sheets of tin or copper in between is going to cause other issues, like making the WiFi directional, and other near-field effects. Argh.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-IHvfPsQgRMk/VmDx5VSS4UI/AAAAAAAACdE/pegfUORpelQ/s1600/IMG_0448.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="http://2.bp.blogspot.com/-IHvfPsQgRMk/VmDx5VSS4UI/AAAAAAAACdE/pegfUORpelQ/s320/IMG_0448.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">So, you're saying the correct solution is a tiny Tinfoil Hat for the electron gnomes?<br />Riiigghhtt...</td></tr>
</tbody></table>
<br />
Also, while the GPS and cable are pretty much right next to each other for illustrative purposes, I can assure you I tried moving the modules as far as I could (cable allowing) and it didn't help. I'm sure I could manage it with a long enough GPS extension cord, but if it can't fit in the one box, It's not very convenient.<br />
<br />
But it you have a choice, plan on spreading the pieces out. That's probably your best bet.<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
So, alas, I don't have any guaranteed solutions to the problem yet. But I wanted to warn 'ye anyways.Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com0tag:blogger.com,1999:blog-7271315367558300569.post-60439618132641578072015-11-25T20:34:00.000-08:002015-12-10T21:59:29.121-08:00The Infinite Reality ShowI tend not to spend too much time talking about the long-term intentions of the work I'm trying to do with Astromech. Partly because I'm working it out as I go, and mostly because I've heard no end of "It's gonna be great!" exhortations in my life that turned out to be vaporware and I don't want to be That Guy.<br />
<br />
I wanted Astromech to get to the point where it could provably do the essentials, before talking about the possibilities. In the last few weeks, those essentials have come together.<br />
<br />
Data acquisition from commodity image sensors, check. Real-time DSP of the data using Fourier transforms, check. FITS Compression of the data using Harr transforms, finished a few days ago. Distributed comms working. Video comms working. Thousands of lines of '3D visualization' code.<br />
<br />
So, what's the point? It's all about mapping reality. Let's take this in stages:<br />
<br />
<b>Observation</b><br />
<br />
If you want to see the universe, you need to point a telescope at the sky. There's a great deal of optical and mechanical engineering involved, but you can shortcut that and buy a surprisingly good 6-inch Maksutov-Cassegrain off ebay for a few hundred bucks.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-AJvlgDx94f4/VlaAyOuov4I/AAAAAAAACbs/cbFAJOxKC3Y/s1600/vlcsnap-2013-06-17-22h48m46s249.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="299" src="http://4.bp.blogspot.com/-AJvlgDx94f4/VlaAyOuov4I/AAAAAAAACbs/cbFAJOxKC3Y/s320/vlcsnap-2013-06-17-22h48m46s249.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Most of science is about taking picture of fuzzy blobs. <br />
Here's my first image of Saturn.</td></tr>
</tbody></table>
<br />
For most people, this is where the hobby ends. Every now and again the dust gets blown off, they observe Saturn, get their Wows, and no actual science is really achieved.<br />
<br />
A smaller cadre of 'serious' amateur astronomers are out every night they can get, some with automated telescopes of surprising power and resolution. Some treat it like a professional photography shoot with less catwalk models and more heavenly bodies, and get quite a good income. But the vast amount of <i>that</i> data just sits on hard drives, not doing science either.<br />
<br />
<br />
<b>Aquisition</b><br />
<br />
For science to happen, you have to write all the numbers down. Eyes are terrible scientific instruments, but it also turns out JPEG or the h264 compression algorithms are equally bad, literally smoothing out the most important data points.<br />
<br />
It's why professional photographers currently make the best amateur astronomers, because they have access to acquisition devices (eg $4k Nikon cameras) which don't apply this consumer-grade degradation.<br />
<br />
When you look into the details, what stands out is that the same <i>hardware</i> is often involved, it's the signal processing chain that's different.<br />
<br />
Here is where we start having to consider our 'capabilities', in terms of how much CPU, memory, and bandwidth you have. If you point a high-resolution camera at the sky and just start recording raw, you will <i>very quickly</i> overwhelm your storage capacity, even if you have a RAID of terabyte drives.<br />
<br />
Trust me on this. Been there, still haven't got the disk space back.<br />
<br />
And the sad thing is, if like me you used "commodity" image capture hardware then the data is scientifically useless. Just pretty pictures.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-rUe_vt9jRB0/VlaAfqLXYNI/AAAAAAAACbk/sJGaDQzpw7I/s1600/vlcsnap-2013-06-06-11h25m40s84.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="256" src="http://1.bp.blogspot.com/-rUe_vt9jRB0/VlaAfqLXYNI/AAAAAAAACbk/sJGaDQzpw7I/s320/vlcsnap-2013-06-06-11h25m40s84.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Video imagery of the moon, taken through my telescope in '13.</td></tr>
</tbody></table>
You need 'raw' access to the pixel data, which is coming in a torrent. A flood. 4000x4000 images, one per few seconds, if you're using a DSLR camera and lucky imaging. Some people who look for meteors use 1024x720 video streams at high framerates. When you see a professional observatory, you're looking at a cool digital camera all right, but one that's literally sitting on it's own building-full of hard drives. That's a <i>big</i> memory card.<br />
<br />
<br />
<b>Signal Processing</b><br />
<br />
If you want to turn that raw video into useful data, you have to bring some fearsome digital signal processing to bear. Just to clean up the noise. Just to run 'quality checks'. Then there's the mission-specific code (the meteor or comet detector algorithms, if that's what you're doing) and the compression you'll need to turn the torrent into a manageable flow you can actually keep.<br />
<br />
Not just to store it to your hard drive. But also to "stream-cast" it to other observers. Video conferencing for telescopes.<br />
<br />
Why? Because when strange things happen in the sky, the first thing astronomers do is call each other up and ask "Are you seeing this?". Some of those events are over with in seconds, and some of the greatest mysteries in astronomy persist because, basically, we can't get our asses into gear to respond fast enough.<br />
<br />
Have we learned nothing from Twitter?<br />
<br />
We can't wait for the data to get schlepped back home, and processed a week later. We need automated telescopes that can get excited, and call in help, while we're over the far side of the paddock having tea with the farmer's daughter.<br />
<br />
<br />
<b>Ground Truth</b><br />
<br />
Up until now we've just been talking about slight improvements to the usual observer tasks. Stuff that's done already. Making the tools of Professional Astronomers more available to amateurs is nice (and as we've seen, that's really all in the DSP.) but what's the point?<br />
<br />
Here we could diverge into talking about an algorithm called "Ray Bundle Adjustment", or even "Wave Phase Optics" but Ray's a complicated guy, so I'll sum it up:<br />
<br />
If you want to reconstruct something in 3D, you need to take pictures of it from multiple angles. You probably guessed that already. There's a big chunk of math for how you combine all the images together, and reconstruct the original camera positions and errors. Those are the "bundles" that are "adjusted" until everything makes sense.<br />
<br />
The more independent views you have on something, the better. Even for 2D imagery. Even aberrations in the sensors become useful, so long as they're consistent. It can create 'super-resolution".<br />
<br />
Beyond that, there's "Light Field Cameras", which use a more thorough understanding of the nature of light to produce better images - specifically that traditional image sensors only record <i>half</i> the relevant information from the incoming photons.<br />
<br />
Most camera sensors record - for each square 'pixel' of light - how much light fell on the sensor (intensity) and it's colour (wavelength). What you don't get is the <i>direction</i> of the incident photon, (it's just assumed) or its <i>phase. </i><br />
<i><br /></i>
For a very long time we thought those other components weren't important, mostly because the human eye can't resolve that information. Insects can perceive these qualities, though. Bees can see polarization, and compound eyes are naturally good at encoding photon direction. We couldn't, so we didn't build our telescopes or optical theory with that in mind.<br />
<br />
Plus, the math is <i>hard</i>. You have to do the equivalent of 4D partial Fourier transforms. Who wants that?<br />
<br />
But when you work through it, you realize that you can consider every telescope pointed at the sky to be one element of a planet-wide wave-phase optics "compound eye" with the <i>existing hardware</i>. (and maybe a polarization filter or two)<br />
<br />
All we need to do (ha!) is connect together the computers of everyone currently pointing a telescope at the sky, and run a global wave-phase computation on all that data, in real-time. (I might be glossing over a few minor critical details - learn enough to prove me wrong.)<br />
<br />
This is not beyond the capability of our machines. Not anymore. The hardware is there. The software isn't. This is what I've been working on with Astromech. A <i>social</i> data acquisition system that assumes you're not doing this alone.<br />
<br />
What you get out of this is "Ground Truth", a term that mostly comes from the land-surveyors who are used to pointing fairly short-ranged flying cameras at a <i>very</i> nearby planet. But it's the same problem.<br />
<br />
This is the stage we can finally say we're "Mapping." Once we got enough good photos of the asteroid Ida, we constructed a topographical map. Once we got enough information on it's orbital mechanics, we could predict where it would be.<br />
<br />
Fundamentally, that allows us to prove our mastery of the maps by asking questions like "If I point a telescope at Ida right now, using these co-ordinates, what would I see?"<br />
<br />
ie: Can I see their house from here?<br />
<br />
<br />
<b>Simulation</b><br />
<br />
To really answer that question means you have a 3D-engine capable of rendering the object using all known information. If we assume Ida hadn't changed much in terms of surface features, then it's pretty easy to "redraw" the asteroid at the position and orientation that the orbital mechanics says.<br />
<br />
Then you just apply all the usual lighting equations, and you'll have a damn passable-looking asteroid on your screen.<br />
<br />
But it's not 'real' anymore. Not exactly. It's not an image that anyone has taken in reality. It's a simulation. A computer-enhanced hallucination. A flight of the imagination.<br />
<br />
Good simulations encode all the physically relevant parameters, and the main point of them is to provide a rigorous test of the phrase "That's funny..." ("How funny is it, exactly?")<br />
<br />
Because by now humans are pretty good at predicting the way rocks tumble. It's kind of our thing. When rocks suddenly act in a way other than predicted (than simulated) it indicates that we've got something wrong. Or something interesting is going on.<br />
<br />
And being wrong, or finding something interesting; that's Science!<br />
<br />
Simulations are also the only way that most of us are ever going to "travel" to these places. Thankfully our brains are wired such that we can hang up our suspenders of disbelief long enough to forget where we are. Imagination plays tricks on us. There are people right now (in VR headsets viewing Curiosity data) who've probably forgotten they're <i>not</i> on Mars.<br />
<br />
Used the right way, that's a gift beyond measure.<br />
<br />
<br />
<b>Sentinels</b><br />
<br />
About the most interesting events we can see is stars blinking on and off when they shouldn't be.<br />
<br />
Yes, this happens. A lot. Sometimes stars just explode.<br />
<br />
Then there's all kinds of 'dimming' events that have little to do with the star itself, just something else passing in front of it. We tend to find exoplanets via transits, for example. Black holes in free space create 'gravitational lenses' that distort the stars behind them like a funhouse mirror, and we might like to know where they are, exactly.<br />
<br />
Lets say we wanted to watch all the stars, in case they do something weird. That's a big job. How big?<br />
<br />
Hell, if we just assign the stars in <i>our</i> galaxy and we get every single person on the planet trained as an astronomer, then each person has to watch vigil over 20 stars. (assuming they could see them at some wavelength.) If we're assigning galaxies too, then everyone gets 10,000 of those.<br />
<br />
Please consider that a moment. If every human were assigned their share of known galaxies, you'd have 10,000 galaxies to watch over. How many do you think you'd get done in a night? How long 'till you checked the same one twice and noticed any upset?<br />
<br />
We're gonna need some help on this.<br />
<br />
And really, there's only one answer. To create little computer programs, based on all our data and simulations and task 200 billion threads to watch over the stars for us, and send a tweet when something funny happens.<br />
<br />
We can't even keep up with NetFlix, how the hell are we going to keep up with the constantly-running terra-pixel sky-show that is our universe?<br />
<br />
I've got a background in AI, but I'll skip the mechanics and go straight to the poetics; we will create a trillion digital dreamers - little AIs that live on starlight, on the information it brings, who are most happy when they can see their allocated dot, and spend all their time imagining what it <i>should </i>look like, and comparing that against the reality. Some dreamers expect the mundane, others look for the fantastic, and bit by bit, this ocean of correlated dreamers will create our great map of the universe.<br />
<br />
Every asteroid. Every comet. Every errant speck of light. Every solar prominence or close approach. We are on the verge of creating this map, and the sentinels who will watch over the stars for us, to keep it accurate.<br />
<br />
There's not a lot of choice.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com2tag:blogger.com,1999:blog-7271315367558300569.post-85002388014892745522015-11-17T21:47:00.002-08:002015-11-22T03:24:22.194-08:00Harr Harr!Here's today's development screenshot of Astromech: (from the virtual DSP desk.)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-hFWf7hI9dxE/VkwClEOzMnI/AAAAAAAACak/sGwZk52kerM/s1600/astromech%2Bharr-transform.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="312" src="http://3.bp.blogspot.com/-hFWf7hI9dxE/VkwClEOzMnI/AAAAAAAACak/sGwZk52kerM/s640/astromech%2Bharr-transform.png" width="640" /></a></div>
<br />
An image that will give your PNG decompressor conniptions, no doubt. The middle screen-full of leafy trees is a live webcam feed from out my window. The pink lines all across it are because it's a shitty webcam that cost $6 off ebay.<br />
<br />
The left-hand screen is a 256x256 real-time Fast-Fourier Transform of the webcam luminance. That's not big news, Astromech has always done that. Its first trick.<br />
<br />
The right-hand screen is the new thing for today. It's a 512x512 "H-Transform" which likely originally stands for "Two-Dimensional Harr Transform". I also call it the "Hubble Transform", because it's the basis of the compression format the Hubble data team invented in order to distribute 600Gb of <i>their</i> pretty pictures.<br />
<br />
The full text I'm following here is <a href="http://fits.gsfc.nasa.gov/registry/tilecompression/tilecompression2.3.pdf">Tiled Image Convention for Storing Compressed Images in FITS
Binary Tables</a> published by NASA.<br />
<br />
Don't let that NASA appellation fool you into thinking there's anything hard about the H-Transform. Compared to the FFT or Cosine Transform or Huffman coding it's very, very simple. And the best thing about the H-Transform is that it's parallelizable on WebGL. (as you can see.)<br />
<br />
That's how I'm doing this in real time, (about 12 frames per second I'd guess, limited by webcam speed) in my browser, and my CPU usage is 20%.<br />
<br />
Why the H-Transform? Why not just use something browser-supplied like h264, or V8 or a stream of JPEG/PNG images? (MJPEG!) which is built into most modern browsers? Well, in a nutshell, because nice as they are, they're not "scientific".<br />
<br />
There a really big difference between a compressor that optimizes for the human perceptual system, and a compressor that tries to preserve the scientific integrity of the source data. The H-Transform is the second type.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-fpbkECuvWBA/VkwUFsFvfdI/AAAAAAAACa0/S1QCmbceo-I/s1600/astromech%2Bharr-transform2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="369" src="http://3.bp.blogspot.com/-fpbkECuvWBA/VkwUFsFvfdI/AAAAAAAACa0/S1QCmbceo-I/s640/astromech%2Bharr-transform2.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Similar to a 'MIPMap', the H-Transform encodes a pyramid of lower-resolution (but higher entropy) <br />
versions of the source image into the lower-left corner, like a fractal. <br />
The larger 'residual' areas become easier to compress.</td></tr>
</tbody></table>
<br />
<br />
That's why NASA trusts data that has been stored in that format. It has certain very nice mathematical properties. It's a 'lossless' compressor, but one with a tuneable 'noise floor'. If that seems a contradiction, welcome to the magic of the quantized H-transform, where 60:1 compression ratios are possible.<br />
<br />
There's a couple of stages to go before that image on the right is turned into a FITS file, but the hard part is done and the rest is just shuffling bits around. Well, assuming the browser will let me save a stream of data to a file. That's really tricky, it seems.<br />
<br />
<b>Update: 22/Nov</b><br />
<br />
The whole thing provably works now, since I've also implemented the inverse H-transform. (there were a few bugs)<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-BLDRlUObhtI/VlGj-9rHVBI/AAAAAAAACbE/jFoFq3jCBCA/s1600/harr-anya-transform.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="288" src="http://4.bp.blogspot.com/-BLDRlUObhtI/VlGj-9rHVBI/AAAAAAAACbE/jFoFq3jCBCA/s640/harr-anya-transform.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The inverse of the transformed cat is also a cat. Well, you'd expect that, surely?</td></tr>
</tbody></table>
Basically in this example, the middle webcam image is encoded into the right-hand image (which looks fractal yet empty - that's the H- transform) and then <i>that </i>is run through the <i>Inverse </i>transform (a separate bit of code that does everything in a different order, <i>using</i> the big mostly-empty texture) to go in the left-hand window.<br />
<br />
It's almost too easy.<br />
<br />
And so the fact that the two left images look boringly identical is a good thing, given the (poor quality to begin with) data has been mangled twice in between by me. Cat sitting on warm computer staring at cursor. It's a common test case around here.<br />
<br />Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com2tag:blogger.com,1999:blog-7271315367558300569.post-75950930977289747612015-11-13T22:23:00.003-08:002015-11-13T23:18:44.725-08:00Fire Map 2015First of all, sorry to everyone who uses my Fire Map. I hadn't noticed that the NSW dots were no longer showing up. I just fixed that. Apologies.<br />
<div>
<br /></div>
<div>
<div>
Since I'm no longer connected with departments that produce internal reports on the coming fire season, I really don't have any idea what the forecast for this Christmas is. However, I've just noticed the fire-line across the Northern Territory:</div>
</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-SuI5-7adjzk/VkbN8QZbarI/AAAAAAAACZ4/fyVbvcnKy3g/s1600/nt%2Bfire%2Bline%2B2015.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="276" src="http://3.bp.blogspot.com/-SuI5-7adjzk/VkbN8QZbarI/AAAAAAAACZ4/fyVbvcnKy3g/s640/nt%2Bfire%2Bline%2B2015.png" width="640" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
That's a lot of red dots, big enough to be visible from space. Those are not good dots.</div>
<div>
<br /></div>
<div>
And from what I remember with my flawed brain, that staggered line is a trend-setter that continues south in waves over the next few months, as those heat conditions spread.</div>
<div>
<br /></div>
<div>
So, it's probably a good time to consider the history of the Fire Map.</div>
<div>
<br /></div>
<div>
<b>The Past</b></div>
<div>
When I started the map, each service ran their own state incident map (and still do) and I knew that there was no federal agency that had the remit to make a country-wide map. Google had a fairly good relationship with NSWRFS and had just added Victoria, but had no plans to add Queensland or beyond. There seemed only one thing to do...</div>
<div>
<br /></div>
<div>
<b>The Present</b></div>
<div>
Frankly, I've been neglecting the Fire Map for Astromech and day jobs. It just kind of sits there, working away, and really only needs occasional checks and updates when one of the services changes their servers in some way. Fortunately, you can always depend on the inertia of government departments, and I've gone for years without to much issue.</div>
<div>
<br /></div>
<div>
<b>The Future</b></div>
<div>
I'm not sure. I just noticed that the Northern Territory <i>finally</i> has their own incident map / feed, and the perfectionist in me always felt annoyed that there was one big empty chunk on the map. I may have to fix that, at which point the tapestry will be complete.</div>
<div>
<br /></div>
<div>
But long term, the map has no future. I nearly shut it down last year when the hotspots went away for a while, meaning there was no effective difference between my map and the Google Crisis Response map, which has come along enormously in the last few years. I was a momentary expert three years ago, but I really haven't kept up. </div>
<div>
<br /></div>
<div>
I shouldn't be doing this. The only reason I do is because there doesn't seem to be anyone else with the same focus, and the needed abilities.</div>
<div>
<br /></div>
<div>
I'd love to hand it on to some official organization that can form relationships with the state agencies that source the data, and thereby have more warning of server changes than "crap, the dots aren't working today." But that doesn't seem likely either. I once had grand ideas about doing bushfire prediction, (which I still think is very possible) but that was when I had access to data, and the people for whom those answers were relevant.</div>
<div>
<br /></div>
<div>
I'd be interested in advice for what I should do in the future. Is the map still needed?<br />
<br />
<br />
<b>Update: NTFRS data online</b><br />
<b><br /></b>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-eoqS0NYPKao/VkbfXmqmwjI/AAAAAAAACaI/JeqK8OIOCy0/s1600/fire%2Bmap%2B2015%2Badded%2Bntfrs.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="243" src="http://1.bp.blogspot.com/-eoqS0NYPKao/VkbfXmqmwjI/AAAAAAAACaI/JeqK8OIOCy0/s640/fire%2Bmap%2B2015%2Badded%2Bntfrs.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">It would have to be Humpty Doo.</td></tr>
</tbody></table>
Adding the Northern Territory to the fire map took almost 1/3 of a packet of Anzac biscuits to accomplish (and two years of waiting). That means the map is complete. Every state is on there, making it, truly, the "Australian Fire Map" at last.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-Z8DObU97Vu8/Vkbf0KH2Q0I/AAAAAAAACaM/cL_EPud2DFI/s1600/fire%2Bmap%2B2015%2Bfull.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="374" src="http://2.bp.blogspot.com/-Z8DObU97Vu8/Vkbf0KH2Q0I/AAAAAAAACaM/cL_EPud2DFI/s640/fire%2Bmap%2B2015%2Bfull.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">My work here is done.</td></tr>
</tbody></table>
<br />
<br /></div>
Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com0tag:blogger.com,1999:blog-7271315367558300569.post-91343706543160788802015-11-13T20:47:00.000-08:002015-11-13T21:30:05.289-08:00Why Video Games were Not ArtI always thought that computer games were a form of Art. Today, I learned why I was wrong.<br />
<br />
First, let's start with a definition of "Art" that is mostly "That which is shown in museums and galleries and whatnot." Statues are shown in museums and galleries, therefore statues are art. Paintings, sculpture, movies, songs. Note these are all <i>forms</i>, rather than specific things.<br />
<br />
So, why aren't computer games Art? Because, in a nutshell, Museums couldn't exhibit them, without fear of prosecution from the rabid armies of copyright lawyers engaged by the industry to protect their products.<br />
<br />
That's it. QED.<br />
<br />
It didn't matter how many soulful designers cried that they were doing more than just extracting quarters by addicting teenagers to blinky lights; they were being shivved in the back by their own legal departments, who enjoyed wielding the power of the precious DMCA.<br />
<br />
If video games were Art, or were recognized to have cultural significance, then they would have status beyond that of mere "product" and society at large would have a different relationship, with more rights of re-use. Can't have that.<br />
<br />
I didn't realize a lot of the internals, before I read the excellent article:<br />
<a href="http://gamasutra.com/view/news/259037/Understanding_this_years_biggest_video_game_copyright_ruling.php">Understanding this years biggest video game copyright ruling</a> at Gamasutra.<br />
<br />
The good news, since the US Copyright Court (ugh, from the "why is this even a thing" category) has now said that Museums can show off old games without fear, there might one day be an exhibit of 'classic games' at your local museum, as perhaps should have been possible all along?!<br />
<br />
Sometimes you only notice how bad the stupid got when it takes a step back. And you think "good start, and another please?"Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com0tag:blogger.com,1999:blog-7271315367558300569.post-62368694979465538462015-11-08T20:40:00.000-08:002015-11-08T21:53:23.679-08:00Astromech - The Road to Beta2Going quiet means I've been getting things done on Astromech. I have a set of specific features I want in the next beta, and in the last week nearly all of them have reached a semi-stable point.<br />
<br />
Probably the best single demo of this is my little homage to the Caffeine molecule:<br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/zwQLJwbQedc/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/zwQLJwbQedc?feature=player_embedded" width="320"></iframe></div>
<br />
<br />
But first, the setbacks. The big one is; I've had to seriously question my use of Google Drive.<br />
<br />
Anyone who saw the original video noticed how heavily I relied on Google Drive for the task of storing the bulk assets in each 'level' as well as building a collaborative editor (using the Docs realtime API) to set up the scene / load script.<br />
<br />
Here's the stage I got to with that, before some rather bad news broke:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-hip81OHXALw/Vj_9nPffd-I/AAAAAAAACZk/OCjnF44oLxs/s1600/Astromech%2BEditor%2B20151110.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="268" src="http://4.bp.blogspot.com/-hip81OHXALw/Vj_9nPffd-I/AAAAAAAACZk/OCjnF44oLxs/s320/Astromech%2BEditor%2B20151110.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A mobile-friendly way to edit assets and DSP 'circuits', backed by the Google real-time collaborative API. <br />
Shame it will probably never see the light of day.</td></tr>
</tbody></table>
And that's just part of it. There's also the Panel designer - a hierarchical 'box layout' editor for all those cool LCARS-like consoles that litter the Astromech levels.<br />
<br />
But unfortunately, Google has announced that they will disable file hosting from Google Drive shortly. I ranted a little about that in my previous post.<br />
<br />
That has two very specific impacts on the Astromech 'GUI' editor. It means the files that it creates can't be read anonymously anymore. So. any Astromech levels based on a script stored in Google Docs will not be accessible to everyone. That's bad enough on it's own to kill that part of the project dead.<br />
<br />
What's the point of collaboratively editing a "public world" file if the world's public can't read it?<br />
<br />
And where do you put all the resources files it references? On another service? Then what's the point of using all the Google Docs API's if the 'real' data is elsewhere?<br />
<br />
Shoving everything into one directory made things nice and manageable, using relative links. Once you've got your resources scattered across half the internet using absolute URI's, it makes things so much harder.<br />
<br />
It's not just that I'd have to add a "Save As..." button to the Google Drive app, I have to re-think the entire premise of how users collaboratively store and work with terabytes of data. Instead of a central dumb (but reliable) fileserver and peer-to-peer clients, I'll probably need a peer-to-peer _server_ layer as well. ie: I need to replace Google by Christmas.<br />
<br />
The old levels still work for now, but the access they depend on is deprecated and goes away next year. But it's the wasted effort in that direction that really hurts. Hopefully I can salvage most of the UI and editors, while backing them with a different datastore.<br />
<br />
Then, there was the whole getUserMedia http:// deprecation thing I had to deal with. Within months, "powerful browser features" (which is basically <i>everything</i> I use.) will not work from http:// servers. Only http<b>s</b>://.<br />
<br />
This really broke me for a couple of days (I even got into a discussion with the security@chromium.org list) because it implied that running Astromech would only be possible from the Internet by paying money to Versign-derivatives. Not, for example, on your own goddamn own computer.<br />
<br />
I settled down a bit when it was pointed out that localhost: is supposed to be considered "secure", (even if it only uses http:// without the SSL) So you <i>will</i> still be able to download and run Astromech on your own machine and use all the features. You can imagine how the alternative would have been maddening.<br />
<br />
However, this still leaves the <i>localnet</i> in limbo. It's no longer clear how you'd run the software on your own desktop and access it, for example, from your own iPad over your own WiFi network. (Why, it's just as simple as creating your own SSL certificate authority for signing local machines, and then installing the certificates on each device, of course... why are you complaining?)<br />
<br />
It was always in the back of my mind that a piece of Astronomy software that only really worked indoors while connected to high-speed internet might not be as useful as it could be. (instead of, say, in a dark paddock filled with amateur astronomers bristling with advanced imaging equipment and local bandwidth but poor global internet connectivity.)<br />
<br />
I really don't want Astromech to be a "local webserver" install, for every individual user/machine. It should be more like running a minecraft server. If you need to install local servers everywhere to get a browser app to work, then what's the point of doing it in a browser? Why not just write a full application?<br />
<br />
And besides, it seems really counter-intuitive that the only way to work with/around the 'increased browser security' is to start installing local code (eg, a node.js micro-server) with full binary access to the machine. That's just security whack-a-mole. If the machine gets boned through an exploit in my code, then it's not <i>their </i>fault for leading me down that path, obviously.<br />
<br />
But the browser makers are determined to deprecate http:// and that's that. It doesn't matter that https:// is flawed, costly, inefficient, and creates barriers to entry.<br />
<br />
OK, so now the good stuff that's made it into Astromech in the last few weeks:<br />
<br />
<br />
<b>iOS + Edge support</b><br />
Astromech now 'works' on iOS9, to the extent it will load and render the scene using WebGL. What it doesn't do very well (or at all) is replace the keyboard/mouse control scheme with something that functions equivalently using touch. I'm probably going the little "thumbsticks in the screen corners" route there, as soon as I get the time.<br />
<br />
Microsoft Edge is running Astromech fairly well, better than IE did, but it also has some feature gaps (like getUserMedia / WebRTC) that effectively disable some of the more advanced features.<br />
<br />
Chrome and Mozilla are still the 'preferred' browser, but all roads are slowly leading to HTML5 compatibility across all devices.<br />
<br />
<br />
<b>Improved Blender/WebGL shaders</b><br />
The first-gen model loader did well with geometry, but badly with surfaces. For a start, only the first texture worked, and there was no real lighting model. So, scenes looked very different in Astromech from how they originally looked in Blender, even if the geometry was correct.<br />
<br />
The 'shader compiler' I wrote has been extended with a full multi-source specular lighting model, with 'sun' and 'point' lights. Technically it does a Lambert/Blinn-Phong pass with fixed lights.<br />
<br />
So, now you can export a fairly generic existing Blender model (instead of carefully building one specifically for Astromech) and it will mostly work as you expected. Common surface materials work. Multiple scene textures work. Bump maps sort-of (they have the common view-independence problem because of the lack of tangent vectors in the collada export, so the bumps always point 'up' instead of 'out', though there's probably a way to solve that. Good for floors though.)<br />
<br />
I still don't have a great solution for transparent surfaces, but then, neither does anyone else.<br />
<br />
<br />
<b>Multiple Scene Models</b><br />
Version 1 only properly loaded a single 3D model as the 'primary scene'. That's been fixed, so you can load an arbitrary number of collada files, and instance them multiple times within the scene at multiple locations.<br />
<br />
eg: In the "Atomic Caffeine" demo, each of the four atoms was modelled/coloured in Blender, and then instanced into the scene as many times as the entire molecule needed.<br />
<br />
New features sometimes magnify minor old problems; in this case the lack of a global lighting model. Since each 'scene' model carries its own lights in its own reference frame, obvious visual inconsistencies occur when you put several models together and rotate some of them. (Although, less jarring than I'd have thought.)<br />
<br />
Fully dynamic lighting is a major overhead with diminishing returns. So, I'll probably go for a compromise, with only a few global dynamic lights.<br />
<br />
<br />
<b>Cannon.js Physics Engine</b><br />
The other side of the multiple-model system is the ability to define a 'physics proxy' (usually a box or sphere with properties of mass and friction) to which the position of the 3D models is attached.<br />
<br />
I've chosen the <a href="http://www.cannonjs.org/">cannon.js</a> physics system to do the heavy lifting. It can connect the proxy objects together with 'hinges', 'springs', and other physical constraints like gravity, and then model the physics over time and update the objects.<br />
<br />
It's extremely efficient (the solver it uses is <i>very</i> advanced) although there are severe practical limits to just how much you can do in real-time. But a little physics is a great way to add some life to an otherwise static scene, and give the user the sense that they're <i>there</i>, and bumping into things.<br />
<br />
<br />
<b>Scriptable UI</b><br />
I've just about finished exposing all the things that Astromech can do as scriptable elements - as opposed to my early examples that used lots of hard-coded javascipt.<br />
<br />
It's slightly less flexible than the raw javascript - at least until I create a 'module' system capable of safely loading arbitrary code. It's still just a set of pre-approved LEGO blocks you can arrange in various ways, but at least the set of blocks is getting bigger.<br />
<br />
Scripts don't all have to run on load. The script can define UI "command buttons" which run parts of the script later... which might load more resources and create new buttons. A common use of command buttons is to provide "teleport" options which can jump you around the map.<br />
<br />
In practice, you can already build a 'conversation tree' system which offers choices dependent on previous choices. (All the buttons would be pre-defined, just shown and hidden using commands activated by other buttons.)<br />
<br />
<br />
<b>Social 'presence' and messaging.</b><br />
The chat system has been functioning for a couple of versions now, based on a websocket 'pub-sub' server that I'm running on an OpenShift cartridge. (Thanks, RedHat!) I've gone through a couple of revisions of this system, and it's been stable and reliable for months.<br />
<br />
Previously, you'd get a 'chat message' when someone connected to the channel ("hailing frequencies open") but in the background the networking code always had the full list of the other participants this entire time, you just couldn't see them. Now, the right-hand of the screen is one long column of everyone else in the level with you.<br />
<br />
This makes everything feel a lot more MMORPG, and future extensions will be things like "friends lists" and private instances that build on this social side, since there's going to be obvious problems if a 'level' gets too popular.<br />
<br />
<br />
<b>File Transfer & Videoconferencing</b><br />
The first features the social list made possible was inter-user private chat (easy) followed by file transfer (not so easy, but mostly working) and video conferencing. (just got the prototype working)<br />
<br />
It's a core idea of astromech that you should be able to exchange data with other people. This is an essential part of that plan.<br />
<br />
The file transfer I'm particularly proud of. To 'transmit', the sender just drags a file out of the File Manager and drops it on the button for the intended recipient. An 'offer' is then sent to the receiver, and turns up in their corresponding list of options for the sender.<br />
<br />
If the receiver clicks on this offer button, the file is downloaded to the browser's "Temporary FileSystem" (you get a little progress message while the transfer is in progress) and then the recipient can either click on the button a second time which open the (now local) file in the browser, or they can drag the file icon back out of the browser to the filesystem again.<br />
<br />
In summary, one user drags the file into their browser. The other user accepts the offer, and then can drag the file back out of <i>their </i>browser. (Well, Chrome) I don't think I can make it simpler.<br />
<br />
Remember, this is peer-to-peer. Although right now all the comms goes through the chat relay, (as private messages) but I have the RTC channels working, so I intend to make that the preferred transport to make it <i>truly</i> peer-to-peer, and reserve the relay server as the 'fallback'.<br />
<br />
<br />
<b>Voice Recognition</b><br />
This was nearly a 'freebie', in that I went - in one morning - from not knowing that browsers offer a full voice-recognition engine javascript API, to having it working by lunchtime.<br />
<br />
Any 'command button' can be given an array of "speech" strings that, if heard by the engine, activates that command button. It's that easy.<br />
<br />
It's good to have an optional prefix word that wakes up the engine but can be missed, because that happens a lot. Originally I used "computer" (duh) but soon changed it to "Scottie!" after shouting at my machine for a little while to transport me to new locations and switch on parts of the engine. Feels much more natural, somehow.<br />
<br />
<br />
<br />
I could go on for pages about all the things I want to do next, the improvement and changes, but I think we'll just stick with what I've actually done so far. <br />
<br />
The major features are now mostly in. There's a ton of clean-up work and major bugfixes that need doing before release, but no more super-bleeding-edge experimental features. I did the hard stuff first.<br />
<br />
A 'Beta2' release isn't far off now. I'm trying to be quick, before the ground shifts under my feet again. It's not easy doing all this single-handed, but I'll get there.<br />
<br />
<br />
<br />Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com0tag:blogger.com,1999:blog-7271315367558300569.post-37979998593606630162015-10-26T17:47:00.000-07:002015-10-26T18:13:53.652-07:00Astromech Updates - Speculars & PhysicsTime for some more pretty pictures, screenshotted just now from Astromech in the other window:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-t25Hvw6jiNE/Vi7BsWCL2kI/AAAAAAAACZE/5ghmh4oh640/s1600/SS%2BImagination%2Band%2BCaffeine.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="345" src="http://3.bp.blogspot.com/-t25Hvw6jiNE/Vi7BsWCL2kI/AAAAAAAACZE/5ghmh4oh640/s640/SS%2BImagination%2Band%2BCaffeine.png" width="640" /></a></td></tr>
<tr><td class="tr-caption">In Soviet Russia, Caffeine goes inside of you!</td></tr>
</tbody></table>
Recognize it? It's my second favorite molecule, the one that's powered much of Astromech's development so far. The still shot doesn't entirely do it justice, so I'll have to make a video shortly. Watching the molecule "fold up" from its usual flat schematic is a small revelation.<br />
<br />
And yes, that's the "Starship Imagination" lurking in the background. You'll be seeing a lot of it. (You gotta have somewhere comfy to sit, when contemplating the universe.)<br />
<br />
The colours? Blue is nitrogen, red is oxygen, grey/white is carbon. Most of what you're looking at are the 'P' orbitals overlapping. Oxygen has two coloured 'lobes' available for bonding, nitrogen has Three, Carbon of course has the full set of four. Things are scaled so a hydrogen atom's 1S orbital would be about a meter across, so the entire molecule is about the 'size' of a small building.<br />
<br />
This is not your normal ball-and-stick molecular model, and yes it's harder to see the core structure as a result... but nature isn't as neat and tidy as our schematics would prefer. This is my own small attempt to better show the reality of what a molecule "looks like", if you could shrink yourself to its scale. Ghost-like waves of probability dominate, not little billiard balls.<br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-qPotF3Nd6tY/Vi7C4yVCDjI/AAAAAAAACZM/A4NnXFGdSqk/s1600/SS%2BImagination%2Band%2BCaffeine%2B2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="298" src="http://4.bp.blogspot.com/-qPotF3Nd6tY/Vi7C4yVCDjI/AAAAAAAACZM/A4NnXFGdSqk/s640/SS%2BImagination%2Band%2BCaffeine%2B2.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Sensors are reading Wake-up-juice, Captain! We're saved!</td></tr>
</tbody></table>
There's been several advances in the code required to make all this happen:<br />
<br />
- Scripting System<br />
- Multiple scene models<br />
- Physics Engine<br />
- Specular Shaders (for that 'shiny' look)<br />
<br />
In particular, the molecule is a huge "hinge and spring" system built with the <a href="http://www.cannonjs.org/">cannon.js</a> physics engine, with imported Blender 3D assets for each atom.<br />
<br />
In fact, since I like to show the code, here's the part of the script that assembles the molecule:<br />
<br />
<span style="font-size: xx-small;"><br /></span>
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"hydrogen", "url":"asset/collada/atomic/hydrogen.dae", "transform":"scale(0.75)" }},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"oxygen", "url":"asset/collada/atomic/oxygen.dae", "transform":"scale(0.75)" }},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"nitrogen", "url":"asset/collada/atomic/nitrogen.dae", "transform":"scale(0.75)" }},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"carbon", "url":"asset/collada/atomic/carbon.dae", "transform":"scale(0.75)" }},</span><br />
<span class="Apple-tab-span" style="white-space: pre;"><span style="font-size: xx-small;"> </span></span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"carbon", "model":"translate( 0 -1 -20)" }}, {"physics":{ "model":"carbon.1", "proxy":{ "type":"sphere", "radius":2 } }},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"carbon", "model":"translate( 0 9 -20)" }}, {"physics":{ "model":"carbon.2", "proxy":{ "type":"sphere", "radius":2 } }},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"carbon", "model":"translate( 16 4 -20)" }}, {"physics":{ "model":"carbon.3", "proxy":{ "type":"sphere", "radius":2 } }},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"nitrogen", "model":"translate( 8 12 -20)" }}, {"physics":{ "model":"nitrogen.1", "proxy":{ "type":"sphere", "radius":2 } }},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"nitrogen", "model":"translate( 8 -4 -20)" }}, {"physics":{ "model":"nitrogen.2", "proxy":{ "type":"sphere", "radius":2 } }},</span><br />
<span style="font-size: xx-small;"><br /></span>
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"carbon", "model":"translate( -8 12 -20)" }}, {"physics":{ "model":"carbon.4", "proxy":{ "type":"sphere", "radius":2 } }},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"carbon", "model":"translate(-16 0 -20)" }}, {"physics":{ "model":"carbon.5", "proxy":{ "type":"sphere", "radius":2 } }},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"nitrogen", "model":"translate( -8 -6 -20)" }}, {"physics":{ "model":"nitrogen.3", "proxy":{ "type":"sphere", "radius":2 } }},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"nitrogen", "model":"translate(-16 8 -20)" }}, {"physics":{ "model":"nitrogen.4", "proxy":{ "type":"sphere", "radius":2 } }},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"oxygen", "model":"translate( -8 20 -20)" }}, {"physics":{ "model":"oxygen.1", "proxy":{ "type":"sphere", "radius":2 } }},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"oxygen", "model":"translate(-20 -6 -20)" }}, {"physics":{ "model":"oxygen.2", "proxy":{ "type":"sphere", "radius":2 } }},</span><br />
<span style="font-size: xx-small;"><br /></span>
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"carbon", "model":"translate(-8 -12 -20)" }}, {"physics":{ "model":"carbon.6", "proxy":{ "type":"sphere", "radius":2 } }},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"carbon", "model":"translate(-20 12 -20)" }}, {"physics":{ "model":"carbon.7", "proxy":{ "type":"sphere", "radius":2 } }},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"collada":{ "id":"carbon", "model":"translate( 16 18 -20)" }}, {"physics":{ "model":"carbon.8", "proxy":{ "type":"sphere", "radius":2 } }},</span><br />
<span class="Apple-tab-span" style="white-space: pre;"><span style="font-size: xx-small;"> </span></span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="font-size: xx-small;">{"physics":{ "constraint":[</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.1", "point":[0,4,0]}, "to":{"model":"carbon.2", "point":[0,0,4]}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.1", "point":[0,0,4]}, "to":{"model":"carbon.2", "point":[0,4,0]}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.1", "point":[4,0,0]}, "to":{"model":"nitrogen.2", "point":[0,0,4]}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.1", "point":[-4,0,0]}, "to":{"model":"nitrogen.3", "point":[4,0,0]}},</span><br />
<span class="Apple-tab-span" style="white-space: pre;"><span style="font-size: xx-small;"> </span></span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.2", "point":[0,4,0]}, "to":{"model":"nitrogen.1", "point":[0,0,4]}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.2", "point":[-4,0,0]}, "to":{"model":"carbon.4", "point":[4,0,0]}},</span><br />
<span class="Apple-tab-span" style="white-space: pre;"><span style="font-size: xx-small;"> </span></span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.3", "point":[0,4,0]}, "to":{"model":"nitrogen.1", "point":[4,0,0]}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.3", "point":[0,0,4]}, "to":{"model":"nitrogen.2", "point":[0,4,0]}},</span><br />
<span class="Apple-tab-span" style="white-space: pre;"><span style="font-size: xx-small;"> </span></span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.4", "point":[0,4,0]}, "to":{"model":"oxygen.1", "point":[0,4,0]}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.4", "point":[0,0,4]}, "to":{"model":"oxygen.1", "point":[4,0,0]}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.4", "point":[-4,0,0]}, "to":{"model":"nitrogen.4", "point":[4,0,0]}},</span><br />
<span style="font-size: xx-small;"><br /></span>
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.5", "point":[0,0,4]}, "to":{"model":"nitrogen.3", "point":[0,4,0]}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.5", "point":[0,4,0]}, "to":{"model":"nitrogen.4", "point":[0,0,4]}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.5", "point":[-4,0,0]}, "to":{"model":"oxygen.2", "point":[4,0,0]}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.5", "point":[4,0,0]}, "to":{"model":"oxygen.2", "point":[0,4,0]}},</span><br />
<span style="font-size: xx-small;"><br /></span>
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.6", "point":[0,4,0]}, "to":{"model":"nitrogen.3", "point":[0,4,0]}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.7", "point":[0,0,4]}, "to":{"model":"nitrogen.4", "point":[0,0,4]}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"point", "force":0.1, "from":{"model":"carbon.8", "point":[-4,0,0]}, "to":{"model":"nitrogen.1", "point":[0,4,0]}},</span><br />
<span style="font-size: xx-small;"><br /></span>
<span class="Apple-tab-span" style="font-size: xx-small; white-space: pre;"> </span><span style="font-size: xx-small;">{"type":"spring", "length":40, "force":0.001, "from":{"model":"carbon.1"}, "to":{"model":"carbon.3"}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"spring", "length":40, "force":0.0001, "from":{"model":"carbon.1"}, "to":{"model":"carbon.4"}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"spring", "length":40, "force":0.0001, "from":{"model":"carbon.1"}, "to":{"model":"carbon.5"}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"spring", "length":40, "force":0.0001, "from":{"model":"carbon.1"}, "to":{"model":"carbon.6"}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"spring", "length":40, "force":0.0001, "from":{"model":"carbon.1"}, "to":{"model":"carbon.7"}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"spring", "length":40, "force":0.0001, "from":{"model":"carbon.1"}, "to":{"model":"nitrogen.1"}},</span><br />
<span class="Apple-tab-span" style="font-size: xx-small; white-space: pre;"> </span><span style="font-size: xx-small;">{"type":"spring", "length":40, "force":0.0001, "from":{"model":"carbon.1"}, "to":{"model":"nitrogen.4"}},</span><br />
<span class="Apple-tab-span" style="font-size: xx-small; white-space: pre;"> </span><span style="font-size: xx-small;">{"type":"spring", "length":30, "force":0.001, "from":{"model":"carbon.6"}, "to":{"model":"oxygen.1"}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"spring", "length":40, "force":0.001, "from":{"model":"carbon.6"}, "to":{"model":"carbon.3"}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"spring", "length":20, "force":0.001, "from":{"model":"carbon.7"}, "to":{"model":"oxygen.1"}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"spring", "length":20, "force":0.001, "from":{"model":"carbon.7"}, "to":{"model":"oxygen.2"}},</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{"type":"spring", "length":40, "force":0.001, "from":{"model":"carbon.8"}, "to":{"model":"oxygen.2"}}</span><br />
<span style="font-size: xx-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>]}},</span><br />
<br />
A little chunky, but that's it. Really. And yes, this is a rather fake system using springs to 'prop open' the molecule rather than simulating all the interatomic repulsive forces, but hey, you need to have room to improve.Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com0tag:blogger.com,1999:blog-7271315367558300569.post-12045958735453528852015-10-15T17:33:00.001-07:002015-10-15T17:33:59.871-07:00Google Drive Hosting? Not anymore.So, this has been a kick in the teeth...<br /><br /><a href="http://googleappsdeveloper.blogspot.com.au/2015/08/deprecating-web-hosting-support-in.html">http://googleappsdeveloper.blogspot.com.au/2015/08/deprecating-web-hosting-support-in.html</a><br /><br />"Beginning August 31st, 2015, web hosting in Google Drive for <a href="https://support.google.com/drive/answer/2881970?hl=en">users</a> and <a href="https://developers.google.com/drive/web/publish-site">developers</a> will be deprecated. You can continue to use this feature for a period of one year until August 31st, 2016, when we will discontinue serving content via googledrive.com/host/[doc id].<br /><br />In the time since we launched web hosting in Drive, a wide variety of public web content hosting services have emerged. After careful consideration, we have decided to discontinue this feature and focus on our core user experience."<br /><br /><br />This is enormously bad. Not just for me, for everyone. I don't know how to communicate what the knock-on effects could be, but this is probably the end of "free" file hosting on the internet. It's certainly a major problem for my Astromech project, which currently uses Google Drive for users to store and publish their 3D assets.<div>
<br /></div>
<div>
If you're happy with an entirely corporate internet landscape, where every byte is bought and paid for by someone and the prerequisite for getting published is a credit card, then welcome to the new world.</div>
<div>
<br /></div>
<div>
But to me, that's not what the internet was for. The people who built the 'net were scientists and students, educators and children, a segment of the population which pretty much by definition doesn't have any money. Just time, and passion, and knowledge.<br /><div>
<br /></div>
<div>
Now if you think "hang on, there's still lots of free file hosting services", then remember that none of them want you serving files out the 'back end' to all an sundry, like a website does. Filenames are obfuscated. Relative paths are broken. Logins are required. On Amazon, a whole eight machines can access the data! Eight!</div>
<div>
<br /></div>
<div>
Sure there's Mega.com and MediaFire, but they want the chance to put splash advertising in front of the download. That doesn't work for AJAX includes.</div>
<div>
<br /></div>
<div>
The keyword here is "direct links". That's what lets you copy a directory of HTML and images to a server and all the hyperlinks work as intended. So /doc.html can refer to /image.png and the server knows to get it from the same place just next-door. If you copy a directory to DropBox, for example, each file gets 'stored' in what seems like a different randomly named directory, (although the 'base' filename is there) And don't even _try_ pulling down the public directory list and extracting the file paths. They're heavily obfuscated.</div>
<div>
<br /></div>
<div>
What really concerns me is that once Google kick everyone off their free hosting, the 'parasites' will move on to DropBox or Amazon or OneDrive and work out how to gank those systems. Then they will fall like dominoes playing a game of hot potato. And the abusers won't stop, because they have a vested interest. Basically, all the porn will just shift to the next service, and the next. Or into hacked accounts, since they're now valuable for their storage space. (And the problem will get twice as hard to fix. It's easy to blow away sock-puppet accounts, not so easy when they're hijacking real ones.)</div>
<div>
<br /></div>
<div>
And you can bet that other services, like blogger, will have to clamp down on their file uploads too. Or go away entirely. Otherwise it's just Google playing whack-a-mole with itself.</div>
<div>
<br /></div>
<div>
What will vanish is all those little one-person technical sites that served a useful purpose, but had no revenue streams. $200 a year is a lot for a vanity site. They will all just quietly turn off August 16 next year, and it will be like GeoCities shutting down, all over again. Vast chunks of unique content will disappear. Individuals have no place anymore. The porn and cat videos and corporations will remain. Sort of like a neutron bomb for the internet.<br /><br />So, thanks Google. You've finally gone off the rails from your core promise of "We want more and better internet for everyone!" and added the caveat "So long as you can pay." And you'll have to go back to downloading exabytes of data that you used to store 'locally'. Hope that works out for you.</div>
</div>
<div>
<br /></div>
<div>
I'm sorry, but monetizing our friendship means you ain't my friend anymore. You pretended, offered me some nice tools, and used my hard work as the bait for the corporate switch, And you gave a one-paragraph transparently fictitious reason for doing so. That's not something I forgive. If you're going to carelessly destroy months of work, then I - at least - deserve the real reason.</div>
<div>
<br /></div>
<div>
"After careful consideration, we have decided..." Fuck you, Google.</div>
Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com0tag:blogger.com,1999:blog-7271315367558300569.post-79779830340104107792015-10-12T22:48:00.001-07:002015-10-12T22:48:55.013-07:00Astromech Beta2 - coming soonWith all projects, you reach that point where it's all downhill from here. In the Sisyphean sense that you rolled that bloody rock all the way up the bloody hill in the expectation that it would, eventually, get easier all of a sudden.<br />
<br />
...you're always looking for that point where you can give it a good shove, and the momentum builds on it's own, enough to maybe get you half-way over that next hill so long as you run fast enough to keep up. That next hill which you only just now can actually see, but really, you always knew was going to be there.<br />
<br />
Well, Astromech is reaching that stage. Several highly-experimental chunks of code are coming together nicely. The first release wasn't much more than a poorly implemented asset manager, allowing a few disconnected media files to be dumped into a 3D environment from common editors.<br />
<br />
None of that connected particularly well with the inner workings of Astromech, which has modules that do everything from Delaunay triangulation to Keplerian orbital mechanics to real-time digital signal processing of data coming from telescope sensors.<br />
<br />
These are all hidden away in javascript modules that aren't finished, let alone documented. There's a scripting interface I haven't had time to work on that's supposed to connect to the inter-client comms system that's only used for chat, right now.<br />
<br />
But everything is working, in it's way. All the prototype problems are solved. The Google Realtime API was giving me some grief until I rewired my head and figured out how it's meant to be used. (quick version: DO NOT store tree-like structures using the API. Flatten them into unordered tabular lists, and keep lots of lists. Otherwise, the multi-user undo/redo system will give you a bad time.)<br />
<br />
What's coming soon is the new Astromech "editor" app, which is a near-complete rework of the old list of lists system. The new set of available objects is too large to maintain this approach, and so new ideas are needed. (Or perhaps, some old ideas need revisiting. Duh duh daaa!)<br />
<br />
Also, the GUI-based editors for the DSP chain and Panel systems are done at last. Getting these two vast wodges available in the pointy-clicky interface has been a long road, but they are absolutely core to the idea of what Astromech is... not just a pretty face, but a tool for accomplishing some nifty science.<br />
<br />
The DSP editor has the job of allowing the user to connect together processing blocks like 2D Fourier transforms, convolutions, etc. similar to the "node editor" in Blender, which creates complicated material and animation effects by mixing together an "algebra" of core operators.<br />
<br />
Blender struggles to implement this functionality in a simple way, and it's got full control over the user's screen, mouse, and keyboard. My editor is usable on an iPad.<br />
<br />
So is the Panel Layout editor, which creates hierarchical box layouts for UI and text displays that are applied to surfaces in the scene. Panels are a kind of sprite-driven micro-language for information displays. They made some early appearances in this blog, but have been missing from the recent demos, until I could make them available for everyone. In essence, chunks of JSON text are rendered to 'compressed' textures, which make use of "superpixels" (or "sprites" in the old days) that come from a common font/symbol swatch used by every panel in the system.<br />
<br />
That means the first display console takes a fair chunk of GPU texture memory for the font swatch, but after that each (high resolution) text panel uses as much extra memory as an old-school character-based Teletext display. There's a massive global saving in texture memory, while still getting crisp-edged text on every surface.<br />
<br />
There's more than a few cutting edges still sticking out, though, so I have to make another smoothing pass or two, a process of simply grinding away until - well, not until it's perfect - but at least has a chance to solve more problems than it would cause, To be worth the time cost of learning how to work it.<br />
<br />
One day, I hope to completely erase the line between "editor" and "scene", and have all these tools available while inside the 3D environment. That's a few hills over, though.<br />
<br />
<br />Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com0tag:blogger.com,1999:blog-7271315367558300569.post-41562483905567331772015-09-07T19:27:00.001-07:002015-09-07T19:27:24.889-07:00Apache Spark OverviewLet me tell you a secret.<br />
<br />
Well, it shouldn't be a secret, but given how few people seem to know about it, we may as well be talking about Higgs-Englert physics.<br />
<br />
It's <a href="http://spark.apache.org/">Apache Spark</a>.<br />
<br />
"WTF is Spark?" you will probably say. You'll have heard about Apache, the venerable webserver software. You might not have really being paying much attention to their background projects which have been merging together over the last year like the arms and legs of some kind of Voltron. ("And I'll form the Head!")<br />
<br />
Apache is making the transition from being the kind of software you run your old website on, to being the kind of software you run Twitter, or Facebook, or eBay on. Or Netflix, which is possibly the best case study for the software we'll be talking about, although that's more <a href="http://cassandra.apache.org/">Apache Cassandra</a>, a topic for another time.<br />
<br />
At that level, there are new problems that are oddly different from the old ones. All these guys use cloud computing resources.. they don't really depend on physical machines. They rent them 'out of the air' for as many hours (or minutes) as they need. This is so they can increase the size of their clusters from say 20-50 to a few hundred for a couple of hours in order to handle peak loads.<br />
<br />
eg: They don't repair servers. For most of those machines, no human will ever ssh into the box. In fact, they are usually put on a countdown for a rolling 'refresh' which shuts down the most ancient servers and replaces them with fresh ones, in the equivalent of a slow A/B testing transfer. <b>Really</b> clever systems stop the rollout of the new software and clone copies of the old, as automated reliability statistics come in.<br />
<br />
But if a box is giving you trouble, you don't spend any of your time on it whatsoever. You mercilessly sent it back to the cloud, after getting a replacement.<br />
<br />
At that level, it's all about "devops". Specifically what they call either <b>orchestration</b> or <b>choreography</b>. Depending, I suppose, on whether you listen to chamber music, or prefer dancing the samba.<br />
<br />
Here's the Netflix devops problem: In each country, there are daily viewing peaks. There are weekly viewing peaks. These peaks are 10x the baseline, and last a couple of hours. Then most people go to bed. This is the predictable part.<br />
<br />
Then there's the unpredictable side. When a beloved actor like Leonard Nimoy dies, there is a tendency for millions of people to go home via the bottle-shop and queue up every movie he's ever done as a kind of binge-tribute. I've heard.<br />
<br />
And that's the kind of situation that your scalable internet service <i>has </i>to handle, if you're going to serve movies on demand to 100 million people. Very rarely, you have to be able to service everyone at once. And you <i><b>cannot</b> </i>FailWhale. That was funny once, when it was Twitter. Once.<br />
<br />
The most amusing thing about Twitter is that once they got past the FailWhale, their company value went from merely silly to completely ludicrous. We are talking 10 million to 200 million, because they proved they were finally in the big league. What was the technical miracle which banished the white Whale? It was Scala... the primary language behind Spark.<br />
<br />
So, what's Spark? In a nutshell, it's next-gen Hadoop.<br />
<br />
"What was hadoop again?" you probably ask, since you probably never used it. Well, it was a giant hack that allowed hundreds of computers to be ganged together and carry out various file-processing tasks across the entire cluster.<br />
<br />
What for? Logfile processing, mainly. The daily re-indexing of Wikipedia. Places like eBay and Amazon used it for their first recommender systems ("other people also bought this!") and all because of the simple necessity of churning through more gigabytes of text than any single computer can manage.<br />
<br />
You have to realize that, to a large extent, the billions of dollars that eBay and Amazon are worth are because of their "people also bought" recommender systems. That list of five other things (five is the optimum psychological number) absolutely <i>must </i>be the best possible, where "best" is defined as "most likely to buy next". This is not advertising, this is lead generation. There are metrics.<br />
<br />
The point of lead generation is to turn each sale into an opportunity for another sale. "Accessorize, accessorize, accessorize!" and when those system break, or just degrade, then the bottom line impact is direct and palpable. Companies live and die by their ability to snowball those sales.<br />
<br />
Netflix had this happen, and they offered a million dollars to the mathematician who could solve it for them. This was the famous "<a href="http://www.netflixprize.com/">Netflix Prize</a>". The resulting algorithm is now known as "Alternating Least Squares", and the details are a topic for another day.<br />
<br />
Spark implements the ALS algorithm in it's standard mlib library. It's core. It's yours. You can have a million dollar algorithm, gratis. If you want to run ALS large scale, and this is most important - in real time - then Spark is the only option.<br />
<br />
The <b>only</b> option, unless you want to spend about a man-century implementing the equivalent of fine-grained distributed transaction control and data storage, and that's just the infrastructure your math needs to sit on top of.<br />
<br />
If you want to grow into the size of one of these services, you need to start with a seed capable of growing that large. Fortunately, in this analogy, those seeds are happily falling off the existing trees, and being blown about by the winds of open source software to the fertile fields of... some poetic metaphor I probably should have shut down a while ago.<br />
<br />
That means Scala, and Cassandra. That means Zookeeper, and message queues. haproxy to spread the load. Graphite to chart the rise and fall of resources. Ansible to spin up the servers. It means dozens of support tools you've never heard of, and would never run by choice if you didn't have a pressing need to get the job done.<br />
<br />
And these are all sub-components needed to support an overarching system like Spark - which schedules "parallel programs" across the entire cluster which are tolerant to the various slings and arrows of the internet.<br />
<br />
There is a level <i>above</i> Spark which is still in formation - exemplified by the Mesos project. That one seeks to be a kind of "distributed hypervisor" that can manage a cluster of machines and run many flavors of Spark, Hadoop, Cassandra or whatever within the single cluster. Otherwise we tend to get 'clusters of clusters' syndrome where each 'machine' is effectively only running one program.<br />
<br />
You have the dev cluster, and the testing cluster, as well as the production cluster of course. Oh, and that's one each for the database cluster, webserver/app cluster, and the small front-end routing clusters or logging cluster that hang off the big clusters...<br />
<br />
Yeah. Fire up the music, let the dance of clusters begin. Oh, and once you put on those magic dancing shoes, you can never <i>ever </i>take them off again until the company dies. This is <i>that </i>fairy-tale.<br />
<br />
Spark is the answer to questions you haven't asked yet. Literally, that's the kind of algorithms it is specialized to run. And it scales <i>all the way</i>. That's its value. That's what Apache is doing these days, trying to close the conceptual gap so both ends, big and small, are using the same base code. I love it.<br />
<br />
But no-one sells it, and the people who do use it in anger are too busy making billions of dollars to spend much time explaining exactly how, or writing documentation. You really gotta tease the information out of them, and watch a lot of their talks at Big Data conferences to see where all the pieces actually fit. There is an enormous learning curve.<br />
<br />
And that's why it's still such a secret.<br />
<br />
And given how many people read my blog, I'm sure will remain so.Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com1tag:blogger.com,1999:blog-7271315367558300569.post-15682764203862144752015-08-06T19:01:00.000-07:002015-09-07T23:03:40.898-07:00Tricopters, Birthdays, and LifeToday's my birthday, so I'm blogging. That alone should tell you how well my life's going at this point. The last year has been a little harsh, to be honest. Much of the good stuff turned out to be various kinds of vaporware, but all the bad stuff happened with full inevitability, all the same. You don't want to know.<br />
<br />
One of the few things that's been working out is the multirotors. Apparently, I'm quite good at them. My latest one is a little 180mm Tricopter made from a handful of hobbyking parts and zip-ties in an afternoon, to a plan that existed solely in my head, but based on experience with previous builds. (Especially the David Windestal tricopter)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-_5TgLpyA0LA/Ve5fx08voNI/AAAAAAAACYA/t2lSwjn7EzA/s1600/IMG_0357%255B1%255D.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="297" src="http://1.bp.blogspot.com/-_5TgLpyA0LA/Ve5fx08voNI/AAAAAAAACYA/t2lSwjn7EzA/s400/IMG_0357%255B1%255D.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">My little one, called "Fourth"</td></tr>
</tbody></table>
<br />
Putting that in Jedi terms; I now construct my own light-sabers - even if my skill at wielding them could still do with some work.<br />
<br />
Yesterday was my first successful FPV session, where 'successful' is defined as "wandering around an empty soccer field for three minutes without crashing". This builds on Tuesday's effort which only managed the first half of that statement, and required a day of CA glue and clamps to fix. Fortunately, crashing upside-down on a soccer goal net is one of the gentlest fails possible, and I feel like, for once, the universe gave me that mistake for free. Thanks, Spidey.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-vW3cxwsDRXc/Ve5f-jQrf6I/AAAAAAAACYI/yftiLLV3lc0/s1600/IMG_0355%255B1%255D.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="297" src="http://1.bp.blogspot.com/-vW3cxwsDRXc/Ve5f-jQrf6I/AAAAAAAACYI/yftiLLV3lc0/s400/IMG_0355%255B1%255D.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">"Dancer", folded up for storage. </td></tr>
</tbody></table>
<br />
ps. reason for crash? A butcherbird took me down. Damn little dark magpie harried me, made me panic, (via my spotter) and he's been insufferable about it ever since. Keeps turning up to gloat. Bird 1, Tricopter -1. Let's keep it there.<br />
<br />
If you want to get into flying robots, I recommend watching Iron Man again, especially the bits where he's still developing the suit and face-plants into the ceiling on his first attempts. He learns via failure, and it's good to have a friend standing by with a fire extinguisher. It's a lot like that, only without JARVIS.<br />
<br />
Well, so far. The HUDs are getting cooler. The equivalent of JARVIS is coming. And I'm sure someone's quad is already rocking a sound system capable of playing "Thunderstruck" at 99dB. If they exist, there's a good chance they'll be at the QLD FPV racing competition happening here in town in a couple of weeks, which I hope to attend. I've got my little 5.8Ghz facebox sorted out, so I'll be riding virtual shotgun as the (currently reigning) best in the world hurls his tech-avatar around a converted sheep-shed.<br />
<br />
Remember that each multirotor is its own little TV station. (pure analog video, baby!) The pilots stay locked on their own channel, but spectators - if we have the right equipment - can frequency-hop from craft to craft at will. I won't be winning any style competitions with my Borg visor compared to the cool Fat Sharks, but it should get the job done.Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com0tag:blogger.com,1999:blog-7271315367558300569.post-67579314071403888522015-03-20T19:33:00.000-07:002015-03-20T19:36:28.023-07:00Cheat your way to Success!First, just in case you're new to my sense of humor, the title is a Joke.<br />
<br />
Also, it's kind of not.<br />
<br />
Lying to other people is still bad, m'kay? Coping with reality is difficult enough, without you inventing bits of it that never existed. I don't mean "cheating" as in "I know I said I agreed to some common rules before starting the game, but, surprise!"<br />
<br />
I mean in the sense of "cheating nature", or even "cheating death". Those are not rules we agreed to in good faith, coming into the game. They weren't even properly written down. And it's also very difficult to take the re-negotiation route with Reality and, for example, talk Time or Gravity into being a little less strict.<br />
<br />
So in that case, it's perfectly fine to, in all good honestly, cheat like hell.<br />
<br />
If in doubt, you know you're doing the right kind of cheating if you'd be happy to arrange a press conference, and rock up like Tony Stark in his Iron Man suit to announce it in advance.<br />
<br />
Aerodynamics is a cheat. Medicine and Science in general is a cheat. The Large Hadron Collider is a complete and utter cheat... deliberately causing billions-to-one chances to come up nine times out of ten. (Talk about weighting the die!) And if you're going up against Ebola, I don't want you playing fair. There is no sportsmanship with pandemics.<br />
<br />
Technology is cheating. And the skillful use of it requires, above all, a kind of joyful cheating that is forever thinking up new things to do with things no-one has noticed, and gets hackers into trouble when they can't distinguish between natural and man-made restrictions.<br />
<br />
If you were in a foot race, suddenly substituting yourself with a robot scooter mid-event would be frowned upon. But in general life, it's a brilliant idea that gives us things like dishwashers and ATMs and insulin pumps.<br />
<br />
So, cheat at life. Cheat the universe! Just be honest with other people when you're doing it, and everything will be fine. They might even help.<br />
<br />Anonymoushttp://www.blogger.com/profile/13912838690573000458noreply@blogger.com0