Web 2.0 is about data exchange and classification. Taking a high-level look at software design process one of areas that is good in theory, that fails in practice is data exchange and classification.

Think about the last software project you worked on – would a new programmer be able to pick up the documentation and work out what is going on? Could they be able to find the documentation? Was there documentation at all?

Documentation is all about data exchange – I bet if you went through your email or project mailing lists you could piece together a decent amount of usable documentation (Implementation decisions, solutions, bug reports etc) – Can Web 2.0 provide the glue to ease the burden of documentation?

Imagine being able to tag bugs and design decisions so searching for a bug returns a link that points to the design documents and comments entered at check-in. This gives the developer background information quickly – and these docs are living and more easily maintainable. As a developer, you don’t have to change to documentation mode – it is already part of your work flow (Other than remembering to enter check-in comments – but you already do that don’t you?)

We don’t necessarly have to limit this to documentation either. Project managers could use it to gauge where the project is at, clients could actually report bugs with out having to learn an obscure bug reporting interface or software terminology. How about timesheets? Add an entry stating you started work at a particular time, and then enter another entry saying you finish at a particular time – tag it wit the project name, maybe what part of the project you were working on – Voila! Or meeting minutes – these often have action items interleaved – tag them and make them searchable. Tag them as completed when you are done.

But I think the best bit is that it wouldn’t require anything more than an email client or a blog-like web interface. EVERYTHING IN ONE PLACE! I know I personally hate having to log in to a different apps for bugs, and meeting minutes etc. We should be letting the software organise our data, and leveraging search.

I think I will be revisiting this one soon…

It is important to step back sometimes, and to look at what is going on. It is all too easy to become completely immersed in things in what is going on in your own life, that you forget how lucky you really are.

Today, I went to the launch of the Breast Surgery Gallery – a system that was developed by a uni mate of mine, Patrick MacQuillan as his honours project a couple of years ago. My honours supervisor (David Glance) was Patrick’s supervisor, and he asked me to do a quick re-design for their website before the launch today, which is why I was invited.

It was quite strange – as we walked into the offices, we noticed their was only two other guys there – the rest were woman. Now nothing was explicitly said, but it is pretty safe to say that a majority of the woman there (If not all) had been affected by breast cancer at some point in their lives.

As I was standing there, listening to the presentation, it hit me that these woman have had to deal with something that I could not even begin to fathom. Yet, they were here smiling – genuinely happy to see each other and to witness the official launch of a product that is actually helping people.

David, who has worked in large software companies for many years said something very profound (Maybe more so when you think about it) – he said: it was good to see a piece of software that is helping people.

We have all probably thought that a system we were writing was going to help someone, but not in the same way as this one did. The stuff that we write day-to-day will probably make someones work day more efficient, or allow them to shop more convienently – ironically the sort of stuff that in the whole scheme of things doesn’t really matter. This software was playing a roll in easing the pychological pain of life changing and life saving surgery, helping the suffers on the road to recovery.

It really isn’t often that software does that…

I was going thorugh the posts from my old, now-defunct blog, seeing if there was anything I could bring over here — it is amazing how much can change in a year. There was an article I wrote about over use of cool techniques. In that article, I made mention to some new fangled technique called “Design Patterns”. At that point, I had no idea what they were and frankly couldn’t care.

Well, after being forced to look at them more closely for a uni assignment, I’m kind of a little hooked.

Design patterns are abstract solutions to common problems. Huh? Yeah, that is what I said. Many programmers strive towards code re-use. Design Patterns encourage thought re-use. Why re-invent the wheel? And because they are abstract (i.e. no code), they can be “ported” to different languages easily.

The Gang of Four introduced 23 of the things. They have put the challenge out for the discovery of more, and they haven’t been too successful as it is believed that almost all problems can be broken down to fit a composition of these rules.

To implement many of these design patterns correctly, you really need OOP features such as Abstraction and Interfacing. As I have pointed out many times, PHP4 doesn’t have these. PHP5 does. However! if you think about it, you can still use design patterns in PHP4 — you just need to be a little bit careful.

I won’t go through ever design pattern just now, but, I’ll outline one, and maybe add to them in the future. [By the way — an awesome book to use for getting your head around design patterns is “Head First Design Patterns” — look it up on Amazon]

The Stategy Pattern

Programmers often get into the habit of extending classes to change functionality — maybe because it is the easiest OOP function to understand. It can lead to problems though when you need to change implementations in a base class. I’ll use an example from a system I was writing the other day: the ubiquitous PHP email sender. The job I was doing required two types of email to be sent — a run-of-the-mill text email and a text email with an attachment.

Because I’m always trying to build my code libraries up, I decided to create set of classes that will do the job. Now, pre-design patterns, something like this may have happened:

Nothing wrong so far, but what happens if we want to add a HTML email? We could probably add another class below the attachments, because the HTML component is an attachment I guess, so we add the HTML file as one of the attachments at design time. This isn’t a great solution though, because one of the necessary bits to a HTML email is the HTML content, and you could theoretically remove it. How about we add another variable specifically for the HTML copy? That would also work, but what if the client decided that they wanted to select between HTML and text emails dynamically at run time? You would end up with something like this code wise:

switch($mailType) {  
  case “text”:  
    $email->setBody($text);  
  break;  
  case “html”:  
    $email->setHTMLBody($htmlText);  
  break;  
}

(This is a bit of a contrived example, so run with me here)

now what happens if you want to add another type of email? You have to modify the case statement and test the whole thing all over again. Everyone hates testing – especially code that you know used to work that you had to change. Two keywords: CHANGE and TESTING. Minimise both and we as programmers are happy.

Now my solution was to create a class that has everything an email needs – To, From , Subject and a repository for headers (Plus a few other bits and pieces, which I won’t mention to reduce clutter). It also has a function called send(). Wow, ground breaking. Where the design gets a little strange, is there is another variable, called $emailType. This variable stores reference to a class of type EmailType (Well it pretends to, PHP4 doesn’t support interfaces). So, any class that implements EmailType can be stored in that variable. One of the abstract classes (again, let’s pretend it is abstract, PHP4 won’t understand) is the createMessage() function. This is where the magic occurs…

Each class that implements that interface know exaclty how it message needs to be contructed. The base email class doesn’t care – as long as it gets a string to tack on to the email it is happy. The creation of messages is de-coupled, meaning you can create a new email class without changing any exisiting code (As long as it implements the interface correctly).

This is an implementation of the strategy pattern!

Formal Description [Ganf of Four]: The strategy pattern defines a family of algorithms, encapsulates each one, and makes them interchangable. Strategy lets the algorithm vary independently from clients that use it.

Which is what we just did… More on design patterns later.

It would be pretty safe to say that PHP is the language of choice for freelance developers and boutique designers. I suppose that this stems from the fact that is freely available, easy to set up and and easy to administer, which results in almost every web host (in Australia at least) running it. In fact, most hosts over here ONLY run PHP.

It would be interesting to see why ASP.NET hasn’t got more market penetration than it has – as much as I hate to say it, I’m putting my money on the fact it costs money.

CGI/PERL scripts are quickly fading away into obscurity – I remember when they were the only choice you had. It’s a security and maintainence thing. RIP mod_perl.

ColdFusion support over here is Oz is virtually non-existent. There are two hosts here in Perth, although it still seems popular in government and places that host their own servers.

New, funky languages such as Python and Ruby are still in their minority. From a configuration stand-point, Python is no better than PERL. It is still effectively a scripting language that has been extended to work with the web. At least PHP was specifically designed as such. Ruby is too new and support intensive. I hope that this changes. It looks cool (Although, I don’t like the syntax – I’m a fan of the curly bracket. So shoot me).

It seems that PHP was in the right place at the right time. When it arrived, dynamic web was still in it’s infancy – you had PERL and that was it. It allowed pretty complex systems to be built easily and cheaply (Sure, Coldfusion and ASP were around then, but were usually out of reach of the tinkerer), now it has so much history, no one wants to let go.

So we are left with old-trusty PHP…

Now, don’t get my wrong – PHP has treated me well. In fact, up until recently, PHP counted for 95% of the coding that I did. But, the lack of some object oriented features urked me – in particular exceptions and interfaces. It is very difficult to implement design patterns properly without the ability to abstract, overload or interface.

Now before any of you point out that PHP 5 has implemented most of this stuff, let me cut you off at the pass. I know PHP 5 supports many of the thinks that I’ve been waiting for, but I’m still stuck using PHP 4 for day-to-day work. Why? BECAUSE YOU CAN’T RUN THEM BOTH AT THE SAME TIME – well, not easily anyway.

I’ve been working with the ASP.NET 2 Beta for the last couple of months and it has no problems co-existing with ASP.Net 1.x, all you do is change a setting in IIS and you are away. Why then, do you have to jump through so many hoops to get PHP 4 and 5 to co-exist? It is for this reason, that most web hosting companies will not support 5 for a long time. From an economic point-of-view, they would be stupid to. Most applications run on 4. Not all of these applications will run of 5.

I have come up with a work-around, which is a pretty neat solution (I’ll present it later on) – it does require two different apache binaries to run and is a bit of a hassle to setup, but it does the job quite well (It would do it even better if I had more than one IP address), but I shouldn’t have to use work arounds.

Perhaps we will be left stifled by the shortcomings of PHP 4, at least in our bread-and-butter jobs, for a while yet. Perhaps one of the “killer” frameworks will eventually hit critical mass and knock it off it podium. Then again. Maybe not…

Talking to the folk at Port80 tonight, I have been re-inspired to start blogging again. This may have something to do with all of the excitement around web essentials 05 (Which I missed – Phooey!). You see all of those lucky enough from Perth to go, have gotten me even more excited about the web industry, and I want to contribute back. So in a way you have Miles, Kay and Adrian to thank for this… he he he.

This blog will mainly be work related. Some mis-guided ramblings I suppose, but that is what blogging is all about isn’t?

I was amused to notice that the last time I started a blog was almost exactly one year ago (10/10/2004 to be exact). The reasons I did it last time surely had to do with avoiding work on my thesis. Ironically, I have another thesis to hand in in a couple of weeks – similar topic different degree… Looking into Gestalt theory again, but this time, I’m adding a part about remote usability testing. I’ve written a cool little AJAX based data logger, which I might atlk about another time.

Anyway – I’m going to try to get some sleep :)

Hi! I’m Myles. I’m a Melbourne-based software engineer and product designer.

I have a background in both frontend and backend development and I have a keen interest in product, design and business.

I hold a strong belief that everything an engineer builds interacts with a person in some way - I want to understand that interaction and make it better.

I believe developers are worth more velocity points on a card wall. They should be included in the whole design process. I want to work in iterative multi-disciplinary teams.

I have a strong interest in IoT development, in particular the ESP8266, the Thread protocol, 6lowpan and 802.15.4. I’m currently automating my house.

I’m a co-author of “Build Mobile Websites and Apps for Smart Devices” (Published by SitePoint).

I’ve also spoken at a number of International Web conferences, including Web Directions and Edge of the Web on the topics of Javascript, Mobile development and OAuth and OpenID.

You can find me on Mastodon and Github

Deploying a Home Assistant Voice Assist on a OrangePI Zero using Balena

This yak shave started when I installed Tandoor so I could store all of our recipes in one place. It has a handy meal planner function that has been super useful for organising our weekly meal plans. It also has a shopping list builder, which is also pretty handy.

However, we currently use a rube goldberg setup of Microsoft Todo and Amazon Alexa with some third party Alexa app for shopping lists, as it means my wife and I can easily share the list, I can see it on my watch while shopping and we can add things to it using voice commands (which is very useful when cooking).

The problem is both Microsoft Todo and Alexa have been on a steady route of enshittification. The latest version of Todo’s watch integration has been getting worse and worse, and Alexa has been slowly getting dementia.

I began to wonder if I could build a little shim app that would allow me to monitor the contents of the Amazon Alexa Shopping list, and populate the Tandoor list, allowing us to get rid of Microsoft Todo completely.

It turns out the enshittification was going to accelerate in June, when the Alexa team will drop support for the Todo list REST api. I’m pretty sure this will mean the Todo integration will die, and one of the main use cases for our Alexa will stop working.

Great.

I’ve been meaning to try out the fruits of the Home Assistant year of the voice for awhile - maybe it was time.

Setting up Home Assistant

First things first, I needed to work out how local voice assistants worked in Home Assistant.

Clearly, my search skills aren’t what they used to be (lol, no - search engines are just shit now), but it seemed like I needed install Rhasspy - which is no longer the case.

The main developer behind Rhasspy now works for Nabu Casa (the entity that builds Home Assistant), and has created a bunch of new packages. The new packages are whisper for speech-to-text, piper for text-to-speech. These packages are glued together using a new protocol called wyoming.

If I was using HASSOS to run home assistant, it would have been a straightforward matter of installing the add-on, and enabling the integrations. However, I run home assistant in a Docker container, which made things slightly more complicated.

After a bit more searching, I managed to find references to the Docker containers that HASSOS use under the hood, and I was able to translate them into docker-compose entries.

---
version: 3.0
services:
  wyoming-piper:
    image: rhasspy/wyoming-piper
    container_name: wyoming-piper
    restart: unless-stopped
    volumes:
      - /config/piper/data:/data
    ports:
      - "10200:10200"
    command: --voice en_US-lessac-medium
    
  wyoming-whisper:
    image: rhasspy/wyoming-whisper
    container_name: wyoming-whisper
    restart: unless-stopped
    volumes:
      - /config/whisper/data:/data
    ports:
      - "10300:10300"
    command: --model tiny-int8 --language en

Next, I added the Wyoming integration. The trick here is to add two instances, one with port 10200 and one with 10300 - the integrations will automatically work out which is which and add the correct entities.

Finally, I setup a new Assistant, using the faster-whipser and piper options for text-to-speech and speech-to-text.

After testing it out using the Assist icon in the web app, I confirmed it was all working!

Time for some hardware

Having to open a web browser, and click and icon to task home assistant to do a thing isn’t quite the same experience as using an Amazon Echo Dot.

First of all, we’d need a wake-word to replace the icon click, and maybe some sort of stand alone hardware to replace the web brower.

Thankfully, there are numerous examples of prior art.

The two main options seem to be using a Raspberry PI or equivalent and a USB microphone and speaker; or an ESP32 with a MEMS microphone and I2S speaker.

While I actully had all the bits to do the latter kicking around my workshop, there are a few disadvantages with this setup.

The ESP32 doesn’t really have the grunt to do a the AI processing to perform the wake word analysis, so it basically constantly streams audio back to home assistant, whcih does the analysis there. I don’t love this - it seems like a bit of a waste of resources to chew up bandwidth, and CPU just to look for a wake word.

Now, they have started another project to use a even smaller and more effecient wake word detector that can run on the ESP, but it’s not ready.

So, I started looking into some sort of cheap and low power embedded linux system that could run wake-words, and probably speech-to-text and text-to-speech, effectively just pushing JSON intents to Home Assistant, and receiving text back.

I have a bunch of OrangePi Zeros kicking around which I purchased ages ago, when they were $10 each (they are now $30 - thanks Covid). Couple that with a USB speakerphone, I reckon these could be a cute little solution.

Because nothing I ever do is easy, I decided to deploy these using Balena. I had had some good success doing this for my zigbee2mqtt device, which is also on an OrangePi zero.

I started out looking at some instructions for setting up Raspberry PIs, and saw there are three components needed: wyoming-mic-external for accessing microphones, wyoming-snd-external for accessing speakers and wyoming-satellite which ties everything together.

I got a copy of all the relevant repositories, and setup the following docker-compose.yaml file, to see how far I got.

version: "2.4"
services:
  microphone:
    build:
      context: ./services/wyoming-mic-external
    ports:
      - "10600:10600"
    devices:
      - /dev/snd:/dev/snd
    group_add:
      - audio
    command:
      - "--device"
      - "sysdefault"
      - "--debug"
  playback:
    build:
      context: ./services/wyoming-snd-external
    ports:
      - "10601:10601"
    devices:
      - /dev/snd:/dev/snd
    group_add:
      - audio
    command:
      - "--device"
      - "sysdefault"
      - "--debug"
  satellite:
    build:
      context: ./services/wyoming-satellite
    ports:
      - "10700:10700"
    command:
      - "--name"
      - "my satellite"
      - "--mic-uri"
      - "tcp://microphone:10600"
      - "--snd-uri"
      - "tcp://playback:10601"
      - "--debug"

Not super far. The default BalenaOS kernel is very bare bones - there were no sound drivers.

Compiling “custom” modules for balena

The BalenaOS image for the OrangePi zero was community submitted, and is very old. One thought I had was to try and figure out how to update it to have the modules I needed, and ideally update it.

But after a bit of research it became clear I needed to leave Zephry, and a bunch of other esoteric embedded things to do that. While that sounds fun, it was beyond the scope of what I wanted to do on this project, so I needed to figure out a different way to get these modules on to the device.

It turns out, a priviledged docker container can insert modules into kernel space, and the accepted way to load custom modules is to build them into a container, and insmod them from there.

Setting up a cross compile docker container

Started down this track, then thought about th example from balena’s github - that doesn’t do any explicit cross compiling. It also builds on the host, but runs on the target. The balena build process must already be setup for cross compiling. Let’s test that theory…

Compiling a test program

I cribbed this little beauty, and saved it as test.c

#include <stdio.h>

int main()
{
  printf("Hello World!");
  return 0;
}

Then created this Dockerfile

FROM debian:latest

RUN apt update && apt install -y build-essential
RUN mkdir -p /usr/src/test
COPY ./test.c /usr/src/test/test.c
WORKDIR /usr/src/test
RUN gcc -o /usr/bin/hello_world test.c

ENTRYPOINT [ "/usr/bin/hello_world" ]

and add this to docker-compose.yml

---
  hello_world:
    build:
      context: ./services/test
    privileged: true
    restart: on-failure

Success!

Compiling a vanilla kernel

Next step - can we recompile a kernel with some modules?

Luckily, the config.gz file was present in the /proc directory, so we can use that as a starting point.

To get it off the device in the first place, I used ssh. When in developer mode, balena devices have an open ssh port on 22222, so after finding the IP address, I ran ssh -t -p 22222 root@192.168.1.244 "gunzip -k -c /proc/config.gz" > .config. Why not scp? It didn’t work. There was something about the way it was setup on the remote that caused it to fail.

Next, we need to enable the sound stuff. Add this to the .config file

CONFIG_SOUND=m
CONFIG_SND=m
# CONFIG_SND_OSSEMUL is not set
CONFIG_SND_PCM_TIMER=y
# CONFIG_SND_HRTIMER is not set
# CONFIG_SND_DYNAMIC_MINORS is not set
CONFIG_SND_SUPPORT_OLD_API=y
CONFIG_SND_PROC_FS=y
CONFIG_SND_VERBOSE_PROCFS=y
# CONFIG_SND_VERBOSE_PRINTK is not set
# CONFIG_SND_DEBUG is not set
# CONFIG_SND_SEQUENCER is not set
CONFIG_SND_DRIVERS=y
# CONFIG_SND_DUMMY is not set
# CONFIG_SND_ALOOP is not set
# CONFIG_SND_MTPAV is not set
# CONFIG_SND_SERIAL_U16550 is not set
# CONFIG_SND_MPU401 is not set

#
# HD-Audio
#
# end of HD-Audio

CONFIG_SND_HDA_PREALLOC_SIZE=64
CONFIG_SND_SPI=y
CONFIG_SND_USB=y
# CONFIG_SND_USB_AUDIO is not set
# CONFIG_SND_USB_UA101 is not set
# CONFIG_SND_USB_CAIAQ is not set
# CONFIG_SND_USB_6FIRE is not set
# CONFIG_SND_USB_HIFACE is not set
# CONFIG_SND_BCD2000 is not set
# CONFIG_SND_USB_POD is not set
# CONFIG_SND_USB_PODHD is not set
# CONFIG_SND_USB_TONEPORT is not set
# CONFIG_SND_USB_VARIAX is not set
# CONFIG_SND_SOC is not set

Using debian:latest failed because (I think) GCC 11 has some issues with some thing in the kernel (I was some sort of double import, I didn’t bother to investigate other than some cursory googling), so I tried to find the oldest version of debian that would still work. At the time of writing, it was debian:buster which comes out of LTS on 30 June 2024. After that, it may not work anymore with missing mirrors. Who knows (In reality, you can probably find a mirror that works…)

So that took 8 hours on my Macbook Pro because Docker. I fired up a real linux machine so I could build things a bit faster, and then saw a previous attempt at do this nonsense. It was a patched kernel from sunxi 5.4.20-sunxi and wondered if I could just load up the modules. I mean, it was a long shot, but it didn’t work:

modprobe: can't load module soundcore (kernel/sound/soundcore.ko): invalid module format

So, let’s build on a real linux box. No good - balena uses QEMU when building containers (makes sense - you can run a docker recipe on a different target - the binaries won’t run).

Back to cross compiling?

docker build . -t wyoming-satellite-kernel-builder docker run --mount type=bind,source=./linux-5.4.18,target=/usr/src/linux --mount type=bind,source=./modules,target=/lib/modules/5.4.18 wyoming-satellite-kernel-builder make oldconfig

FROM alpine
RUN mkdir -p /lib/modules/5.4.18/kernel
COPY modules.builtin /lib/modules/5.4.18/modules.builtin
COPY sound /lib/modules/5.4.18/kernel/sound
ENTRYPOINT ["/bin/sh"]