In software-defined radio, there are well-established ways of visually representing the signal(s) in the entire bandwidth available from the hardware; we create a plot where the horizontal axis is frequency (using the Fourier transform to obtain the data). Then either the vertical axis is amplitude (creating an ordinary graph, sometimes called panorama) or the vertical axis is time and color is amplitude (creating a waterfall plot).
Here is an example of ShinySDR's spectrum display which includes both types (y=amplitude above and y=time below):
A further refinement is to display in the graph not just the most recent data but average or overlay many. In the above image, the blue fill color in the upper section is an overlay (both color and height correspond to amplitude), the green line is the average, and the red line is the peak amplitude over the same time interval.
We can see signals across an immensely wide spectrum (subject to hardware limitations), but is there a way to hear them meaningfully? Yes, there is, with caveats.
What's pictured above is a small portion of the band assigned to aviation use — they are used primarily for communication between aircraft in flight and air traffic control ground stations. The most significant thing about these communications is that there are a lot of different frequencies for different purposes, so if you're trying to hear “what's in the area”, you have to monitor all of them.
The conventional solution to this problem is a scanner, which is a radio receiver programmed to rapidly step through a range of frequencies and stop if a signal is detected. Scanners have disadvantages: they will miss the beginning of a signal, and they require a threshold set to trade off between missing weak signals and false-triggering on noise.
An alternative, specific to AM modulation (which is used by aircraft), is to make a receiver with very poor selectivity: the ability to receive only a specific channel and ignore other signals. (Historically, when RF electronic design was less well understood and components had worse characteristics, selectivity was a specification one would care about, but only if one lived in an area with closely-spaced radio stations — today, every receiver has good selectivity.)
I'm going to explain how to build an unselective receiver in software, and then refine this to create spatial audio — that is, the frequency of the signal shall correspond to the stereo panning of the output audio. This is the analogue of the spectrum display in audio.
Of course, this is an AM receiver and so it will only make intelligible sound for amplitude-modulated signals. However, many signals will produce some sound in an AM receiver. The exception is that a clean frequency-modulated (FM) or phase-modulated signal will produce silence, because its amplitude is theoretically constant, but this silence is still audibly distinct from background noise (if the signal is intermittent), and transmitted signals often do not have perfect constant amplitude.
A normal software AM demodulator has a structure like the following block diagram (some irrelevant details omitted). The RF signal is low-pass filtered to select the desired signal, then demodulated by taking the magnitude (which produces an audio signal with a DC offset corresponding to the carrier).
In order to produce an unselective receiver, we omit the RF filter step, and therefore also the downsampling — therefore demodulating at the RF sample rate. The resulting real signal must be low-pass filtered and downsampled to produce a usable audio sample rate (and because the high-frequency content is not interesting; see below), so we have now “just” swapped the two main components of the receiver.
This simple change works quite well. Two or more simultaneous AM signals can be received with clear stereo separation.
One interesting outcome is that, unlike the normal AM receiver, the audio noise when there is no signal is quieter (assuming AGC is present before the demodulator block in both cases) — this conveniently means that no squelch function is needed.
The reason for this is obvious-in-hindsight: loosely speaking, most of the noise power will be at RF frequencies and outside of the audio passband. In order to have a strong output signal, the input signal must contain a significant amount of power in a narrow band to serve as the AM carrier and sideband. (I haven't put any math to this theory, so it could be nonsense.)
In order to produce the spatial audio, we want the audio signal amplitude, in a single stereo channel, to vary with frequency. And that is simply a filter with a sawtooth frequency response. The signal path is split for the two stereo channels, with opposite-slope filters. (AGC must be applied before the split.)
An undesired effect is that near the band edges, since the filter has a steep but not perfectly sharp transition from full-left to full-right, there is a lot of slope detection (output from frequency-modulated signals) that does not occur anywhere else. Of course,
This design can of course be applied to more than two audio channels; using surround sound would avoid the need for steepness of the filter at the edges and map the inherently circular digitized spectrum to a circular space, so it's worth trying.
I've implemented this in ShinySDR (and it is perhaps the first novel DSP feature I've put in). Just click the “AM unselective” mode button.
Some “directions for future research”:
As I mentioned above, this is useless for listening to FM signals. Is some technique which can do the same for FM? Naïvely creating an “unselective FM receiver” seems like it would be a recipe for horrible noise, because to a FM demodulator, noise looks like a very loud signal (because the apparent frequency is jumping randomly within the band, and frequency maps to amplitude of the output).
If we declare that the output need not be intelligible at all, is there a way to make a receiver that will respond to localized signal power independent of modulation? Can we make an unmodulated carrier act like an AM signal? (CW receivers do this using the BFO but that is dependent on input frequency.)
The usual definition of the decibel is of course that the dB value y is related to the proportion x by
y = 10 · log10(x).
It bothers me a bit that there's two operations in there. After all, if we expect that y can be manipulated as a logarithm is, shouldn't there be simply some log base we can use, since changing log base is also a multiplication (rather, division, but same difference) operation? With a small amount of algebra I found that there is:
y = log(100.1)(x).
Of course, this is not all that additionally useful in most cases. If you're using a calculator or a programming language, you usually have loge and maybe log10, and 10·log10 will have less floating-point error than involving the irrational value 100.1. If you're doing things by hand, you either have a table (or memorized approximations) of dB (or log10) and are done already, or you have a tedious job which carrying around 100.1 is not going to help.
As vaguely promised before, another update on what I've been working on for the past couple of years:
Specifically, it is in the same space as Gqrx, SDR#, HDSDR, etc.: a program which runs on your computer (as opposed to embedded in a standalone radio) and uses a peripheral device (rtl-sdr, HackRF, USRP, etc.) for the RF interface. Given such a device, it can be used to listen to or otherwise decode a variety of radio transmissions (including the AM and FM broadcast bands everyone knows, but also shortwave, amateur radio, two-way radios, certain kinds of telemetry including aircraft positions, and more as I get around to it).
ShinySDR is basically my “I want my own one of these” project (the UI still shows signs of “I’ll just do what Gqrx did for now”), but it does have some unique features. I'll just quote myself from the README:
I (Kevin Reid) created ShinySDR out of dissatisfaction with the user interface of other SDR applications that were available to me. The overall goal is to make, not necessarily the most capable or efficient SDR application, but rather one which is, shall we say, not clunky.
Here’s some reasons for you to use ShinySDR:
Remote operation via browser-based UI: The receiver can be listened to and remotely controlled over a LAN or the Internet, as well as from the same machine the actual hardware is connected to. Required network bandwidth: 3 Mb/s to 8 Mb/s, depending on settings.
Phone/tablet compatible (though not pretty yet). Internet access is not required for local or LAN operation.
Persistent waterfall display: You can zoom, pan, and retune without losing any of the displayed history, whereas many other programs will discard anything which is temporarily offscreen, or the whole thing if the window is resized. If you zoom in to get a look at one signal, you can zoom out again.
Frequency database: Jump to favorite stations; catalog signals you hear; import published tables of band, channel, and station info; take notes. (Note: Saving changes to disk is not yet well-tested.)
Map: Plot station locations from the frequency database, position data from APRS and ADS-B, and mark your own location on the map. (Caveat: No basemap, i.e. streets and borders, is currently present.)
- Audio: AM, FM, WFM, SSB, CW.
- Other: APRS, Mode S/ADS-B, VOR.
If you’re a developer, here’s why you should consider working on ShinySDR (or: here’s why I wrote my own rather than contributing to another application):
All server code is Python, and has no mandatory build or install step.
Plugin system allows adding support for new modes (types of modulation) and hardware devices.
Demodulators prototyped in GNU Radio Companion can be turned into plugins with very little additional code. Control UI can be automatically generated or customized and is based on a generic networking layer.
On the other hand, you may find that the shiny thing is lacking substance: if you’re looking for functional features, we do not have the most modes, the best filters, or the lowest CPU usage. Many features are half-implemented (though I try not to have things that blatantly don’t work). There’s probably lots of code that will make a real DSP expert cringe.
Now that I've finally written this introduction post, I hope to get around to further posts related to the project.
At the moment, I'm working on adding the ability to transmit (given appropriate hardware), and secondarily improving the frequency database subsystem (particularly to have a useful collection of built-in databases and allow you to pick which ones you want to see).
Side note: ShinySDR may hold the current record for most popular program I've written by myself; at least, it's got 106 stars on GitHub. (Speaking of which: ShinySDR doesn't have a page anywhere on my own web site. Need to fix that — probably starting with a general
topics/radio. Eventually I hope to have a publicly accessible demo instance, but there’s a few things I want to do to make it more multiuser and robust first.)
My interactive presentation on digital signal processing (previous post with video) is now available on the web, at visual-dsp.switchb.org! More details, source code, etc. at the site.
(P.S. I'll also be at the next meetup, which is tomorrow, January 21, but I don’t have another talk planned. (Why yes, I did procrastinate getting this site set up until a convenient semi-deadline.))
I have really failed to get around to blogging what I've been doing lately, which is all software-defined radio. Let's start fixing that, in reverse order.
Yesterday, I went to a Bay Area SDR meetup, “Cyberspectrum” organized by Balint Seeber and gave a presentation of visual representations of digital signals and DSP operations. It was very well received. This video is a recording of the entire event, with my talk starting at 12:30.
The theme of the game is “be consistent”. It's a minimalist-styled 2D platformer. The core mechanic is that whatever you do the first time, the game makes it so that that was the right action. Examples of how this could work:
At the start, you're standing at the center of a 2×2 checkerboard of background colors (plus appropriate greebles, not perfect squares). Say the top left and bottom right is darkish and the other quadrants are lightish. If you move left, then the darkish stuff is sky, the lightish stuff is ground, and the level extends to the left. If you move right, the darkish stuff is ground, and the level extends to the right.
The first time you need to jump, if you press W or up then that's the jump key, or if you press the space bar then that's the jump key. The other key does something else. (This might interact poorly with an initial “push all the keys to see what they do”, though.)
You meet a floaty pointy thing. If you walk into it, it turns out to be a pickup. If you shoot it or jump on it, it turns out to be an enemy.
If you jump in the little pool of water, the game has underwater sections or secrets. If you jump over the little pool, water is deadly.
(I could say some meta-commentary about how I haven't been blogging much and I've made a resolution to get back to it and it'll be good for me and so on, but I think I've done that too many times already, so let's get right to the actual thing...)
When I wrote Cubes (a browser-based “Minecraft-like”), one of the components I built was a facility for key-bindings — that is, allowing the user to choose which keys (or mouse buttons, or gamepad buttons) to assign to which functions (move left, fly up, place block, etc.) and then generically handling calling the right functions when the event occurs.
Now, I want to use that in some other programs. But in order for it to exist as a separate library, it needs a name. I have failed to think of any good ones for months. Suggestions wanted.
Preferably, the name should hint at that it supports the gamepad API as well as keyboard and mouse. It should not end in “.js” because cliche. Also for reference, the other library that arose out of Cubes development I named Measviz (which I chose as a portmanteau and for having almost zero existing usage according to web searches).
(The working draft name is
web-input-mapper, which is fairly descriptive but also thoroughly clunky.)
If you're feeling virtuous:
- Figure out what's going on.
- Figure out why what's going on wasn't immediately obvious.
- Make it so that such failures are caught and reported obviously.
- Make it so that the rest of the system recovers from such failures.
- Write a test for the bug, and a couple more while you're at it.
- Write the actual fix.
(I ought to do more actual concrete blogging, like what I've been doing lately. This crossed my mind as a light and easy piece — I actually followed part of this procedure yesterday after pushing a version (of what, I'll get to later) that was rather broken.)
One of the nice things about Common Lisp is the pervasive use of (its notion of) symbol objects for names. For those unfamiliar, I'll give a quick introduction to the relevant parts of their semantics before going on to my actual proposal for a “good parts version”.
A CL symbol is an object (value, if you prefer). A symbol has a name (which is a string). A CL package is a map from strings to symbols (and the string key is always equal to the symbol's name). A symbol may be in zero or more packages. (Note in particular that symbol names need not be unique except within a single package.)
Everywhere in CL that something is named — a variable, a function, a class, etc. — the name is a symbol object. (This is not impractical because the syntax makes it easy to write symbols; in fact, easier than writing strings, because they are unquoted.)
The significance of this is that the programmer need never give significance to characters within a string name in order to avoid collisions. Namespacing of explicitly written symbols is handled by packages; namespacing of programmatically generated symbols is handled by simply never putting them in any package (thus, they are accessible only by passing references); these are known as gensyms.
Now, I don't mean to say that CL is perfect; it fails by way of conflating too many different facilities on a single symbol (lexical variables, dynamic variables, global non-lexical definitions, ...), and some of the multiple purposes motivate programmers to use naming conventions. But I think that there is value in the symbol system because it discourages the mistake of providing an interface which requires inventing unique string names.
(One thinking along capability lines might ask — why use names rather than references at all? Narrowly, think about method names (selectors, for the Smalltalk/ObjC fans) and module exports; broadly, distribution and bootstrapping.)
So, here’s my current thought on a “good parts version”, specifically designed for an E-style language with deep equality/immutability and no global mutable state.
There is a notion of name, which includes three concrete types:
- A symbol is an object which has a string-valued name, and whose identity depends solely on that string.
- A gensym also has a name, but has an unique identity (selfish, in E terms). Some applications might reject gensyms since they are not data.
- A space-name holds two names and its identity depends solely on that combination. (That is, it is a “pair” or “cons” specifically of names.)
Note that these three kinds of objects are all immutable, and use no table structures, and yet can produce the same characteristics of names which I mentioned above. (For implementation, the identity of a name as above defined can be turned into pointer identity using hash consing, a generalization of interning.) Some particular examples and notes:
- A CL symbol in a package corresponds to a pair of two symbols, or perhaps a gensym and a symbol. This correspondence is not exact, of course. (In particular, there is no notion here of the set of exported symbols in a package. But that's the sort of thing you have to be willing to give up to obtain a system without global mutable state. And you can still imagine 'linting' for unexpected symbols.)
- The space-name type means that names can be arbitrary binary trees. If we consistently give the left side a “namespace” interpretation and the right side a “local name” one, then we have a system, I think, where people can carve out all sorts of namespaces without ever fearing collisions or conflicts, should it become necessary. Which probably means it's massively overdesigned (cf. "worse is better").
- Actual use case example: Suppose one wishes to define (for arbitrary use) a subtype of some well-known interface, which adds one method. There is a risk that your choice of name for that method conflicts with someone else's different subtype. Under this system, you can construct a space-name whose two components are a large random number (i.e. a unique ID) acting as the namespace, and a symbol which is your chosen simple name. One can imagine syntax and tools which make it easy to forget about the large random number and merely use the simple name.
- It's unclear to me how these names would be used inside the lexical variable syntax of a language, if they would at all; I suspect the answer is that they would not be, or mostly confined to machine-generated-code cases. The primary focus here is improving the default characteristics of a straightforwardly written program which uses a map from names to values in some way.
(This is all very half-baked — I'm just publishing it on the grounds described in my previous post: in the long run I'll have more ideas than I ever implement, and this is statistically likely to be one of them, so I might as well publish it and hope someone else finds some use for it; if nothing else, I can stop feeling any obligation to remember it in full detail.)