Nothern Utah WebSDR Logo - A skep with a Yagi Northern Utah WebSDR
Using "csdr" for auxiliary receiver streams on a WebSDR system

Preliminary Version

What is this?

On another page (see:  "Duplicating receiver streams using ALSA and asoundrc" - Link) we discussed using tools in ALSA to take an already-existing raw I/Q stream from a "sound card" type receiver (which could be an actual sound card - or it could be an RSP1a running at up to 768 kHz as described on this page:  Operating a WebSDR receiver at 384 or 768 kHz using the 16 bit signal path - Link) and "duplicating" this stream so that we could use our already-existing receiver data elsewhere.

One of the "use cases" mentioned with regard to doing this was to send this audio to some other service - perhaps a fixed-frequency receiver that might be used for decoding FT-8, WSPR or other types of digital signals - or it could be a much more wideband use, such as "hearing" a large chunk of a band for use with a CW or RTTY skimmer.  If a fairly wideband receiver is being used on a WebSDR, then it only makes sense that we could "re-use" its raw I/Q data in other applications.

Example:

Let's consider an example of an RSP1a tuned to 7125 kHz and operating with a 768 kHz sample rate.  As such, it "hears" all of the U.S. 40 meter amateur band (7000-7300 kHz) - and more - so it's a good candidate for receiving a specific segment of that band.  As it is, the "raw" 768 kHz I/Q data isn't terribly useful, but we can use the "csdr" tools written by HA7ILM and available on GitHub (Link is HERE) as well as from the more-current GIT maintained by DD5JFK (Link is HERE).

Comments:
We must first get our audio from the stream that we have created and one of the ways that this may be done is to use "arecord".  This utility is convenient in that it does everything that we need it to - including convert the sample type output by it to the floating-point format required by csdr - and we can invoke it thusly:

arecord -D f_loop0_out3 -c 2 -f float_le -r 768000

This invocation uses as its input device "f_loop0_out3" as its source, specifies that there are two audio channels (the "-c 2" part), that the output format is little-endian floating point (the "-f float_le" portion) and that the rate that we want is 768 kHz (e.g. "-r 768000).

A problem with "arecord" and why you may not be able to use it

While you might think "arecord" would work for this, it has a problem:  It will stop after output 2 GB of data.  Apparently, this is due to the 2 GB limit of .WAV files and even if you are in "stream" mode using STDIO, this will still cause - in our case, with 768 kHz and two channels with four bytes per sample (because we are converting to floating point using the "-f float_le" argument) - it to stop after 349.5 seconds.

If all you need are short recordings of just a few minutes each - for WSPR decoding or to do analysis, such as verifying that a receiver is working properly and you need just a short audio file - then this might work out for you.  If you need a continuous source of receiver data, having it stop after a while is a "show stopper".

Work-around using "fplay":


The work-around is to use the "fplay" utility discussed on this page:  "Creating "fplay", a version of the Linux "aplay" utility, but without the speed limit" - (Link) instead.  As it turns out, "arecord" and "aplay" are pretty much the same program so we can make "aplay" (and "fplay") act like "arecord"  with the addition of the -C argument.


Instead of the above, we simply do:

fplay -C -D f_loop0_out3 -c 2 -f float_le -r 768000

If you used the implementation in the "Duplicating receiver streams" article mentioned earlier (with the "plug-in") we can specify our rate in this line and ALSA will convert it for us.  For example, if we wished, instead, to use a 384 kHz bandwidth for our 40 meter receiver, we could have specified "-r 384000" instead and ALSA would have resampled and filtered it - doing a better job at anti-alias filtering than the SDRPlay driver can!  If you have enough processing power to do this, running the receiver at higher-than-needed bandwidth and allowing ALSA to resample is recommended.

At this point "fplay" is spitting our I/Q receiver data in floating point format out to STDOUT for "csdr" to catch - but we could have configured it to spit out 16 bit signed, little-endian data by using "-f s16_le" instead - see the comment below about using csdr itself to convert formats.

Let's configure this to capture the 40 meter FT-8 segment found at 7074 kHz, using USB for reception:

Let's add to the above this massively-long line command line:

csdr shift_addition_cc 0.06640625 | csdr fir_decimate_cc 16 0.015625  | csdr bandpass_fir_fft_cc 0.004 0.05 0.005 | csdr realpart_cf | csdr agc_ff --profile slow | csdr limit_ff csdr | convert_f_s16

As mentioned above, we are using the "-f float_le" argument in "fplay" to convert it to a format that csdr wants, but if we already have a 16 bit signed little-endian stream we could have preceded the above with "csdr convert_s16_f" to do the same job.

Taking these statements one-at-a-time:
At this point its worth noting that our audio is mono:  Many applications will be happy with that - but let's take this one more step and cause our receiver audio to be piped to the default audio device using "aplay":

csdr shift_addition_cc 0.06640625 | csdr fir_decimate_cc 16 0.015625 HAMMING | csdr bandpass_fir_fft_cc 0.004 0.05 0.005 | csdr realpart_cf | csdr agc_ff --profile slow | csdr limit_ff | csdr convert_f_s16
 | csdr mono2stereo_s16 | aplay -r 48000 -c 2 -f s16_le &

In the above we recognize the individual steps we described above - but have added two more so that we can hear it on a speaker connected to our default audio device::
The final version of the command line turns out to be:

fplay -C -D f_loop0_out3 -c 2 -f float_le -r 768000 | csdr shift_addition_cc 0.06640625 | csdr fir_decimate_cc 16 0.015625 HAMMING | csdr bandpass_fir_fft_cc 0.004 0.05 0.005 | csdr realpart_cf | csdr agc_ff --profile slow | csdr limit_ff | csdr convert_f_s16 | csdr mono2stereo_s16 | aplay -r 48000 -c 2 -f s16_le &


With the "&" at the end of the command line, this function will run "forever" but it may be stopped by doing "killall csdr", which will also stop any related "fplay", "aplay" and "arecord" instances.

Configuration for a CW skimmer:

A "CW Skimmer" (see this web page:  http://www.dxatlas.com/CwSkimmer/ ) is software that will inhale a large chunk of an amateur band and decode all CW signals within that passband and optionally log and report them.  With the hardware above, it's possible to analyze the majority of the 40 meter CW band, so let's construct a hypothetical configuration, starting with our requirements:
Because half of the 92 kHz is 46 kHz, we'll put the center of our "new" virtual receiver at 7046, allowing it to cover from 7000 to at least 7092 kHz, so we'll first calculate the "shift" value:
With our frequency shifted, we could, in theory, convert directly down to 96 kHz, but since we want 92 kHz of usable bandwidth, a filter that will remove signals beyond +/- 46 kHz from the center implemented at a sample rate of 768 kHz will be quite "expensive" in terms of CPU resources, so let's get to 192 kHz first - a decimation of 4 - and do filter there where it's likely less costly.  Since the available bandwidth at 192 kHz is +/- 96 kHz - and we want +/- 46 kHz (for a total of 92 kHz bandwidth) we want 48% of that - so we'll go with a transition bandwidth value of 0.24, according to the formula above - and this should sufficiently remove any aliasing in the 768 to 192 kHz conversion.  As mentioned earlier, we can gain a bit of performance with little CPU penalty by specifying a window filter, so we'll use "HAMMING".
At 192 kHz, we can now do better filtering we we can apply band-pass filtering as follows using "bandpass_fir_fft_cc":
We are still at 192 kHz, but since we have done filtering that should prevent aliasing, we can now convert from 192 to 96 kHz while applying minimal additional anti-alias filtering, so we'll do that now:
We are finally at 96 kHz and since the CW Skimmer can accept I/Q audio, we won't do anything else other than apply an AGC and a clipper as follows:
The final piece is the conversion from floating-point to signed-16 bit (little-endian) which is done with the statement:  csdr convert f_s16

Again, since we need the I/Q we will not get rid of the "imaginary" data which means we have two audio channels - which also means that we don't need to do the mono-to-stereo conversion, either.



Example:  Piping converted audio from a Linux computer to VLC

If we wish, we can then take the 48 kHz audio stream in our first example (the one prior to the CW Skimmer) and make it available to another computer - Linux or Windows - on the network as follows:

fplay -C -D f_loop0_out3 -c 2 -f float_le -r 768000 | csdr shift_addition_cc 0.06640625 | csdr fir_decimate_cc 16 0.015625 HAMMING | csdr bandpass_fir_fft_cc 0.004 0.05 0.005 | csdr realpart_cf | csdr agc_ff | csdr convert_f_s16 | csdr mono2stereo_s16 | nc -lk4 -p <port>

Where <port> is the port number on the computer running the above to which you connect.  The above uses "netcat" (nc) to stream the raw audio data (from STDIO) via a TCP connection, if a remote device makes that connection.

You can test this by pointing your browser to the address and port of your audio stream:  If the IP address of the computer streaming the audio is 192.168.0.32 and you picked "12345" as the port number, if you navigate to "192.168.0.32:12345" you should get a screen full of garbage correlating to the raw audio data - and you'll likely see the data sent by the browser to the audio server on that screen.

As practical example, if you wish to connect to the above stream using VLC - perhaps on your Windows PC - we would do the following:
From the command line the arguments would be:

<path>vlc --intf dummy --volume=100 --demux=rawaud --rawaud-channels=2 --rawaud-samplerate=48000 --rawaud-fourcc=s16l tcp://<ip_addr>:<port> vlc://quit

Where:


Additional comments:

Selection of intermediate sample rates

If you have been following along closely, you'll note that the 768 kHz - or even 384, 192 or 96 kHz - sample rates integer-divide nicely into 48 kHz.  If, for some reason, you were to need a rate like 44.1 kHz, you would need to readjust the source sample rate - but there's a problem here:  When using the ALSA input, the only valid sample rates for the PA3FWM WebSDR server appear to be 24, 48, 96, 192, 384 and 768 kHz, so the use of a sample rate that doesn't divide neatly in 24 or 48 can't be used with "fir_decimate_cc".

There is another function - "fractional_decimator_ff" that, as the name implies, will take fractional decimation rates, but this involves a bit more CPU power - and a bit of care must be taken when doing this as unlike integer rates, fractional rates are not necessarily exact meaning that you may end up with minor sample rate errors that could cause buffer under/overrun issues.

This also brings up another point:  In the example above, we decimated from 768 kHz down to 48 kHz in one step - but if we need only 2.4 kHz of bandwidth why didn't we go all of the way down to, say, 6 kHz?  To do this would require a very filter prior to the decimation step as we would need to make sure that signals more than, say, 2.5 kHz away from "zero" would need to be significantly attenuated - perhaps by more than 60 dB.  While theoretically possible, this would likely require more processing power than decimating to a higher frequency like 48 kHz first where filtering is less critical.

Practically speaking, there's no reason why multiple decimation steps couldn't be done - say from 768 to 192 kHz, and then again from 192 to 48 kHz:  A bit of testing would need to be done to verify the efficacy of the filtering (e.g. suitably-reduced aliasing artifacts at each step) and to determine which method (one or two steps of decimation) minimizes CPU requirements.

Another factor affected by the "final" sample rate (e.g. after all decimation is done) is that of the filtering:  A higher sample rate requires sharper filters (and more CPU power) to achieve a given "shape factor" (e.g. steepness of a filter at its edges) so a lower sample rate may be beneficial in that regard - but this is a trade-off as getting to a lower sample rate with more decimation steps may incur a CPU usage penalty higher than such filtering.

How much CPU does it use:

Since we are moving the audio data unmodified, it uses negligible CPU power (each instance shows up as "0.0% utilization) and minuscule RAM:  Certainly, anything that you plan to do with this data - even shoveling out out the LAN port - is likely to take more processing power than replicating this data!


Conclusion:


In the examples above we show how a higher-bandwidth stream can be converted to a lower-bandwidth stream for use with other applications using CSDR, leveraging an already-existing receiver system for other uses.  We also demonstrate how this audio data can be exported across a network (probably a LAN) for use by other servers for additional processing, such as CW Skimming, WSPR and FT-4/8 reception, etc.

It's likely that there are better ways to do the above tasks and if you know of methods that are "lighter weight" in terms of resource utilization, please let me know using the contact information below.




Additional information:
 Back to the Northern Utah WebSDR landing page