|
Northern
Utah WebSDR
Getting the raw I/Q data from the receiver
|
What's this about?
In the other pages related to higher-bandwidth (>192 kHz) 16 bit receivers on the PA3FWM WebSDR software it was (probably)mentioned
somewhere that getting the raw I/Q data from the receiver -
in our case, the SDRPlay RSP1a - but the RSP2 and RSP2pro has also bee found to work. When an audio sound card and a
"softrock" type receiver are used, getting raw I/Q data into the WebSDR
is simply a matter of specifying the sound card's hardware ID, but what
if you are not using a sound card + softrock?
Recently, USB-interfaced Software Defined Radio tuners have become much
more affordable - one of these devices being the SDRPlay RSP1a which,
at the time of this writing, is available for approximately US$130:
Other, similar devices are available from other suppliers that
could be made to work, but we'll focus largely on the RSP1a in this
article.
Getting Raw I/Q data via "rsp_tcp":
There are a number of different software packages available for the
RSP1a - including those using the Soapy interface - the closest thing
to a "universal" SDR package - but this package isn't particularly
useful to us: It doesn't natively support high-bandwidth (e.g. greater than audio bandwidth)
raw I/Q data and it adds another layer of software and complexity - and
likely delay - that is unnecessary and undesired as what we really want is to simply pipe the raw I/Q data to be sent into the ALSA sound system, anyway.
There is a bit of software provided by the SDRPlay folks that looks as though it might be useful - namely the "rsp_tcp" package found on Github (link to that package is HERE) that allows raw I/Q data to be sent via a UDP on a network connection (which could be a loopback or even a "real" socket over a LAN) but it turns out that as it presented, this package is not useful
for us in this context for one simple reason: Critical settings
for LNA Gain and attenuation are fixed in the source code, and these
defaults are not at all suitable for any purpose whatsoever.
Simply put, the version linked above has the "LNAState" parameter
set to "4" which put so much attenuation in front of the LNA that it's
sensitivity on any frequency - even the lower HF bands - is uselessly
poor, and for some (insane!) reason
there are no means of configuring it from the command line. The
other issue with this code is that it has only an 8-bit pipeline
meaning that one key of the RSP1a over an RTL-SDR dongle is lost.
To be fair, it is intended that the gain and other parameters be
controlled via the TCP link, but this isn't useful to us if all we want
is to pipe the I/Q data directly into ALSA where no such control exists.
A more useful fork of the package may be found here.
This version, produced in part by ON5HB, addresses some of the
shortcomings of the original "rsp_tcp" in that the gain and LNA
parameters may be specified - and it even has a more user-friendly
means of configuring the AGC in the SDRPlay API, but its bit depth is
still limited to 8 bits - although a previous version did have the ability to specify the use of the full 16 bit data path from the API.
Neither of these programs will get the raw I/Q data into ALSA, but one may use the Linux netcat utility to intercept the TCP stream and pipe it into ALSA via the Linux "aplay" utility. For example, consider this:
rsp_tcp -a 127.0.0.1 -p 1234 -f 5000000 -s 192000 -b 16 &
This line, assuming the use of a version of the "rsp_tcp" program that does support the full 16 bit data path (you'll have to find this version yourself - but it is given as an example, above)
will configure a receiver with a bandwidth of 192 kHz centered at 5 MHz
and send it out via the "loopback" interface on port 1234.
Before we can send this to ALSA, we need to set up an loopback device, so we can do: sudo modprobe snd_aloop
Now, we set up a "receiver" for the I/Q stream being sent via TCP by doing:
nc 127.0.0.1 1234 | aplay -D <sound_device_name> -r 192000 -c 2 -f s16_le
Which will take the TCP data and send it to the sound device of our choice. If we do aplay -l
on the command line after starting "snd_aloop" as shown above, we'll
see a device called "Loopback" in the list, and we'd note which "card"
number this was. If, for example, our loopback device was listed
as card #2, we would put in our "aplay" statement the device name of:
hw:2,1,0 - this is our "input" device on our loopback interface
In "websdr.cfg" we would use hw:2,1,1 as the source
of the I/Q audio as our sound card. Note that the "swapiq"
directive may be required in "websdr.cfg" if the resulting spectrum was
inverted.
Again, for the above to be successful, the version of "rsp_tcp" required to work as described above would need to include:
- The ability to send the raw I/Q data as 16 bit data.
Some versions support a "-b" command-line option as depicted
above - but most do not.
- The ability to configure the "LNAState" variable of the API to set the LNA gain.
- The ability to set the "gain reduction" value.
The above configuration was tested and found to work
satisfactorily, but there were several issues, but the most important
was that the latency, as compared to the "sound card +
softrock" approach, was significant - around 750 milliseconds.
This seemed to be "baked in" the TCP + netcat approach and
couldn't easily be reduced. The other issue was that the AGC
built into the SDRPlay API didn't seem to be well-suited for
multi-user, wideband operation - and it was difficult to configure in a
manner that was easily understood.
A discussion about the above may be found in THIS thread on the "websdr" group at groups.io.
The "direct" approach:
After a few iterations, a more "direct" approach was finally
taken: The "rsp_tcp" code base was scrapped entirely and a
stripped-down, minimalist utility called "sdrplayalsa"
was written with the sole purpose of facilitating the shoveling of raw
I/Q data from the SDRPLay API into ALSA. This package may be
found on Github here.
This package is a collaborative effort, the majority of it having
been written by Gary W., AB1IP with additional work done by those noted
in the source code. A binary compiled for 64 bit x86 that is
known to run on Ubutntu 18.x and 20.x may be downloaded here - but, of course, using this binary would require a bit of trust on your part which is why the source is available!
While this program is capable of putting this data directly into an
ALSA loopback device, this has been a bit more difficult to manage
owing to inconsistent buffering, causing audio drop-outs due to
over/under running the audio buffer - but it also supports being able
to stream the raw I/Q data directly vis the Linux stdout interface - that is, having the program send raw data directly into aplay (as depicted above) which then feeds it into an ALSA loopback device.
A few comments:
- The code was originally tested with the RSP1a as it is the best fit for WebSDR use in terms of price and performance.
- The "sdrplayalsa" code has since been tested and found to work with the RPS2 and RSP2pro. (We have not tried it with the RSPdx or RSPduo - mainly becase we don't have any.)
- For best performance, always precede any receiver with a band-specific band-pass filter to reduce the probability of front-end overload.
The source code is available above, so if you wish to tweak it for
a wider variety of devices, feel free to do so - but if you do, please
let us know so that we can make that information available to others.
Version changes for "sdrplayalsa":
- 20220223: Fixed a problem where the decimation/rate
multiplier values weren't being selected for rates other than 768 ksps.
- 20220304: Added the "-R" parameter where "custom" sample rates may be selected within the constraints of the hardware - see below.
- 20230214: Fixed issues with direct-to-ALSA pipelining so
the use of STDOUT to "aplay" or "fplay" is optional if one wishes to
output to the audio system (e.g. ALSA); Added "-L" parameter to
set latency (buffering) when using the "-o" parameter to send to an
audio device; Added code to do a more graceful shutdown/exit from
the API; Reworked the way the AGC data is conveyed to the API and
how the current gain reduction setting (e.g. "RF AGC")
is written to a file; Added more detailed configuration/status
information and cleaned up the UI and command-line interface.
The above changes are included in the .GIT and the binary linked above.
Using sdrplayalsa:
The sdrplayalsa binary,
once produced, is copied into a usable directory and set to be
executable by the WebSDR owner. On the Northern Utah WebSDR, it
is placed in the ~/cfg directly where it be called in a configuration script that will call "config_rsp_devices.sh", and this script is called prior to starting the WebSDR server itself.
Invoking sdrplayalsa -h will give a list of available options - and these options are discussed in detail, below.
Configuring the receiver tuning and sample rate:
-f <freq> - "freq" is the center (tuner) frequency in Hz.
-r
<sr> - "sr" is the sample rate, in SPS, of the device. For the PA3FWM WebSDR
software, this must be 96000, 192000, 384000 or 768000, but values of 96000 or 192000 are not recommended due to aliasing/filtering issues. See the discussion below in the section "Minimization of Images" if you wish to operate at 96000 or 192000.
-R <exp> - This parameter is used to determine the decimation rate given the current sample rate: Usable values are 0-5. This parameter is discussed in more detail, below.
Additional receiver configuration parameters:
-d - This parameter, used by itself, will show the currently UNUSED
devices and their serial numbers. If a device is currently in use (e.g. configured and being used for receiving)
will NOT show up on this list. This is useful for determining the
serial number of the connected device so that it may be be assigned for
a particular frequency - see "-i", below.
-h - Show usage (e.g. "help")
-i <xxxx> - "xxx" is the serial number - partial or full
- of the target device for this configuration. This is very
useful when multiple receivers are used that may be preceded with
band-specified pass filters - which are highly recommended!
Typically, one uses the last 4-6 characters (5-6 is recommended) of the unit's
serial number printed on the RSP itself to uniquely specify a device.
-B <bw> - "bw" is
the receiver's baseband filter selection. This filter, applied
post-converter within the SDRPlay itself, is applied in the signal path
at the input of the A/D converter, the value specifying the -3dB
bandwidth. Valid values, in kHz, are: 200, 300, 600, and
1536 kHz. It is recommended that 200 or 300 be used at a sample
rate of 384 kHz while 600 is reasonable for a sample rate of 768 kHz.
-l <lnastate> - "lnastate" sets the amount of "gain reduction" related to the LNA in the receiver. The "LNAstate" setting is very critical for usable receiver performance and it will be discussed at length below.
For the RSP1a: 0 = 0dB, 1 = 6dB, 2 = 12dB, 3 = 18dB, 4 = 37dB, 5 = 42dB, 6 = 61dB. NOTE: A value of "4" or higher will likely result in very poor receiver sensitivity and should probably never be used.
For the RSP2 and RSP2pro:
0=0dB, 1=10dB, 2=15dB, 3=21dB, 4=24dB, 5=34dB, 6=39dB, 7=45dB,
8=64dB. NOTE: A value of "5" or higher will likely result
in very poor receiver sensitivity and should never be used.
As of the time of writing, neither the RSPduo or RSPdx
have been tested. It is expected that the RSPdx will also work,
but it, too, different amounts of gain reduction for given "LNAstate"
values.
-L <latency in uSec> - This parameter, which is used only with the "-o" parameter, below, sets the amount of latency (buffering) to be implemented in the ALSA device's pipeline. The minimum allowed value is 35000 (35 msec) and the default is 50000. Depending on the computer and loading, the minimum value is likely to be 50000 to avoid buffer underruns (which manifest as brief drop-outs). A value of 75000-100000 (75-100 msec) is suggested.
-o
<dev> - "dev" is the output device, which could be an ALSA
device: Use with the "-L" parameter, above. If unspecified, STDOUT will be used. As noted above, STDOUT is typically used in conjunction with "aplay".
-v - If specified, this will enable verbose output, used for troubleshooting.
-w <msec> - This specifies, in milliseconds, how often verbose output will be sent to the console if "-v" is used.
-W
- If specified, this will enable "wideband signal mode" which will
enforce stronger filtering, but this can dramatically increase CPU
usage. "Wideband signal mode" enables additional DSP bandpass
filtering which can increase the amount of usable bandwidth while
reducing aliasing at the extreme edges of the receiver response. (This parameter may not work properly at this time with all configuration.)
AGC configuration:
Even though the SDRPlay API includes its own AGC, testing showed
that this AGC didn't seem to be appropriate for multi-user
wide-bandwidth used - but the main problem is that its precise
operation and configuration is not at all well documented, making it
difficult to set up properly. Because of this, an AGC algorithm
was included in the sdrplayalsa program that may be well-understood and characterized.
AGC operation:
The purpose of AGC in any
receiver is to adjust the signal path to accommodate the current signal
conditions - and this is very important with an SDR receiver to keep
the input signal range within the capabilities of the A/D converter:
Too little signal into the A/D converter, quantization distortion
will occur causing the generation of spurious signals, while too much
signal will effectively "waste" A/D range, increasing the likelihood of
clipping under high-signal conditions. With a wideband receiver,
knowing where to set the signal level is more difficult than with a
single-user narrowband receiver as both weak and strong signals must be
accommodate simultaneously.
With a wideband receiver, the total signal power
is what must be carefully regulated - but at wider bandwidths, this is
arguably easier as the aggregate of disparate signals is more akin to
random noise, the average level changing fairly slowly over time - but
there is always the probability that occasional A/D clipping will
happen with impulse noise such as lightning static. If the
percentage of clipped samples is low, the amount of signal degradation
across the receiver bandwidth will be negligible - particularly from
the perspective of the individual users who will be using virtual
receivers operating at narrow bandwidth (e.g. around 3 kHz).
Any AGC system has a threshold above which AGC action (decrease of gain to prevent overload)
will occur: Signals above this threshold will cause the gain to
be reduced to prevent overload/clipping of later stages and it this
case, this level is specified as an A/D input value, and because the
I/Q signal being sourced by the API is a 16 bit signed number, it can
range from about -32768 to 32767, so our threshold would be expressed
as the absolute value of the input signal. Knowing the precise
threshold level to set to cause AGC action is going to be a matter of
observation and experimentation over time.
Unlike a typical receiver's AGC, there is also a threshold below which the AGC will increase gain again and the reason for this is that when looking at a wide frequency swath, the total
power level is going to change far less than that of the individual
signals, so we would want to "ride" the average level over time - which
also means that once we decrease gain, the best strategy would be to
keep in there until the average signal level had dropped down again for
a reasonable amount of time. In other words, the AGC should be
reasonably quick on its "attack" to reduce gain, but be rather slow to
increase it again - just like a typical AGC.
Configuring the AGC:
The AGC configuration in sdrplayalsa is configured using the following parameters:
-n - Including this parameter will enable the AGC in the driver,
configured with parameters a, b, c, e, g, G, s, S, x, y, z below.
-a <high> -
"high" is the A/D value above which a "high signal" condition is
declared. The
A/D value to which it is compared is the absolute of the 16 bit signed
value from the API in which full scale is 32768 and this parameter
would be set no higher than one-half of this value. The default is 16384.
-b <low> -
"low" is the A/D value below which a "low signal" condition is
declared. A typical value would be one quarter of the value of
"-a". The default is 8192.
-c <msec> - This
specifies the period, in milliseconds, of the sampling window for AGC
operation and during this period, the minimum and maximum signal levels
are monitored. It should be remembered that the total number of
samples analyzed is the ratio between the sample rate and this period.
For example, if the period is 100 msec (1/10th of a second) and the sample rate is 768000, 76800 samples will occur during this period.
-e<file> -
If this parameter is specified, a text file of the name "file"
will be written with the gain value, relative to the
"-g" parameter, indicating the amount of gain reduction.
This file is
updated ONLY AND IF the gain changes. If "-g" is set to
30, an internal gain reduction value of 40 will result in a value
of 10, indicating 10 dB of gain reduction. This information may
be made available to the WebSDR's clients (e.g. javascript running on the users' computers)
to provide a correction factor of the S-meter to compensate for the
gain reduction. When "sdrplayalsa" is invoked, the file is
created (if it doesn't exist) and a value of "0" is written into it.
-g <min> -
"min" is the MINIMUM amount of gain reduction in dB and may be in the
range of 20-59. When AGC operation is NOT enabled, this sets the fixed amount of gain reduction and when the AGC IS
enabled, this sets the amount of gain reduction. Internally, this
value is subtracted from the actual gain setting when the "-e"
parameter is used to indicate the amount of gain reduction with respect
to the minimum amount of gain reduction (e.g. no gain reduction = 0). The minimum value is 20.
-G <max> - "max" is the MAXIMUM amount of gain reduction: The value may be 20-59, the default is 59.
-s <dec> - "dec" is the step size in dB when the gain reduction value is reduced by the AGC.
-S <inc> - "inc" is the step size in dB when the gain reduction value is increased by the AGC.
-x
<samp> - "samp" is the number of samples that are above the "high signal"
value set by "-a" that must occur during the period specified by "-c" in order to cause a gain reduction of the number of dB specified by "-S". This
parameter is used to prevent gain reduction on too-few A/D samples (e.g. extremely brief impulse noise) and prevent over-active gain reduction operation.
-y
<inc> - "inc" is the MINIMUM time, in milliseconds, allowed between gain
reductions. This value sets the timer to prevent too many gain reduction operations in too-little time.
-z<dec> -
"dec" is the MINIMUM time, in milliseconds, allowed between gain
increases. This values sets the timer to prevent the
gain from being increased too quickly when a high-signal condition
subsides.
"Custom" sample rates using the "-R" parameter:
There is another parameter, "-R", that is used to specify the
decimation of the sample rate. If this parameter is invoked with
a number from 0-5, the "-r" parameter will allow any sample rate (within reason) to be entered. The program will determine if the resulting rate is within the parameters (described below) and if not, it will bail out.
The RSP's A/D converter internally runs at a higher rate than you might thin:
The actual A/D converter in the RSP1a is capable of operating down to
2.048 Msps, and with "14 bits" of sampling, can operate thusly below
8.064 Msps. In order to get rates lower
than 2.048 Msps, "decimation" must occur, which is the act of throwing
away a bunch of the samples, leaving us with fewer to deal with and a
much lower bit rate - but in doing so, spectrum of the data prior to decimation must be filtered such that the energy above the Nyquist limit is removed or else aliasing will occur.
As it turns out, the RSP1a's hardware and software can decimate by
powers of 2, with values from 1 through 32 - which corresponds to two
raised to the "n" power, as in: 20=1, 21=2, 22=4, 23=8, 24=16 and 25=32.
By default, sdrplayalsa utilizes only the rates of 96, 192, 384 and 768
kHz, which utilize a decimation of 32, 16, 8 and 4, respectively
starting from a sample rate of 3.072 Msps: In other words, no matter which of the above sample rates you chose, it is derived from the A/D converter operating at 3.072 Msps.
Why might you need odd sample rates?
In theory, any sample rate is thus possible as long as it is below 8.094 Msps (with a decimation of 1) and 64 ksps (2.048 Msps with a decimation of 32),
allowing "custom" rates to be specified. Again, it's worth noting
that the current version of the WebSDR software won't work properly
with all rates as noted above - but the built-in resampling of ALSA
will allow the input of about any rate up through 768 ksps and convert
it to the rate requested by the WebSDR program.
For example, if a WebSDR sample rate of 384 kHz is desired, one wish to
configure the RSP1a with a rate of 425 ksps and let ALSA convert it it
a lower rate. This might be
desirable if ALSA's low-pass filtering and downsampling is superior to
that of the SDRPlay driver - or you may even wish to pipe the raw data
through another utility to do the downsampling and filtering (e.g. SOX, CSDR).
Using this parameter to change CPU loading and/or quality:
Taking 768 ksps as an example, the default configuration by
sdrplayalsa is to use a decimation of 4, meaning that the A/D converter
is running at 3.072 Msps. According to Nyquist theory, if we
oversample, we can "create" additional lower-order bits, so
oversampling by 4 implies that we have created (sqrt(4) ) 2 more bits
of data from whatever we had and improved the useful dynamics of the
A/D converter at the "low signal" end of the range by 12 dB. This
would imply that if we run the A/D converter even higher, we could gain even more bits - and that means that if we specify a rate of 768 ksps with a decimation of 8 (e.g. -R 3, where 23 = 8) the A/D converter is now running at 6.144 Msps. In theory, we now have ( sqrt(8) ) = 2.8 additional bits (about 17dB) of additional useful (low signal) A/D range. In theory, this amount is likely to be noise-limited by the A/D converter itself and the other circuitry (e.g. amplifiers, thermal noise from other components, etc.) and is likely to be less than the idea.
Even if performance is improved in this manner, the penalty is higher
CPU usage - likely to be between 5 and 15% on an X86 3 GHz core -
although the amount may vary.
Conversely, one could also use this parameter to reduce CPU usage.
While the math above shows that you can't use lower than 3.072
Msps to get 768 ksps given the limitations of the decimation values (e.g. we have only 1, 2, 4, 8, 16 and 32)
lower rates may benefit. For example, for a 192 kHz receiver
configuration it may be possible to acheive overall lower CPU
utilization if a rate of 256 ksps were specified with a -R paramter of
3 (e.g. 23=decimation value of 8), which would configure the A/D converter to its lowest rate (2.048 Msps) and have ALSA down-sample to 192 kHz.
Discussion of AGC parameters:
AGC Timing and thresholds:
The
"-c" parameter sets the time sample ("window")
period of the AGC algorithm. If
set to "100" (100msec) the AGC window will have looked at the
number of samples that occurred during that time (e.g. 19200 for
192kHz, 76800 for 768 kHz, etc.) With this in mind, the value of
of "-x" must be chosen appropriately. Because "-x" is the number of samples during the period that must exceed the value of "-a" (high signal threshold) in order to trigger a gain reduction, that means that if the sample rate is 768 kHz (76800 samples during the period) and "-x"
is set at 1000, that means that (1000/76800) = 1.3% of the samples
must have exceeded this value. This value should be scaled
accordingly for different sample rates and AGC window periods.
Care should be taken in choosing these parameters as it isn't desirable
to have the AGC action occur too quickly at too-low a threshold
or too quickly. Of course, a gain reduction in response to
high-signal
conditions should occur more rapidly than a gain reduction so the
suggested "-S" parameter is 3 dB and the "-y" parameter is 200 msec,
meaning that the gain can be reduced (AGC "attack") by 3dB every
200 milliseconds. For a gain increase, the parameter of "-s" at 1 dB
and "-z" of 5000 msec means that the gain will be increased (e.g.
AGC decay) by 1 dB every 5 seconds at the fastest, provided that not a
single sample exceeded the value set by "-b".
A typical value
of "-a" at 16384 represents half-scale of the A/D converter - about 6
dB below clipping. Experience has shown that while this will not
prevent ALL A/D clipping events, a small fraction of clipped samples (a
few samples out of every thousand) are unlikely to be noticed in an
acquisition channel which is effectively oversampled compared to the
typical receiver bandwidth (e.g. a 3 kHz SSB bandwidth on a
receiver sampling at 768 kHz). The threshold for the AGC decay (e.g.
increase in gain) using the "-b" parameter is typically 4096 - a
value that is about 1/8th of A/D full-scale and only if NO A/D samples above this value occur
during the window set by "-c" will the gain be increased again.
LNA and gain settings:
The "-g" parameter is used to specify
the MINIMUM amount of gain reduction (lowest value is 20). Typically,
this will be set, along with the "-l" parameter, so that under "no
signal" conditions, a reasonable A/D value (around 500) will be
obtained so that weak signals - and the antenna system noise floor
- may be just detected. A setting of "-l" of 4 or greater is NOT likely to be usable for any receiver configuration! It has been observed that at least for HF, a "-g" parameter of lower than 30 will NOT
improve sensitivity, but simply raise the noise floor and increase the
probability of overload. Practically speaking, this won't affect
performance because of AGC action, but it is preferrable to specify the
minimum gain necessary to comfortably hear the noise floor of the
receive system - with the noise floor increasing by about 6 dB when one
switches the antenna input of the receiver from a 50 ohm load to the
antenna system under "dead band" conditions: More gain than this
will NOT improve performance! Remember: The "-g" value specifies a "gain reduction", so a higher number means less gain.
A warning is appropriate here as the "-l"
parameter sets the amount of attenuation IN FRONT of the LNA and
this sets the system noise figure, and "-l" should NOT be set too high
to prevent "seeing" the antenna system noise floor under no-signal
conditions! Typically, these parameters are set - WHEN THE BAND IS
DEAD AND THERE ARE NO SIGNALS - by setting the "-g" parameter to 20
and then increasing the "-l" parameter until a difference of about 6 dB
can NO LONGER be seen when connecting/ disconnecting the antenna - at which point the "-l" parameter is reduced again so that this 6 dB difference is again visible.
The "-g" parameter is then set to obtain the needed no-signal A/D value - which should be in the 200-500 range. Too-low
a "-g" value will result in too-easily triggered AGC action, but a
too-high a "-g" value will limit that available operational AGC range (e.g. setting "-g" to 50 means that only as much as 9 dB of AGC can occur as the highest value is 59.)
It
is only after the "-l" and "-g" parameters are established that the
"gain" value in "websdr.cfg" is set to provide S-meter calibration.
Example configuration using "sdrplayalsa" with output directly into ALSA:
The following shows the use of "sdrplayalsa" output directly into ALSA using the "-o" parameter.
( ./sdrplayalsa -i 4798 -r 768000 -f 14150000 -n -B 600 -g 37 -l 3
-a 16384 -b 4096 -c 100 -x 1000 -y 200 -z 5000 -s 1 -S 3 -e
~/pub/g1.txt -o plug:f_loop0_in -L 65000 ) < /dev/null 2>> $logfile
&
Breaking this down:
- -i 4798 This
indicates that the receiver with a serial number ending in "4798" is to
be used for this configuration. This is done so that we can be
sure that the receiver is matched to the band-specific filter on the RF
input in the event that we have multiple
receivers. This may represent either the entire serial number or
a portion of it: The use of the last 4 or 5 characters of the
serial number should suffice.
- -r 768000 Sample rate of 768 kHz. Valid values are 768000, 384000, 192000 and 96000 samples/sec.
- -f 14150000 Center frequency of 14.150 MHz.
- -n AGC is enabled. Note that this is NOT the AGC built into the RSP API, but a custom AGC written into "sdrplayalsa".
- -B 600 A -3dB
bandwidth of 600 kHz is specified. Valid bandwidth values are
1536, 600, 300 and 200 kHz. A value of 600 kHz is recommended for
a sample rate of 768 kHz and 300 for 384 kHz. As noted below, a
value of "200" is a bit wide to be used directly at 192 kHz without the
appearances of images.
- -g 37 A minimum gain reduction of 37dB is set. What this means is that when the "gain file" (produced by the "-e" paramter)contains
zero, the RSP's gain is set to 37 dB: A value of "1" in
the gain file would indicate a a setting of 38 dB, and so-on, up to the
maximum value allowed by the RSP. Note that a "gain reduction"
value of below 30 for the RSP1a has limited utility as the main
contribution of noise to the signal path below this value appears to be
from the LNA - see the article "Observations on the SDRPlay RSP1a when used at HF frequencies" - link.
- -l 3 LNA
setting of 3. In general, the most useful values at HF are 1, 2
or 3 with the lower values being more appropriate for the higher bands (20-10 meters). The the aformentioned "Observations on the SDRPlay RSP1a when used at HF frequencies" article for more information.
- -a 16384 The
AGC "attack" threshold is 16384 in A/D units where full-scale is 32768.
In other words, if the maximum "-a" value during an AGC "window"
period is 16384, a digitized value above this is considered to be a
"high" signal level above this threshold. If the number of samples that exceed this value is greater than that specified in parameter "-x" (see below) then the AGC may decrease the gain to reduce signal level if the time specified in the "-y" parameter has elapsed since the last gain decrease.
- -b 4096 The
AGC "decay" threshold is 4096 in A/D units. If the there are no
A/D values below this value during and AGC "window" period, this is
considered to be a "low" signal level. If this is the case, the
gain may be decreased if the time specified in the "-z" parameter has
elapsed since the last decrease.
- -c 100 The AGC
"window" period is 100 milliseconds. The "window" is a block of
time during which the peak value of the samples is tracked to determine
if the the input signals exceed the "attack" threshold ("-a") or are below the "decay" threshold ("-b").
The total number of samples that occur during this window should
be calculated by multiplying the sample rate by this period, in
seconds. For example, 100 msec = 0.1 seconds, so if the sample
rate is 768000, we would get 76800 samples during this period.
- -x 1000 During the AGC "window" period, at least 1000 samples above the value of "-a" (16384) are required to trigger an AGC attack (gain decrease). This parameter prevents a small number of samples that exceed the "-a" threshold (e.g. a brief "click" or pop, energy from a lightning crash)
from causing an unnecessary AGC gain decrease. The number of
samples during the AGC window period should be considered when choosing
thus number. For example, at 768 kHz with a "-c" parameter of 100
msec (76800 samples during that window) a
threshold count of 1000 samples out of 76800 samples would represent
about 1.3% of all samples exceeding the "-a" value, but 1000 samples at
a sample rate of 192 kHz would mean 5.2% of all samples. Testing
has shown that setting this so that between 1% and 1.5% of the total
samples exceeding the "-a" parameter to cause a gain decrease is a
reasonable threshold.
- -y 200 A
minimum of 200 milliseconds between successive AGC gain decreases -
part of the "fast" attack of the AGC action where the gain is
decreased. This is to prevent a too-sudden and fast decrease in
gain reduction - perhaps caused by a static crash or a very strong
signal.
- -z 5000 A
minimum of 5000 milliseconds between successive AGC gain increases -
part of the "slow" decay of the AGC action where gain is increased.
This prevents too-fast an increase in gain after a strong signal
has appeared.
- -s 1 A gain
increase step size of 1 dB. This is part of the "slow" decay of
the AGC action. Setting a step size of just 1 dB along with the
"-z" parameter to allow a gain increase only as often as every 5
seconds prevent rapid fluctuations of AGC under normal band conditions.
- -S 3 A gain
decrease step size of 3 dB. This is part of the "fast" attack of
the AGC action. This allows "fairly" rapid gain decrease should a
strong signal appear, but it also prevents the gain from being
decreased too rapidly should the strong signal be from a noise
burst/static crash.
- -e ~/pub/g1.txt
This specifies that a file called "g1.txt" will be written
in the "pub" directory with the current gain setting, relative to the
"-g" parameter, when AGC action occurs. The number ("1", in this case) in the file name
is indicative of the WebSDR "band" that it represents (e.g. the WebSDR software has up to eight "bands" numbered 0-7, so band "1" would be the second band.) NOTE: This file MUST be located in the "pub" directory along with other public files so that it is accessible to users' browsers. The websdr's javascript code must be modified to use this value - see the section "Using the gain files", below.
- -o plug:f_loop_in This portion specifies the OUTPUT device as an ALSA (or possibly other sound system). In this case we are using a virtual audio device defined in ".asoundrc": More about how this is done may be found in the articles "Configuring high-rade audio loopback devices" - link and "Duplicating receiver I/Q data streams using ALSA and asoundrc" - link.
- -L 65000 This parameter specifies the amount of "Latency" (e.g. buffering)
to use in the output. The use of a buffer is necessary to prevent
audio drop-outs that might occur due to operating system pre-emption
and other "busy-ness" that might prevent the consistent flow of
data.
Example configuration using "sdrplayalsa" and sending raw I/Q data to "STDOUT":
In some cases you may
wish to use STDOUT rather than an ALSA device as the recipient of data.
One example why one might wish to do this would be to send the
raw I/Q data over TCP or UDP using "netcat" or similar program, send it
into another program for further processing (e.g. "csdr" for demodulation/filtering or into another SDR program). This may be done via the typical methods of using a "pipe".
In the example below we are using STDOUT to send the raw I/Q data to
the "fplay" utility - a version of "aplay" modified to operate through
at least 768 kHz - to pass audio through to ALSA.
( ./sdrplayalsa -i 4798 -r 768000 -f 14150000 -n -B 600 -g 37 -l 3
-a 16384 -b 4096 -c 100 -x 1000 -y 200 -z 5000 -s 1 -S 3 -e
~/pub/g1.txt | ./fplay -D plug:f_loop0_in -r 768000 -c 2 -f
s16_le --disable-softvol -B 40000 ) < /dev/null 2>> $logfile
&
For the "sdrplayalsa" portion:
Everything is the same as the example above except that the
"-o" and "-L" parameters weren't used. Because the "-o" parameter
was not used on sdrplayalsa, the raw I/Q output is sent via STDOUT, and we happen to use be using the fplay utility to send it on to ALSA as described below.
For the "fplay" portion:
The Linux "pipe" character ( | ) follows the invocation of sdrplayalsa meaning that the raw I/Q data is fed into fplay via STDOUT. "fplay" is a version of "aplay" modified to accept audio rates higher than 192 kHz and is described on the "Making fplay" page.
- -D plug:f_loop0_in:
This specifies that the output device is "f_loop0_in" which is a
virtual audio device using "snd-fastloop": which is a modified version
of "snd-aloop" capable of audio rates higher than 192 kHz and is
described on the "High Rate Loopback" page.
- -r 768000: A sample rate of 768 kHz
- -c 2: 2-channel audio
- -f s16_le: 16 bit, little-endian audio samples
- --disable_softvol: No volume adjustment is to be allowed
- -B 40000: A buffer of 40000 uSec is used
The remainder of the parameters pipe errors and diagnostic
information to the logfile as shown: "$logfile" may be specified
as an actual file for debugging or as "/dev/null" in normal operation
to prevent output from the programs from cluttering up the console.
The final ampersand ( & ) indicates that the two programs (sdrplayalsa and fplay) should run in background mode.
Which should I use - fplay or directly into ALSA?
If it works for you, the "direct to ALSA" approach using the "-o" (and "-L") parameters is easier as you won't need to compile/install the "fplay" utility - but they both seem to work equally well as long as the buffering is set sufficiently large to avoid brief audio drop-outs.
Testing has shown that a "-B" setting of 40000 when used with "fplay" (or aplay) has about the same amount of overall delay as a "Latency" (-L)
setting of about 60000 when sdrplayalsa is used with the "-o" parameter
to put data directly into ALSA - likely because the use of STDIO,
itself, adds a small amount of delay. Further testing has shown
that a "-B" buffer setting of 35000 used with "fplay" and a "-L"
setting of 60000 with direct ALSA output using the "-o" parameter are
both on the "ragged edge" of (mostly) avoiding audio drop-outs caused by underruns: A "-B" (for "fplay") setting of 50000 and a "-L" (using the "-o" parameter for direct-to-ALSA output) setting of 65000 are the minimum that you might want to consider to minimize drop-outs as system load changes.
Minimization of images:
Like any receiver, filtering is required to avoid image responses - and
the RSP series is no different. With the current version of the sdrplayalsa
driver, there "-w" option - which adds filtering in software - is not
yet supported so one must rely on the built-in hardware filtering
specified by the "-B"
parameter. For 768 kHz, the best fit is the 600 kHz filter, but
even this results in images within about 50 kHz of the top and bottom
edges of coverage. For the lower bands (160-12 meters)
the bands are much smaller than 768 kHz and images within the amateur
bands can be avoided by placing the amateur band in the center.
For smaller bands like 30, 17 and 12 meters, one could use a 192 kHz sample rate from the RSP1a and select 200 kHz as the "-B"
parameter - but this will result in very noticeable images within 30
kHz or so of the band edges. If one configures the receiver for 384 kHz (still using the 200 kHz filter) the ALSA system - by way of the looback - can resample and filter down to 192 with very good results.
The loopback system - which is defined in the ".asoundrc" file (see the discussion in the page "Configuring high-rate audio loopback devices" - link)
will operate at whatever rate the WebSDR is configured: If you
specify the bandwidth in the "websdr.cfg" file at 384 kHz, no matter what the rate is being input to the loopback device, the output will be resampled, as necessary - to 384 kHz.
In the process of resampling, ALSA does a good job of filtering
and images that would be present if you were to configure the RSP
itself to yield 192 kHz will be absent.
For example:
Suppose that you have a 30 meter
receiver and want to cover 192 kHz bandwidth. In the U.S., you
would be tempted to set the receiver's center frequency at 10080 kHz so
that you would include the 10 MHz WWV signal, but if this is done
images from very strong signals from the 31 Meter SWBC band may appear in the top of the 30 meter band coverage.
If the RSP is configured using "sdrplayalsa" for a 384 kHz rate (-r parameter) with the 200 kHz bandwidth filter (-b parameter) specified (to keep the signal input to the A/D converter from "seeing" signals outside the desired range)you
can configure the rate in "websdr.cfg" as 192 kHz, which will direct
ALSA to down-sample and filter the signal from 384 to 192 kHz - and do
a very good
job of removing images in the process!
This scheme cannot work to from a higher rate down to a 768 kHz rate (implying, perhaps, a rate of 1536 kHz or higher prior to resampling) as the ALSA system in current Linux
systems will not accept higher than 768 kHz without significant modification of the sound kernel. In theory, such down-sampling could be done in "sdrplayalsa", but this would be a future project.
Comments:
The above statement is not
strictly true if you are willing to use other utililties to process the
raw I/Q data. As described above, sdrplayalsa doesn't need to output directly to the ALSA sound system, but it can also output via STDIO and then to ALSA via "fplay". Because the pipeline between sdrplayalsa and "fplay" does not involve ALSA, and uses STDIO/pipelining there's no obvious sample rate limit so one could, instead, use a utility like csdr to filter and decimate a higher-rate stream down to 768 kHz to produce the desired anti-alias filtering - at the expense of CPU utilization.
For more information about the use of csdr see the following page: Using "csdr" for auxiliary receiver streams on a WebSDR system - link. While this page describes the use of csdr to processes "auxiliary" streams the same techniques could apply to the pipelined output of sdrplayalsa before it is applied to the ALSA sound system.
If one does start at a higher bandwidth - particularly using a higher value of the -B (bandwidth) parameter than one might otherwise choose - then the A/D converter (and thus the AGC) will then be able to "see" signals above and below the bandwidth that is ultimately set by csdr (e.g. not visibile to the user). These "extra" signals will
affect the AGC - and, of course, the wider the bandwidth, the less
"energy per bit per Hertz" you are allocating to the raw I/Q data
meaning that there is the possibility that ultimate receiver
performance will degrade as the bandwidth is increased and more signal
energy is present at the input of the A/D converter.
The use of csdr also provides for the possibility
of covering more bands with a single SDRPlay receiver: If, for
example, the sample rate were set to 3072 kHz, a single receiver would
be able to cover frequencies from a few kHz through at least 160
meters: Using the Linux "pee" command (part of the "moreutils" package - link) to split the pipe from sdrplayalsa it may be possible to send this to multiple instances of csdr which would "tune" and filter out several 768 kHz wide segments, each of which could, in turn, invoke fplay
to send those instances to different ALSA devices. Again, the
bandwidth limit of the 16 bit pipeline is 768 kHz and the maximum
number of "bands" allowed by the PA3FWM WebSDR software is eight - and
as mentioned above, one should also consider the signal dynamics and
overall performance as the RF input bandwidth is increased.
Using the "gain" files:
The Northern Utah WebSDR's servers have modified javascript code that will periodically (every 625 milliseconds) read the "gain" file from the server (which is why it should be placed within the ~/pub directory structure) and use this data (which is just the number of dB of extra attenuation)
to offset the S-meter reading so that it - and any S-meter graph files
that you may be creating - are corrected for RF AGC changes at the
receiver: The current value of "RF AGC" is also displayed on the
screen, next to the numerical S-meter reading.
When the WebSDR is started, the script that configures the SDRPlay devices and sdrplayalsa also creates eight "gain" files, each containing the value of "-1": When an instance of sdrplayalsa is started it will immediately place the value of "0" (zero) in the appropriate gain file. If the javascript "sees" a value of "-1", it will know that the receiver in question does not
have the ability to change AGC gain but any other value will cause the
S-meter correction and "RF AGC" values to be modified accordingly.
Comments:
- The "gainfile" produced by the "-e" parameter will
not exist
unless/until "sdrplayalsa" is started using the "-e" parameter with the
appropriate configuration. It is recommended that one use the
shortest file name practical that is also descriptive, such as "g0.txt"
through "g7.txt" to represent WebSDR bands "0" through "7".
- It is recommended that one produce a script that, when the
"sdrplayalsa" is started, delete the original gain files
("rm -f g*.txt") and replaced each of them with a file containing the
value "-1" (negative one), as in "echo -1 > g0.txt", "echo =1 > g1.txt" and so on.
- When sdrplayalsa starts up, the files relevant to the bands covered by an RSP will then be replaced with a zero (e.g. no RF AGC attenuation) but the bands without
an RSP will have a file with a value of "-1" indicating to the
javascript running on the users' computers that indicates that there is
not a receiver with AGC action available, which will also prevent the
text "RF AGC" from appearing below the S-meter for those receivers.
If you are interested in this code, please contact us using the information at the bottom of this page.
Using a RAM disk for the gain and log files
Although they don't produce a much data at all, the gain and log
files may be frequently written and with solid-state drives, it's
prudent to minimize the number of writes to the storage media.
Also, it's possible that when sdrplayalsa writes to the storage that it may "block" the rest of the code (possibly resulting in brief underruns or audio drop-outs) briefly if the system is busy and can't write to the storage device immediately.
Whether or not either of the above are really of concern, it's easy to configure and mount temporary storage devices.
Temporary storage for the gain file:
Because the gain file must
be publically aceessible, it must be located within the "\pub"
directory tree - and in keeping with the above, we'll call this
subdirectory "t" for "temporary": As a user other than sysop, create the subdirectory as follows: mdkir ~/pub/t
At the moment, this directory uses "normal" disk storage so we must cause it to be mounted as a "temporary" disk (using RAM) upon boot-up, so we need to add a line to /etc/fstab. First, we need to edit this so we do: sudo nano /etc/fstab - or you can use your favorite text editor.
Now, add the following lines to /etc/fstab:
# For sdrplayalsa gainfile
tempfs /home/websdr/pub/t tmpfs size=1M,mode=0777 0 0
This will mount to the directory previously created a RAM disk of 1
Megabyte in size. Since this mount is "owned" by sysop, we must
allow "everyone" to read/write it (e.g. "mode=0777").
This also assumes that the WebSDR runs in a directory structure
named "websdr" - but if not, you'll have to rename it accordingly.
1 Megabyte of storage is orders of magnitude more than is needed
to store a handful of very small files, but you can make it any size
that you like - within reason, of course.
Once the system is rebooted, you'll be able to see that ~/pub/t
is "owned" by "sysop" rather than as the user that created it in the
first place - but you should be able to read from and write to it.
At this point, be sure to modify the invokation of "sdrplayalsa" so that it uses this new location, as in: -e ~/pub/t/g1.txt which
specifies that the gain file be written to "~/pub/t". Of course,
the users' Javascript code that would read this gain file must also
know to look for this file - but in "/t".
Logfiles in RAM disk:
Even if you don't plan to use the gain file, it's also a good idea to use a RAM disk to store any log files produced by sdrplayalsa
in the event that it, too, could be affected by "blocking" should the
OS briefly hang the driver when trying to write to the log. The
procedure is as above, with some minor changes. First, let's
presume that we wish to store these log files in ~/cfg/temp, so we
would create that subdirectory: mkdir /~cfg/temp
Next, we create temporary storage and mount it at the above location as follows by adding the following lines to /etc/fstab:
# For sdrplayalsa log file
tempfs /home/websdr/cfg/temp tmpfs size=100M,mode=0777 0 0
Again, this won't take effect until a reboot.
This time we create a RAM disk of 100 Megabytes in size which should be
enough to store a significant number of log entries - but make sure
that you have plenty of free RAM to accommodate this. Note that
there is no check to prevent the size of the log file to exceed 100
Megabytes and if it does so, it's possible that sdrplayalsa
would crash: Generating 100 Megabytes of log entries over a
period of many months - even if multiple receivers are used - is very
unlikely if the receiver is operating normally and if the system is
rebooted even occasionally, this should never happen - but if this
bothers you, you might consider scheduling a "cleanup" once a week or
so that would rename/copy the "old" files elsewhere - or it could
simply trim the file a given number of lines: There are plenty of
examples of limiting file size using scripts (called by a "cron" job) to be found on the web.
Conclusion:
The above information should be enough to get the loopback system up
and running. Again, the above has been tested and found to work
on Ubuntu 18.x and 20.04, but it is not known if this same procedure
will work for other Linux distributions like Debian. If you try
it on another distribution, please let us know if it works - and what
it took to make it work if it didn't do so "out of the box". At
the time of writing, we don't have a Debian system on which to
try/troubleshoot this installation.
Please understand that we don't have the time or inclination to try the
above procedure on OSes other than Ubuntu 18 and 20 (there are so many!) so again, please
let us know about your successes or failures so that we can let others know.
Acknowledgment: I would
like to thank Gary Wong, AB1IP, for taking the time to produce and test
the base "sdrplayalsa" utility.
Additional
information:
- For general information about this WebSDR system -
including contact info - go to the about
page (link).
- For the latest news about this system and current issues,
visit the latest news
page (link).
- For more information about this server you may contact
Clint, KA7OEI using his callsign at arrl dot net.
- For more information about the WebSDR project in general -
including information about other WebSDR servers worldwide and
additional technical information - go to http://www.websdr.org
Back to the Northern Utah WebSDR landing page