FLDIGI Morse Decoder CER vs SNR Testing
Summary
I wanted to get more factual data on how different FLDIGI CW options influence the character error rate (CER) at different signal-to-noise ratios (SNR). In this test I focused on comparing SOM decoder to legacy decoder option and Matched Filter to FFT filter option. I also tested a few cases where I used ~ 2x wider FFT filter (68 Hz vs. 35 Hz bandwidth). Otherwise I used the default options of FLDIGI.Based on the test results below the following appears to be true:
- Adjusting FFT filter bandwidth has much bigger impact on CER than changing between legacy and SOM decoder.
- Matched filter automatically sets the filter bandwidth to optimal for given CW speed.
- SOM decoder gives about 5% better CER compared to legacy decoder @ -13dB SNR, but this advantage gets smaller as SNR decreases.
The test results are summarized in the figure 1 below. With 3 kHz wide audio signals you can get near perfect copy (CER below 0.02) at SNR -10 dB using Matched filter or setting FFT filter at 34 Hz corresponding to 20 WPM Morse speed.
 |
| Fig 1. FLDIGI Morse Code decoder CER vs. SNR |
Test Cases and Setup
My goals for this this experiment were to
- Compare FLDIGI legacy vs. SOM decoder CER at different SNR levels.
- Compare FLDIGI Matched Filter vs. FFT filter (bandwidth set to 35Hz) CER at different SNR levels.
- Compare CER with FFT filter bandwith set to 68 Hz vs. 35 Hz.
I used the following software versions to conduct this test:
First I generated a test audio file with 20 WPM Morse code using WinMorse and converted it to 16 bit 8 Khz Mono Wav file using Audacity. I played the audio file with PathSim configured for AWGN S/N Test simulation. Both AWGN Noise source and "Input Source" have 3 kHZ band pass filter after them as demonstrated in the figure below.
 |
| Fig 2. PathSim Configuration |
Sound output from PathSim (8 kHz 16 Bit Mono) was connected to FLDIGI input using Virtual Audio cable and level was adjusted at 10 in Windows 7 volume mixer so that FLDIGI waterfall is not saturated. Audio signal had 3 Khz bandwidth as verified on FLDIGI waterfall display.
Audio file length was 11 min 27 seconds. PathSim "RunTime" was adjusted to 11 minutes - this provided only 869 of 904 total character in the file. The lack of last 27+ seconds creates a systematic error of 35 missing characters causing baseline error of CER 0.03871 - this is already subtracted from results below.
I changed audio SNR in PathSim in steps from -20 dB, -15 dB, -14dB, -13 dB, -10 dB and 0 dB also did a reference test case without any injected noise. I compared FLDIGI decoded characters against sent characters using RTTY Text Comparison tool by Alex VE3NEA. As FLDIGI sometimes emitted prosigns (like <AR> ) I edited these to a single '*' character before entering to 'received' field in the RTTY Text Comparison tool. Note that this tool was designed for RTTY characters so BER numbers are not applicable for Morse code as RTTY has fixed bit length baudot code whereas Morse code bit length varies by character. The character error rate (CER) is still be applicable.
I plotted the following measured CER results against SNR values with Graph.
Test Results
Reference case
No noise CER 0.0011 First character incorrectly coded
Test cases
SNR CER Test0 db 0.00001 SOM Decoder with FFT Filter @68 Hz
-10 dB 0.01219 Legacy decoder with Matched Filter @35 Hz
-10 dB 0.01439 SOM decoder with Matched Filter @35 Hz
-10 dB 0.01439 Legacy decoder with FFT Filter @35 Hz
-10 dB 0.00999 SOM Decoder with FFT Filter @35 Hz
-10 db 0.20799 SOM Decoder with FFT Filter @68 Hz
-13 dB 0.19469 Legacy decoder with Matched Filter @35 Hz
-13 dB 0.18589 SOM decoder with Matched Filter @35 Hz
-13 dB 0.16149 Legacy decoder with FFT Filter @35 Hz
-13 dB 0.15379 SOM Decoder with FFT Filter @35 Hz
-13 db 0.66479 SOM Decoder with FFT Filter @68 Hz
-14 dB 0.37499 Legacy decoder with Matched Filter @35 Hz
-14 dB 0.35729 SOM decoder with Matched Filter @35 Hz
-14 dB 0.30199 Legacy decoder with FFT Filter @35 Hz
-14 dB 0.30089 SOM Decoder with FFT Filter @35 Hz
-15 dB 0.50669 Legacy decoder with Matched Filter @35 Hz
-15 dB 0.49779 SOM decoder with Matched Filter @35 Hz
-15 dB 0.47899 Legacy decoder with FFT Filter @35 Hz
-15 dB 0.46569 SOM Decoder with FFT Filter @35 Hz
-15 db 0.71789 SOM Decoder with FFT Filter @68 Hz
-20 dB 0.74229 Legacy decoder with Matched Filter @35 Hz
-20 dB 0.68479 SOM decoder with Matched Filter @35 Hz
-20 dB 0.72569 Legacy decoder with FFT Filter @35 Hz
-20 dB 0.68029 SOM Decoder with FFT Filter @35 Hz
-20 db 0.75439 SOM Decoder with FFT Filter @68 Hz
Other Observations on FLDIGI Performance
At low SNR values (-20 dB) FLDIGI seemed to lose CW speed tracking, especially if Matched filter was used. The RxWPM display showed received speed variation between 23 and 30 WPM while sent signal was at steady 20 WPM. This may explain some of the decoding errors. In real life cases if you know the Morse speed it is probably better to set it manually when trying to decode very noisy signals.73
Mauri AG1LE
No comments:
Post a Comment