New Morse Decoder - Part 4

In the previous blog entry  I shared some test results of the new experimental Bayesian Morse decoder.  Thanks to the alpha testers I found the bug that was causing the sensitivity to signal amplitudes and causing overflow. I have fix this bug in the software over the last few months.

CQ WW WPX  CW contest was in May 24-25 and it presented a good opportunity to put the new FLDIGI software version to test.  I worked some stations and let the software running for 48 hours to test the stability.

In the figure 1 the first 2 1/2 lines are decoded using the new Bayesian CW decoder. The following 2 lines is the same audio material decoded using the legacy CW decoder.  CW settings are also visible in the picture.  Matched filter bandwidth was set to 50Hz based on Rx speed of 32 WPM.

 The legacy decoder is doing a pretty good job following ZF1A working CW contest at 7005.52 KHz.  At first look it appears that the new Bayesian decoder is having a lot of difficulties.   Let's have a closer look what is really going on here.

Figure 1.  FLDIGI  CW Decoder testing 

In the audio recording  ZF1A made 5 contacts, with VE1RGB, UR4MF, KP2M, SM6BZV and WA2MCR in 1 min 31 seconds.

I let the captured audio file run in a loop for two times using both FLDIGI CW decoder versions.  The decoded text is shown below, broken into separate lines by each QSO for clarity.

FLDIGI - the new experimental Bayesian Morse decoder: 
RN 512 X 
ZF1A N VE1RGB 5NN513 X 
ZF1A --R4MF 5NN 0MO0 N T E E E E E E E TU
 ------TM N T E XJ0TT 6NN 515 X 
ZF1A N QT SM6BZV 5NM516 X 
ZF1A N WA2MCR 5NN 517 
NN 5 --2 B
ZF1A N VE1RGB 5MK 613 X 
ZF1A N KR4MF 5NN 514 X 
ZF1A N KP2M 6NN 515 TU 
ZF1A N OT SM6BZV 5NN 516 X
ZH1A N WA2MCR 5NN 517 

The first line should have been decoded as NN 512 TU but  the Bayesian decoder misdecoded last 2 letters. What was originally  TU  was decoded as X.

Let's look at the signal waveform (Fig 2.). It is a close call when looking at the waveform timing...same decoding error happens in almost all the cases above.

Figure 2.  TU or  X ? 

So what happened in the QSO with UR4MF? Let's look at waterfall picture (Fig 3.) for possible clues.
UR2MF is very visible at 692 Hz frequency but what is the faint signal that starts 200 msec before letter U? It is approximately 70Hz below UR2MF and starts "dah-dit".

Figure 3. UR4MF with another station 70 Hz below

The new Bayesian decoder is sensitive enough to be able to pick up this other station, but unfortunately the selected 50 Hz filter bandwidth is not enough to separate these two stations from each other, thus causing what appears an error in decoding.  Legacy decoder did not even notice this other station.



FLDIGI - legacy Morse decoder: 
6N S W2 TU 
F1 VE1RGB 5NN 513 TU 
ZF1A UR4MF N 514 TU 
ZF1A KP2M 5NN 515 TU 
ZF1 SM6BZV 5NN 516 TU 
ZF1A WA2MCR 5NN 17 
NN S W2 TU 
F1 VE1RGB 5NN 513 TU 
ZF1A UR4MF E N 514 TU 
ZF1A KP2M 5NN 515 TU 
ZF1 SM6BZV 5NN 516 TU 
ZF1A WA2MCR 5NN 17


First line should be NN 512 TU but deep QSB in the signal mangled the decoding into 6N S W2   TU.
See figure 4 on how the amplitude goes down rapidly between 5 and 1. The first "dit" in number one is barely visible in waterfall figure 5 below. While legacy decoder made an error in this case  the new Bayesian decoder didn't seem to have any problem despite this deep and rapid QSB.

Once again, the Bayesian decoder appears to be much more sensitive and automatically able to pickup signals even under deep QSB conditions.

Figure 4. QSB 

Figure 5. QSB in waterfall

CONCLUSIONS

Testing the performance of the new, experimental Bayesian Morse decoder with real world contest signals did show some surprising behaviors. What appeared initially to be errors in decoding turned out to be real signals that impacted the probabilistic decoding process. It also appears that the Bayesian method is much more sensitive and may need a bit different strategy related to signal pre-processing and pre-filtering compared to the legacy Morse decoder currently implemented in FLDIGI.

I am working on a better test framework to experiment more systematically the impact of various parameters to find the performance limits of the new experimental Bayesian Morse decoder.   Early results show for example that minimum CER  (character error rate) is dependent on SNR of signal as expected, but the CW speed dependent pre-filter bandwidth has some interesting effects on CER as demonstrated in figure 6.  Different graphs show how the speed (in WPM) setting impacts CER at different pre-filter bandwidth  (for example 20 WPM  ~ 33.3Hz, 80 WPM ~ 133.3 Hz ).  You would expect best CER performance with the most narrow filter setting that matches the actual speed (in this case actual speed was 20 WPM).  However,  as seen in figure 6  this is not consistently the case with signals between SNR  +2 ... +15 dB.

Figure 6.  CER vs. SNR testing at different WPM settings