| Jim Reisert : DX4WIN | QSL Manager Database |
This note attempts to describe how the DX4WIN QSL Manager database is maintained.
The QSL Manager database is an integral part of the DX4WIN Country File. As of January 2022, it has over 90,000 QSL manager records. The QSL Manager Database also contains QSL Manager addresses, if I can find them. If a QSL manager manages two or more DX stations, then an address may be found in the database, otherwise there is no address.
I was not going to take ownership of the QSL Manager database unless I limit the amount of work I had to do. I had two goals:
There are many methods through which to obtain a QSL (or confirmation) these days. If available, this information is added to the manager callsign. This is the current list:
In addition, you may see these "managers" listed:
If you see one (or more) of these, it means that it is the only method that the DX station or QSL manager uses.
One of the easiest way to acquire data is from the comments from DX spots made on the DX Cluster. Many people comments in their spots, such as "QSL to" or "VIA". This makes it easy to filter the spots for the relevant information. The fact that different people make the spots creates diversity - the chance of everyone broadcasting incorrect information is low.
QRZ.com is another major source of data. Each day, QRZ publishes a list of newly-added callsigns, some of which have QSL managers. These data are collected regularly and checked for QSL manager additions.
Two other sources of data are DXnews and DX-World.
After the data has been accumulated, it needs to be processed to remove incorrect information. This becomes a probability problem - how likely is this manager (or callsign) correct?
First, unique "DX via MANAGER" instances are thrown out immediately, as they are likely to be incorrect.
Next, a program compares the data with itself, looking for other calls with similar managers, and other managers with similar calls. If the ratio of two calls managed by the same station is high enough, the program will discard the lesser of the two. The same thing happens when there are two managers listed for the same call.
Most USA stations with USA managers are discarded. These can be looked up in the callbook.
Portable stations who answer their own QSLs are also discarded. One can infer the manager from the callsign, so it doesn't make sense to waste space in the manager database for these.
A separate exception file keeps a list of known bad call/manager pairs that the program didn't find. This file is maintained by hand.
Due to the way the calls/managers are filtered, some legitimate calls get filtered out, and must be added back in.
In addition, contest callsigns are often spotted without the manager, so these need to be looked up periodically and added to the file.
A separate addition file keeps all these manual additions to the database.
Callsigns are assumed to have no more than 3 consecutive letters (i.e. W1ABC). Anything more than that is a potentially bad callsign. However, there are many exceptions, such as 8J3YAGI, DA0UBOOT, etc.
Callsigns do not generally end in a number, but there are also exceptions to this rule, such as V6T1, 5A27, XE21, etc.
Manager callsigns should contain both numbers and letters, but the program allows some exceptions, such as JARL, YASME, etc.
A separate file holds all these special (known good) callsigns.
People change their callsigns over time, so it's important for the database to have the manager's current callsign.
There is a separate file which maps old callsigns to new, for example: ex-PA0GAM is PG5M, ex-WB2DND is N1DG, etc.
One analysis program takes all the inputs and uses them to create the QSL manager database:
The output data is sorted first by DX callsign, then by manager callsign. The output file is pretty clean. It's then compared with the output from the last release to see what changes took place. This makes it easy to spot new bad calls which must be added to the exception file. There are a few new exceptions with each release.
Finally, the data is sorted once more and some additional data is added. These are mostly cases where a QSL manager is "reused". The analysis program allows a manager to be associated only ONCE with a given DX callsign. This particular issue comes up with 3V8BB, HF0POL and a few other callsigns. The final output is sorted first by date, then by call, then by manager. This makes it easy to see the managers for a given call in chronological order. The final output also has a VERSION date appended to the end of the file, to indicate when it was created.
Data collection is simplified by using a free, diverse source of data: DX cluster spots. I reguarly collect DX spots from the DX Cluster. These spots are saved to a file, and a program converts the raw spot format to DX4WIN QSL manager format (Call, Date, Manager) to be fed into the analysis program.
The analysis program automates the validation of data. After running the program, the output is compared to the previous release - new calls will stand out, and multiple or incorrect managers can be identified. Also, the log file is checked for possible special callsigns - they will be discarded, but can be added back in later.
Here are some statistics as of 12 January 2022:
It takes several minutes to build a new list of QSL managers from DX spots. However, once the DX spots initial database is built, updates can be added in more-or-less no time. Other than the DX spot processing, the rest of the process takes about 40 seconds on an older laptop. Processing on my main desktop computer is much faster.