A System to Judge Information Reliability

In today’s world with a vast array of sources of information.  Judging whether articles are useful or whether they are less than useful (if not an absolute fabrication) is always a concern.  In the intelligence community this judgement is known as evaluation.  Evaluation allows an individual or organization to judge the level of confidence they have in the information provided or accessed (by various means).  NATO and by consequence Canada has adopted a simple evaluation rating system.

The system is not perfect, it is purposely simple to allow a quick method to evaluate information. It is not analysis nor does it necessarily take into account a deliberate program to deceive.  Evaluation is a good first step but is only a first step.  As with all information, any particular information must be ‘multi-sourced’ and each source evaluated on its own.

The table below is the NATO evaluation table utilized in the early part of the intelligence cycle to evaluate information.  It is used in conjunction with other methods and is not used by itself for detailed work.  For most people, it is a very useful all by itself.  The table provides an alphanumeric system to create a “grade” for each piece of information.  For example, A1 would be a piece of information that is the very top and could be relied upon as completely true.  A B3 would be from a source that has had minor issues and is reporting an incident, etc., that could be very possible.  This would result in follow up action to verify the actual quality of the information.  E5 would be the least truthful.  F6 is not the worst; it just means there is no history or other information to help verify the information.

This system is not a replacement for follow up with both analysis and study or action to further understand the information.

There are issues that may cause problems:

Circular information reporting – one organization reports on a subject which is used by other individuals or organizations also reporting on the same story.  They may add their own interpretation to the story. The problem is that the original story may be incorrect in the first place.  As ‘new’ stories are reported the information gains credibility despite the original story being incorrect.

Originator’s motivation – why is it being written?  Who is writing it?  Do they have a political bent?  Could there be a deliberate attempt to create their own spin?

Receiver’s motivation – everyone has their own prejudices.  Every time one receives a piece of information these prejudices or points of view ‘colour’ one’s evaluation.  Information that does not match one’s own view can sometimes be evaluated as lesser information.  Understanding one’s own prejudices is critical.

Information is wrong – with the 24 hour news cycle, incorrect material often gets published. Larger news organizations usually have sufficient capability to check information before it is used.  This is not true for freelance or smaller organizations.  Some malevolent actors will actually go out of their way to create a false narrative.

These are not the only problems but are mentioned to caution one from being too quick to evaluate.  It is true with every human that errors will occur despite one’s best efforts.  Knowing oneself before one evaluates is a good place to start.

A PDF of this Information Note is available here.