The Limits of Open Source
10/01/2019
WARNING: This article is intended for computer scientists or, at the very least, people who are extremely curious about combating hackers.
This story begins with the observation of unusual messages in the system logs:
Aug 31 08:43:31 host sshd[14231]: dispatch_protocol_error: type 90 seq 5 [preauth] Aug 31 08:44:01 host sshd[14231]: dispatch_protocol_error: type 96 seq 6 [preauth] Aug 31 08:44:01 host sshd[14231]: dispatch_protocol_error: type 97 seq 7 [preauth]
At first glance, a system administrator’s instinct might suggest ignoring these messages. That will indeed be the conclusion of the analysis they prompted. If you’re here just to know what to do with them, reading further is optional.
On August 31, it’s still somewhat vacation time. So, there’s a bit more time, and one would like to know what these mysterious "type 90, 96, or 97" messages with a "dispatch_protocol_error" mean.
For the uninitiated, it’s worth noting that these error messages come from the sshd daemon. Simply put, a daemon is a background system process. The sshd process manages remote connections. In the context of server administration, it’s both an essential and unavoidable component and, consequently, the primary target for hackers.
A methodical internet search about these mysterious error messages yields no conclusive results. So, we must go to the source of the information: the OpenSSH source code.
This gives us over 740 files totaling more than 200,000 lines of code... finding the desired information is a true treasure hunt.
dispatch.c
The instruction producing the "dispatch_protocol_error: type t seq n" error message is easy to find in the dispatch_protocol_error() function.
auth2.c
However, this function is not called directly anywhere; instead, a pointer to this function is used in the instruction ssh_dispatch_init(ssh, &dispatch_protocol_error).
dispatch.c
By examining what the ssh_dispatch_init() function does with its dflt parameter, we discover that the pointer to the function responsible for the mysterious messages is stored in the ssh->dispatch[i] structure.
packet.c
By searching for the use of this information, we learn that the function is called in ssh_dispatch_run() via the instruction (*ssh->dispatch[type])(type, seqnr, ssh). At this point, type is already defined earlier in the same function by one of the instructions ssh_packet_read_seqnr(ssh, &type, &seqnr) or ssh_packet_read_poll_seqnr(ssh, &type, &seqnr).
dispatch.c
By studying the code of these two functions, we deduce that the famous type is defined in the ssh_packet_read_poll2() function. It is first initialized to the value SSH_MSG_NONE and then set by sshbuf_get_u8(state->incoming_packet, typep).
ssh2.h
Here, we find an interesting clue: SSH_MSG_NONE, which leads to the treasure trove of information we were seeking:
/* channel related messages */ #define SSH2_MSG_CHANNEL_OPEN 90 #define SSH2_MSG_CHANNEL_OPEN_CONFIRMATION 91 #define SSH2_MSG_CHANNEL_OPEN_FAILURE 92 #define SSH2_MSG_CHANNEL_WINDOW_ADJUST 93 #define SSH2_MSG_CHANNEL_DATA 94 #define SSH2_MSG_CHANNEL_EXTENDED_DATA 95 #define SSH2_MSG_CHANNEL_EOF 96 #define SSH2_MSG_CHANNEL_CLOSE 97 #define SSH2_MSG_CHANNEL_REQUEST 98 #define SSH2_MSG_CHANNEL_SUCCESS 99 #define SSH2_MSG_CHANNEL_FAILURE 100
So, these mysterious "type 90, 96, or 97" messages with a "dispatch_protocol_error" correspond to:
SSH2_MSG_CHANNEL_OPEN SSH2_MSG_CHANNEL_EOF SSH2_MSG_CHANNEL_CLOSE
And now it’s much clearer!
These are likely connection attempts made with inconsistent packets. Such an approach suggests a technique similar to that used to exploit the CVE-2018-15473 vulnerability: generating errors intentionally to indirectly gather information about the target.
Regarding measures to counter this new form of attack, there’s nothing more to do to strengthen the server’s defenses beyond what’s already in place.
All of the above is merely a preamble to the real subject of this article: a reflection on open source.
First, it directly illustrates the reflection in the article "WHAT IS BETTER THAN OPEN SOURCE?": concise documentation would be far more useful alongside source files; it would avoid laborious reverse engineering to decode the hidden meanings of error messages.
But above all, the analysis conducted during this investigation into an essential and critical component of internet-exposed servers is chilling.
It reveals an artificially complicated programming style. For example: why on earth go through a pointer to a function laboriously stored in multiple instances instead of simply calling the function itself? The rest follows suit, with an avid use of pointers, a very powerful feature when used well but a formidable source of errors.
This likely has little impact on performance but results in code that is difficult to understand, thus prone to bugs and hard to maintain; a true breeding ground for security vulnerabilities and a very favorable terrain for backdoors.
This leads to the opposite result of what open source aims to achieve!