Contact

Subscribe via Email

Subscribe via RSS

Categories

Creative Commons Attribution 4.0 International License
© Rakhesh Sasidharan

KiTTY with Fedora 22/ OpenSSH 6.9 gives a blank screen and hangs

The KiTTY SSH client for Windows gives a blank screen and hangs when connecting to my Fedora 22 box (which runs OpenSSH 6.9). KiTTY is a fork of PuTTY, which I am happy with, so there’s no particular reason for me to use KiTTY except that I am checking it out.

kitty hungAfter a while it gives an error that the server closed the connection.

kitty fatalI have an Ubuntu Server 14.10 and CentOS 6.7 and KiTTY works fine with them. So it seems to be related to the version of OpenSSH in Fedora 22.

Fedora 22 logs show entries like these:

Looks like key-exchange was failing?

/*

Note to self: An alternative command syntax to the above is this:

I also noticed that including SELinux in the output gave a bit more info. Not that it helped but it’s worth mentioning here as a reference to myself. For that I include the sshd.service unit and also the SELinux context for SSHD.

Tip: After typing journalctl it is possible to keep pressing TAB to get the fields and their values. That’s how I discovered the SSHD context above.

On the topic of journalctl: this intro post by its creator and this DigitalOcean tutorial are worth a read.

Another useful tip with journalctrl: by default it doesn’t wrap the output but you can scroll left and right with the keyboard. This isn’t useful when you want to copy-paste the logs somewhere. The work-around is to use the --no-pager switch so everything’s dumped out together and then pipe it into less which wraps the text:

*/

I turned on SSH packet and raw data logging in KiTTY.

kitty logginFrom the log files I could see a sequence of events like this:

And with that it stops. So it looked like the DH Key Exchange was failing. Which confirms what I saw on the server side too.

SSH protocol 2 supports DSA, ECDSA, Ed25519 and RSA keys when clients establish a connection to the server. However, the first step is a Key Exchange for forward-secrecy and for this the DH algorithm is used.  From the sshd(8) manpage:

For protocol 2, forward security is provided through a Diffie-Hellman key agreement.  This key agreement results in a shared session key.  The rest of the session is encrypted using a symmetric cipher, currently 128-bit AES, Blowfish, 3DES, CAST128, Arcfour, 192-bit AES, or 256-bit AES.  The client selects the encryption algorithm to use from those offered by the server.  Additionally, session integrity is provided through a cryptographic message authentication code (hmac-md5, hmac-sha1, umac-64, umac-128, hmac-ripemd160, hmac-sha2-256 or hmac-sha2-512).

From the sshd_config(5) manpage:

The supported algorithms are:

curve25519-sha256@libssh.org
diffie-hellman-group1-sha1
diffie-hellman-group14-sha1
diffie-hellman-group-exchange-sha1
diffie-hellman-group-exchange-sha256
ecdh-sha2-nistp256
ecdh-sha2-nistp384
ecdh-sha2-nistp521

The default is:

curve25519-sha256@libssh.org,
ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,
diffie-hellman-group-exchange-sha256,
diffie-hellman-group14-sha1

The KexAlgorithms option can be used to change this but the point is all these are variants of the DH algorithm.

KiTTY has the following algorithm selection options:

kitty options

For protocol 2 only the first three (Diffie-Hellman options) apply. The fourth one (RSA-based key exchange) only applies for protocol 1. Comparing this with the preferred order for OpenSSH only the first and second options are common:

  • Diffie-Hellman group exchange (a.k.a. diffie-hellman-group-exchange-sha256)
  • Diffie-Hellman group 14 (a.k.a. diffie-hellman-group14-sha1)

Of these the second one (diffie-hellman-group14-sha1) is the older one. It is defined in RFC 4253. This RFC defines two key exchange algorithms: diffie-hellman-group14-sha1 and diffie-hellman-group1-sha1. (The group number defines the number of bits in the key. Group 1 has 768 bits, group 14 has 2048 bits. Larger is better). (Correction: Turns out diffie-hellman-group1-sha1 is actually Group 2).

The key exchange algorithms of RFC 4253 are updated via RFC 4419 – which defines two additional algorithms: diffie-hellman-group-exchange-sha1 and diffie-hellman-group-exchange-sha256. I am not very clear about these but it looks like they remove any weaknesses to do with groups that define the fixed number of bits and introduces the ability for the client and server to negotiate a custom group (of 1024 – 8192 bits). From RFC 4419:

Currently, SSH performs the initial key exchange using the “diffie-hellman-group1-sha1” method [RFC4253].  This method prescribes a fixed group on which all operations are performed.

The security of the Diffie-Hellman key exchange is based on the difficulty of solving the Discrete Logarithm Problem (DLP). Since we expect that the SSH protocol will be in use for many years in the future, we fear that extensive precomputation and more efficient algorithms to compute the discrete logarithm over a fixed group might pose a security threat to the SSH protocol.

The ability to propose new groups will reduce the incentive to use precomputation for more efficient calculation of the discrete logarithm.  The server can constantly compute new groups in the background.

So, to summarize: the common algorithms between KiTTY and OpenSSH are diffie-hellman-group-exchange-sha256 and diffie-hellman-group14-sha1. By default the preferred order between KiTTY client and OpenSSH server are in the order I specified above. In terms of security diffie-hellman-group-exchange-sha256 is preferred over diffie-hellman-group14-sha1. For some reason, however, KiTTY breaks with the newer version of OpenSSH when it comes to this stage.

Since diffie-hellman-group-exchange-sha256 is what both would be using I changed the preferred order on KiTTY such that  diffie-hellman-group14-sha1 is selected. 

kitty orderOnce I did that KiTTY connected successfully with OpenSSH – so that worked around the problem for now albeit by using a weaker algorithm – but at least it works. I wonder why diffie-hellman-group-exchange-sha256 breaks though.

I downloaded the latest version of PuTTY and gave that a shot with the default options (which are same as KiTTY). That worked fine! From the PuTTY logs I could see that at the point where KiTTY failed PuTTY manages to negotiate a key successfully:

Comparing the configuration settings of PuTTY and KiTTY I found the following:

putty fix

If I change the setting from “Auto” to “Yes”, PuTTY also hangs like KiTTY. So that narrows down the problem. RFC 4419 is the one I mentioned above, which introduced the new algorithms. Looks like that has been updated and KiTTY doesn’t support the new algorithms whereas PuTTY is able to.

PuTTY’s changelog didn’t have anything but its wishlist contained mention of what was going on. Apparently it isn’t a case of an update to RFC 4419 or any new algorithms, it is just a case of OpenSSH now strictly implementing the message format of RFC 4419 and thus breaking clients who do not implement this yet. To understand this have a look at this bit from the log entries of PuTTY and KiTTY at the point where KiTTY fails:

Pretty identical except for the type number. KiTTY uses 30 while PuTTY now uses 34. If you look at RFC 4419 these numbers are defined thus:

The following message numbers have been defined in this document.

They are in a name space private to this document and not assigned by IANA.

     #define SSH_MSG_KEX_DH_GEX_REQUEST_OLD  30
     #define SSH_MSG_KEX_DH_GEX_REQUEST      34
     #define SSH_MSG_KEX_DH_GEX_GROUP        31
     #define SSH_MSG_KEX_DH_GEX_INIT         32
     #define SSH_MSG_KEX_DH_GEX_REPLY        33

SSH_MSG_KEX_DH_GEX_REQUEST_OLD is used for backward compatibility. Instead of sending “min || n || max”, the client only sends “n”.  In addition, the hash is calculated using only “n” instead of “min || n || max”.

So RFC 4419 has updated the type number to 34 and set aside 30 for backward compatibility. Ideally clients should be sending type number 34 as their request. From the PuTTY wishlist link I saw the OpenSSH commit that removed the server from accepting type 30 any more. That’s why KiTTY was breaking, while PuTTY had updated their message type to the correct one and was successfully connecting! :)

(While digging around I saw that WinSCP too is (was?) affected.)

After Windows Update KB 3061518 many websites stop working in IE

At work every time some of our IT staff would access the BES server web console, IE would fail. Not with a 404 Page not found error but with a web site cannot be found error (with helpful hints to check your Internet connection, DNS, etc). 

The web console worked fine on my machine. I could ping the server from all machines and telnet to port 443 (HTTPS port) from all machines. The IE security, SSL, and certificate related settings were the same across all machines (including mine). Firefox gave the same error on all the machines – something about cipher suites – this was odd, but at least consistent across all machines (and we use IE with the console usually so wasn’t sure if Firefox always gave this error or it was just a recent occurrence). 

Since it didn’t seem to be a browser specific setting, and the Firefox cipher error was odd, I felt it must be something at the machine level. Unlike Firefox (and Chrome) which use their own SSL suite IE uses the Windows Secure Channel provider so there must be something different between my install of Windows and the problematic users. I had a look at the Event Viewer when opening the site in IE and found the following error: 

image001Couldn’t find much hits for that error The internal error state is 808 but at least it was Schannel related like I suspected. 

Time to find out if there were any difference in the Windows updates between my machine and the users. The following command gives a list of Windows Updates and the installed date:

The result was something along these lines (these are updates that were installed in the last two weeks on the problematic machine but not on my machine):

Since the problem started recently it must be one of the updates installed on the 20th. Going through each of the KB articles I finally hit gold with the last one – KB 3061518. Here’s what the update does:

This security update resolves a vulnerability in Windows. The vulnerability could allow information disclosure when Secure Channel (Schannel) allows the use of a weak Diffie-Hellman ephemeral (DHE) key length of 512 bits in an encrypted Transport Layer Security (TLS) session. Allowing 512-bit DHE keys makes DHE key exchanges weak and vulnerable to various attacks. For an attack to be successful, a server has to support 512-bit DHE key lengths. Windows TLS servers send a default DHE key length of 1,024 bits. After you install this security update, the minimum allowed DHE key length on client computers is changed to 1,024 bits by default, instead of the previous minimum allowed key length of 512 bits.

The workaround is simple. Either fix TLS on the webserver so its key length is 1024 bits or make a registry change on client computers so a key length of 512 bits is acceptable. I tested the latter on the user machine and that got the web console working, thus confirming the problem. Save the following as a .reg file and double click:

Reading more about the update I learnt that it’s a response to the logjam vulnerability against the Diffie-Hellman Key Exchange. I had forgotten about the Diffie-Hellman Key Exchange but thankfully I had written a blog post about this just a few months ago so I can go back and refresh my memory. 

Basically the Diffie-Hellman is an algorithm that helps two parties derive a shared secret key in public, such that anyone snooping on the conversation between these two parties has no idea what the shared secret key is. This secret key can then be used to encrypt further communication between these parties (using symmetric encryption, which is way faster). A good thing about Diffie-Hellman is that you can have an ephemeral version too in which every time the two parties talk to each other they generate a new shared secret to encrypt that conversation. This ephemeral Diffie-Hellman version is generally considered pretty secure and recommended (because if someone does ever break the encryption of one conversation, they still have no way of knowing what was talked about in other conversations). The ephemeral version can also use elliptic curves to generate the shared secret, this has the advantage of also being computationally faster. 

Anyways, ephemeral Diffie-Hellman is still a secure algorithm but there’s an unexpected problem with it. You see, back in the old days the US had export restrictions on crypto, and so web servers had this mode wherein they could be asked to use export ciphers and they will intentionally weaken security. A while back this “feature” was in the news thanks to the FREAK attack. Now it’s in the limelight again thanks to the logjam attack (which is what Microsoft’s KB fix above aims to fix). Unlike FREAK, though, which was an implementation bug, logjam is just expected behavior – just that no one still remembered this is how the system is supposed to behave now!

Here’s what happens. As usual the two parties (client and web server) will talk to each other in the open and come up with a shared secret key using the (ephemeral) Diffie-Hellman Key Exchange. In practice this should be foolproof but what could happen is that if the web server supports these export ciphers mode I mentioned above, someone could come in between the client-server conversation and ask the server to switch to this export mode (all that’s required is that the client – or rather someone pretending to be the client – ask the server for export grade ciphers). When the server gets this request it will intentionally choose weak parameters from its end as part of the Diffie-Hellman Key Exchange (specifically it will choose a 512 bits key that will be used to generate the ephmeral key later). The server won’t tell the client it’s choosing weaker parameters because it thinks the client asked, so the client is none the wiser that someone is playing mischief in between. The client will use these weaker parameters with the result that now the third party can try and decrypt the conversation between the client-server because the key they agree upon is a weaker key. 

Note that this has nothing to do with the key size of the certificate. And it has nothing to do with Diffie-Hellman Key Exchange being weak. It is only about servers still supporting this export mode and so the possibility that someone could get a server to use weaker parameters during the Key Exchange. 

The fix is simple really. You, the client, don’t have much of a say on what the server does. But you can insist that if a server could choose a 512 bits key as part of the Key Exchange process then you will simply refuse to deal with said server. And that’s what the KB fix above does. Once installed, if Schannel (on the client) notices that the web server it is talking to allows for 512 bits Diffie-Hellman keys it will simply refuse to talk to that web server. Period! (Note the emphasis here: even if the server only allows for 512 bits, i.e. it is not actually offering a weaker Diffie-Hellman parameter but merely supports export ciphers, the client will stop talking to it!). 

On the server side, the fix is again simple. You have two options – (1) disable export ciphers (see this and this articles for how to on Windows servers) and/ or (2) use Elliptic Curve Diffie-Hellman Key Exchange algorithms (because they don’t have the problems of a regular Diffie-Hellman Key Exchange). Do either of these and you are safe. (In our case I’ll have to get our admins looking after this BES console to implement one of these fixes on the server side). 

And that’s it! After a long time I had something fun to do. A simple problem of a website not opening turned out to be an interesting exercise in troubleshooting and offered some learning afterwards. :) Before I conclude here’s two links that are worth reading for more info on logjam:  CloudFlare postwebsite of the team who discovered logjam, with their recommendations for various web servers.

Cheers.

Update: This post by Matthew Green is a must read on logjam. Key takeaways (in case you weren’t already aware):

  • Logjam only applies to the non-Elliptic Curve variants of DHE. Regular DHE depends on the difficulty of solving the Discrete Logarithm problem. ECDHE depends on difficulty of solving Elliptic Curves.
  • Discrete Logarithm is still difficult to solve, but for small values of its parameters (e.g. 512 bits) it is relatively easier. 
  • Things are also made easier by the fact that most web servers use a common set of prime numbers. So attackers can precompute a table of prime numbers and use these to degrade the connection so it uses one of these weaker set of prime numbers (whose attack tables are already with them) and use these to quickly decrypt. 
  • Read the rest of that post for lots more interesting details. Thanks to Bruce Schneier’s blog for pointing to the post. 

Also, remember: this is about the key exchange. Not about server identity. The server can use an RSA certificate to verify its identity but use Diffie-Hellman for key exchange (and that is the preferred scenario in fact as Diffie-Hellman is better). Here’s RFC 4492 which lists five different key exchange algorithms. Moral of the story is, use Diffie-Hellman as usual but either disable the export grade stuff (so there’s no chance of using weaker parameters) or switch to the better Elliptic Curve Diffie-Hellman. This has nothing to do with the RSA or DSA certificates that the server might otherwise use

Notes on TLS/SSL, RSA, DSA, EDH, ECDHE, and so on …

The CloudFlare guys make excellent technical posts. Recently they introduced Keyless SSL (which is a way of conducting the SSL protocol wherein the server you are talking to does not necessarily need to have the private key) and as part of the post going into its technical details they talk about the SSL protocol in general. Below are my notes on this and a few other posts. Crypto intrigues me as I like encryption and privacy so this is an area of interest. 

Note: This is not a summary of the CloudFlare blog post. It was inspired by that post but I talk about a lot more basic stuff below. 

The TLS/SSL protocol

First things first – what we refer to as Secure Sockets Layer (SSL) protocol is not really SSL but Transport Layer Security (TLS).

  • Versions 1.0 to 3.0 of SSL were called, well … SSL 1.0 to SSL 3.0. 
  • TLS 1.0 was the upgrade from SSL 3.0. It is very similar to SSL such that TLS 1.0 is often referred to as SSL 3.1. 
  • Although the differences between TLS 1.0 and SSL 3.0 are not huge, the two cannot talk to each other. TLS 1.0, however, includes a mode wherein it can talk to SSL 3.0 but this decreases security. 

The world still refers to TLS as SSL but keep in mind it’s really TLS. TLS has three versions so far – TLS 1.0, TLS 1.1, and TLS 1.2. A fourth version, TLS 1.3, is currently in draft form. I would be lying if I said I know the differences between these versions (or even the differences between SSL 3.0 and TLS 1.0) so it’s best to check the RFCs for more info!

TLS/SSL goals

The TLS/SSL protocol has two goals:

  1. Authenticate the two parties that are talking with each other (authentication of a server is the more common scenario – such as when you visit your bank’s website for instance – but authentication of the user/ client too is supported and used in some scenarios). 
  2. Protect the conversation between the two parties.

Both goals are achieved via encryption.

Encryption is a way of “locking” data such that only a person who has a “key” to unlock it can read it. Encryption mechanisms are essentially algorithms: you take a message, follow the steps of the algorithm, and end up with an encrypted gobbledygook. All encryption algorithms make use of keys to lock and unlock the message – either a single key (which both encrypts and decrypts the message) or two keys (either can encrypt and decrypt).

Shared key encryption/ Symmetric encryption

A very simple encryption algorithm is the Caesar Cipher where all you do is take some text and replace the letters with letters that are a specified number away from it (for example you could replace “A” with “B”, “B” with “C”, and so on or “A” with “C”, “B” with “D”, and so on … in the first case you replace with a letter one away from it, in the second case you replace with a letter two away from it). For the Caesar Cipher the key is simply the “number of letters” away that you choose. Thus for instance, if both parties decide to use a key of 5, the encrypting algorithm will replace each letter with one that’s 5 letters away, while the decrypting algorithm will replace each letter with one that’s 5 letters before. The key in this case is a shared key – both parties need to know it beforehand. 

Encryption algorithms where a single shared key is used are known as symmetric encryption algorithms. The operations are symmetrical – you do something to encrypt, and you undo that something to decrypt. TLS/SSL uses symmetric encryption to protect the conversation between two parties. Examples of the symmetric encryption algorithms TLS/SSL uses are AES (preferred), Triple DES, and RC4 (not preferred). Examples of popular symmetric  algorithms can  be found on this Wikipedia page

Symmetric encryption has the obvious disadvantage that you need a way of securely sharing the key beforehand, which may not always be practical (and if you can securely share the key then why not securely share your message too?).

Public key encryption/ Asymmetric encryption

Encryption algorithms that use two keys are known as asymmetric encryption or public key encryption. The name is because such algorithms make use of two keys – one of which is secret/ private and the other is public/ known to all. Here the operations aren’t symmetrical- you do something to encrypt, and you do something else to decrypt. The two keys are special in that they are mathematically linked and anything that’s encrypted by one of the keys can be decrypted only by the second key. A popular public key encryption algorithm is RSA. This CloudFlare blog post on Elliptic Curve Cryptography (ECC), which is itself an example of a public key encryption algorithm, has a good explanation of how RSA works. 

Public key encryption can be used for encryption as well as authentication. Say there are two parties, each party will keep its private key to itself and publish the public key. If the first party wants to encrypt a message for the second party, it can encrypt it using the public key of the second party. Only the second party will be able to decrypt the message as only it holds the private key. Thus the second party is authenticated as only the second party holds the private key corresponding to the public key.

Public key cryptography algorithms can also be used to generate digital signatures. If the first party wants to send a message to the second party and sign it, such that the second party can be sure it came from the first party, all the first party needs to do is send the message as usual but this time take a hash of the message encrypt that with its private key. When the second party receives this message it can decrypt the hash via the public key of the first party, make a hash itself of the message, and compare the two. If the hashes match it proves that the message wasn’t tampered in progress and also that the first party has indeed signed it as only its private key locked message can be unlocked by the public key. Very cool stuff actually!

Not all public key cryptography algorithms are good at encryption and signing nor are they required to be so. RSA, for instance, is good at encryption & decryption. Another algorithm, DSA (Digital Signature Algorithm) is good at signing & validation. RSA can do signing & validation too but that’s due to the nature of its algorithm

A question of trust

While public key encryption can be used for authentication there is a problem. What happens if a third party publishes its public key on the network but claims that it is the second party. Obviously it’s a fraud but how is the first party to know of that? Two ways really: one way is the first party can perhaps call or through some other means verify with the second party as to what its public key is and thus choose the correct one – this is tricky because it has to verify the identity somehow and be sure about it – or, the second way, there can be some trusted authority that verifies this for everyone – such a trusted authority will confirm that such and such public key really belongs to the second party. 

These two ways of finding if you can trust someone are called the Web of Trust (WoT) and Public Key Infrastructure (PKI) respectively. Both achieve the same thing, the difference being the former is decentralized while the latter is centralized. Both of these make use of something called certificates – which is basically a digital document that contains the public key as well as some information on the owner of the public key (details such as the email address, web address, how long the certificate is valid for, etc). 

Web of Trust (WoT)

In a WoT model everyone uploads their certificates to certain public servers. When a first party searches for certificates belonging to a second party it can find them – both legitimate ones (i.e. actually belonging to the second party) as well as illegitimate ones (i.e. falsely uploaded by other parties claiming to the be second party). By default though the first party doesn’t trust these certificates. It only trusts a certificate once it verifies through some other means as to which is the legitimate one – maybe it calls up the second party or meets the party in person or gets details from its website. The first party can also trust a certificate if someone else it already trusts has marked a certificate as trusted. Thus each party in effect builds up a web of certificates – it trusts a few and it trusts whatever is trusted by the few that it trusts. 

To add to the pool once a party trusts a certificate it can indicate so for others to see. This is done by signing the certificate (which is similar to the signing process I mentioned earlier). So if the first party has somehow verified that a certificate it found really belongs to the second party, it can sign it and upload that information to the public servers. Anyone else then searching for certificates of the second party will come across the true certificate too and find the signature of the first party. 

WoT is what you use with programs such as PGP and GnuPG. 

Public Key Infrastructure (PKI)

In a PKI model there are designated Certificate Authorities (CA) which are trusted by everyone. Each party sends their certificates to the CA to get it signed. The CA verifies that the party is who it claims to be and then signs it. There are many classes of validation – domain validation (the CA verifies that the requester can manage the domain the certificate is for), organization validation (the CA also verifies that the requester actually exists), and extended validation (a much more comprehensive validation than the other two). 

Certificate Authorities have a tree structure. There are certain CAs – called root CAs – which are implicitly trusted by everyone. Their certificates are self-signed or unsigned but trusted by everyone. These root CAs sign certificates of other CAs who in-turn might sign certificates of other CAs or of a requester. Thus there’s a chain of trust – a root CA trust an intermediary CA, who trusts another intermediary CA, who trusts (signs) the certificate of a party. Because of this chain of trust, anyone who trusts the root CA will trust the certificate signed by one of its intermediaries. 

 It’s probably worth pointing out that you don’t really need to get your certificate signed by a CA. For instance, say I want to encrypt all traffic between my computer and this blog and so I create a certificate for the blog. I will be the only person using this – all my regular visitors will visit the blog unencrypted. In such a case I don’t have to bother with them not trusting my certificate. I trust my certificate as I know what its public key and details look like, so I can install the certificate and use an https link when browsing the blog, everyone else can use the regular http link. I don’t need to get it signed by a CA for my single person use. It’s only if I want the general public to trust the certificate that I must involve a CA. 

PKI is what you use with Internet Browsers such as Firefox, Chrome, etc. PKI is also what you use with email programs such as Outlook, Thunderbird, etc to encrypt communication with a server (these emails program may also use WoT to encrypt communication between a sender & recipient). 

TLS/SSL (contd)

From here on I’ll use the words “client” and “server” interchangeably with “first party” and “second party”. The intent is the same, just that it’s easier to think of one of one party as the client and the other as a server. 

TLS/SSL uses both asymmetric and symmetric encryption. TLS/SSL clients use asymmetric encryption to authenticate the server (and vice-versa too if required) and as part of that authentication they also share with each other a symmetric encryption key which they’ll use to encrypt the rest of their conversation. TLS/SSL uses both types of encryption algorithms because asymmetric encryption is computationally expensive (by design) and so it is not practical to encrypt the entire conversation using asymmetric encryption (see this StackExchange answer for more reasons). Better to use use asymmetric encryption to authenticate and bootstrap symmetric encryption. 

When a TLS/SSL client contacts a TLS/SSL the server sends the client its certificate. The client validates it using the PKI. Assuming the validation succeeds, client and server perform a “handshake” (a series of steps) the end result of which is (1) authentication and (2) the establishment of a “session key” which is the symmetric key used for encrypting the rest of the conversation. 

CloudFlare’s blog post on Keyless SSL goes into more details of the handshake. There are two types of handshakes possible: RSA handshakes and Diffie-Hellman handshakes. The two types of handshakes differ in terms of what algorithms are used for authentication and what algorithms are used for session key generation. 

RSA handshake

RSA handshakes are based on the RSA algorithm which I mentioned earlier under public key encryption. 

An RSA handshake uses RSA certificates for authentication (RSA certificates contain a public RSA key). Once authentication is successful, the client creates a random session key and encrypts with the public key of the server (this encryption uses the RSA algorithm). The server can decrypt this session key with its private key, and going forward both client & server use this session key to encrypt further traffic (this encryption does not use the RSA algorithm, it uses one of the symmetric key algorithms).

 The RSA handshake has a drawback in that if someone were to capture and store past encrypted traffic, and if the server’s private key were to somehow leak, then such a person could easily decrypt the session key and thus decrypt the past traffic. The server’s private key plays a very crucial role here as it not only authenticates the server, it also protects the session key. 

Diffie-Hellman handshake

Diffie-Hellman handshakes are based on the Diffie-Hellman key exchange algorithm. 

The Diffie-Hellman key exchange is an interesting algorithm. It doesn’t do any encryption or authentication by itself. Instead, it offers a way for two parties to generate a shared key in public (i.e. anyone can snoop in on the conversation that takes place to generate the secret key) but the shared key is secret and only the two parties know of it (i.e. the third party snooping in on the conversation can’t deduce the shared key). A good explanation of how Diffie-Hellman does this can be found in this blog post. Essentially: (1) the two parties agree upon a large prime number and a smaller number in public, (2) each party then picks a secret number (the private key) for itself and calculates another number (the public key) based on this secret number, the prime number, and the smaller number, (3) the public keys are shared to each other and using each others public key, the prime number, and the small number, each party can calculate the (same) shared key. The beauty of the math involved in this algorithm is that even though a snooper knows the prime number, the small number, and the two public keys, it still cannot deduce the private keys or the shared key!

There are two versions of the Diffie-Hellman algorithm:

  • a fixed/ static version, where both parties use the same public/ private keys (and hence same shared key) across all their conversations; and
  • an ephemeral version, where one party keeps changing its public/ private key (and hence the shared key)

Since the Diffie-Hellman algorithm does not do authentication it needs some other mechanism to authenticate the client and server. It can use an RSA certificate (certificates containing an RSA public key) or a non-RSA certificates – for example DSA certificates (certificates containing a DSA public key) and ECDSA (Elliptic Curve Digital Signature Algorithm) certificates (ECDSA is a variant of DSA that uses Elliptic Curve Cryptography (ECC). ECDSA certificates contain an ECC public key. ECC keys are better than RSA & DSA keys in that the ECC algorithm is harder to break. So not only are ECC keys more future proof, you can also use smaller length keys (for instance a 256-bit ECC key is as secure as a 3248-bit RSA key) and hence the certificates are of a smaller size). 

The fixed/ static version of Diffie-Hellman requires a Diffie-Hellman certificate for authentication (see here and here). Along with the public key of the server, this certificate also contains the prime number and smaller number required by the Diffie-Hellman algorithm. Since these numbers are a part of the certificate itself and cannot change, Diffie-Hellman certificates only work with fixed/ static Diffie-Hellman algorithms and vice-versa. A Diffie-Hellman handshake that uses the fixed/ static Diffie-Hellman algorithm has the same drawback as a RSA handshake. If the server’s private key is leaked, past traffic can be decrypted.  

The ephemeral version of Diffie-Hellman (often referred to as EDH (Ephermeral Diffie-Hellman) or DHE (Diffie-Hellman Ephemeral)) works with RSA certificates, DSA certificates, and ECDSA certificates. EDH/ DEH is computationally expensive as it is not easy to keep generating a new prime number and small number for every connection. A variant of EDH/ DEH that uses elliptic curves – known as Elliptic Curve Diffie-Hellman Ephemeral (ECDHE) – doesn’t have the performance hit of EDH/ DEH and is preferred. 

A Diffie-Hellman handshake that uses EDH/ DEH or ECDHE doesn’t have the drawback of an RSA handshake. The server’s private key is only used to authenticate it, not for generating/ protecting the shared session key. This feature of EDH/DHE and ECDHE wherein the shared keys are generated for each connection and are shared keys themselves are random and independent of each other is known as Perfect Forward Secrecy (PFS). (Perfect Forward Secrecy is a stronger form of Forward Secrecy. The latter does not require the shared keys themselves be independent of each other, only that the shared keys not be related to the server’s private/ public key). 

To be continued …

Writing this post took longer than I expected so I’ll conclude here. I wanted to explore TLS/SSL in the context of Windows and Active Directory, but I got side-tracked talking about handshakes and RSA, ECDHE, etc. Am glad I went down that route though. I was aware of elliptic curves and ECDHE, ECDSA etc. but had never really explored them in detail until now nor written down a cumulative understanding of it all.