ASN.1 objects: binary DER- and BER-encoded files vs PEM-encoded text files

Almost all the security-related files used by this toolkit - X.509 certificates, key files, CMS data files and so forth - contain ASN.1 objects encoded in BER- or DER-encoding. You'll also come across PEM-encoded files. This section attempts to explain the differences and similarities between these terms.

Some definitions

ASN.1 (Abstract Syntax Notation One)

ASN.1 (Abstract Syntax Notation One) is a formal but flexible notation that allows you to specify abstract objects used in high-level telecommunications protocols. You can build up definitions of complex types using simple types like integers and bit strings together with structured types such as sets and sequences.

Basic Encoding Rules (BER)

BER describes how to represent or encode values of each ASN.1 type as a string of eight-bit octets. There is generally more than one way to BER-encode a given value.

Distinguished Encoding Rules (DER)

Distinguished Encoding Rules (DER) is a subset of BER which gives a unique encoding to each ASN.1 value.

For most purposes here, you can consider "BER-encoded" and "DER-encoded" to mean the same thing.

DER- and BER-encoded files are binary files. A PEM file is an alternative way to encode the same data in a text format (these are also referred to as "Textual Encodings", see [RFC7468]). PEM encoding is the base64 encoding [RFC4648] of the DER-encoded data sandwiched between the encapsulating boundaries of the form "-----BEGIN XXX-----" and "-----END XXX-----". For example,

-----BEGIN CERTIFICATE-----
MIHgMIGaAgEBMA0GCSqG...
-----END CERTIFICATE-----

PEM-encoded files are conventionally meant to be named .pem but can have .txt or the same extension as their DER-encoded sibling, e.g. .cer. We don't care. Having a .txt extension makes it easy to examine the file. Note that we ignore the actual label in the encapsulation: we decode the inner base64 and examine the decoded bytes directly. Other applications may not be so tolerant.

In this toolkit, where the input requires a file, we accept both DER-encoded binary files and their equivalent PEM-encoded text files automatically. We also in general accept the same data files encoded with BER-encoding. In addition, you can pass the PEM-encoded data in a string (a PEM String) instead of passing a filename. See PEM string alternative.

Exception

We do not accept the really old encrypted form of PEM files with "Proc-Type" parameters that look like

-----BEGIN EC PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,085FFED98A156DBB

hPvIVkMx6lMS3JB6so8d8JHt+...
-----END EC PRIVATE KEY-----

But we do accept the modern form with just base64 characters sandwiched between the "-----BEGIN XXX-----" and "-----END XXX-----" boundaries, for example.

-----BEGIN ENCRYPTED PRIVATE KEY-----
MIICojAcBgoqhkiG9w0BDAEDMA4ECHPQz6NdAmoFAgIH0ASCAoBKn9KXr+dm
Vtc0ZhEog7t3Prs4rJazwUsXExU78ePLMquxLi/cPmqtyjb472r6XUOa...
-----END ENCRYPTED PRIVATE KEY-----

Example

RSAPublicKey ::= SEQUENCE {
    modulus           INTEGER,  -- n
    publicExponent    INTEGER   -- e
}

  0  71: SEQUENCE {
  2  64: . INTEGER
       : . . 0A 66 79 1D C6 98 81 68 DE 7A B7 74 19 BB 7F B0
       : . . C0 01 C6 27 10 27 00 75 14 29 42 E1 9A 8D 8C 51
       : . . D0 53 B3 E3 78 2A 1D E5 DC 5A F4 EB E9 94 68 17
       : . . 01 14 A1 DF E6 7C DC 9A 9A F5 5D 65 56 20 BB AB
 68   3: . INTEGER 65537
       : . }

30 47  --SEQUENCE/71=0x47
   02 40  --INTEGER/64=0x40
      0a 66 79 1d c6 98 81 68 de 7a b7 74 19 bb 7f b0
      c0 01 c6 27 10 27 00 75 14 29 42 e1 9a 8d 8c 51
      d0 53 b3 e3 78 2a 1d e5 dc 5a f4 eb e9 94 68 17
      01 14 a1 df e6 7c dc 9a 9a f5 5d 65 56 20 bb ab
   02 03  --INTEGER/3=0x3
      01 00 01
      --65537
--(73 bytes)

000000  30 47 02 40 0a 66 79 1d c6 98 81 68 de 7a b7 74  0G.@.fy....h.z.t
000010  19 bb 7f b0 c0 01 c6 27 10 27 00 75 14 29 42 e1  .......'.'.u.)B.
000020  9a 8d 8c 51 d0 53 b3 e3 78 2a 1d e5 dc 5a f4 eb  ...Q.S..x*...Z..
000030  e9 94 68 17 01 14 a1 df e6 7c dc 9a 9a f5 5d 65  ..h......|....]e
000040  56 20 bb ab 02 03 01 00 01                       V .......

-----BEGIN RSA PUBLIC KEY-----
MEcCQApmeR3GmIFo3nq3dBm7f7DAAcYnECcAdRQpQuGajYxR0FOz43gqHeXcWvTr
6ZRoFwEUod/mfNyamvVdZVYgu6sCAwEAAQ==
-----END RSA PUBLIC KEY-----

All ASN.1 objects used for security applications consist of an outer SEQUENCE wrapping the data inside. That means the first byte in a binary DER-encoded object will always be 0x30 (the tag value for a SEQUENCE), and the first letter in a base64/PEM encoding will be the letter capital 'M'.

[Contents] [Index]