CodExt

CodExt is a library (Python 3.10 or later) that extends the native codecs library (namely for adding new custom encodings and character mappings) and provides 120+ new codecs, hence its name combining CODecs EXTension. It also features a guess mode for decoding multiple layers of encoding and CLI tools for convenience.

$ pip install codext

$ codext -i test.txt encode dna-1
GTGAGCGGGTATGTGA

$ echo -en "test" | codext encode morse
- . ... -

$ echo -en "test" | codext encode braille
⠞⠑⠎⠞

$ echo -en "test" | codext encode base100
👫👜👪👫

$ echo -en "Test string" | codext encode reverse
gnirts tseT

$ echo -en "Test string" | codext encode reverse morse
--. -. .. .-. - ... / - ... . -

$ echo -en "Test string" | codext encode reverse morse dna-2
AGTCAGTCAGTGAGAAAGTCAGTGAGAAAGTGAGTGAGAAAGTGAGTCAGTGAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTTAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTGAGAAAGTC

$ echo -en "Test string" | codext encode reverse morse dna-2 octal
101107124103101107124103101107124107101107101101101107124103101107124107101107101101101107124107101107124107101107101101101107124107101107124103101107124107101107101101101107124103101107101101101107124107101107124107101107124107101107101101101107124124101107101101101107124103101107101101101107124107101107124107101107124107101107101101101107124107101107101101101107124103

$ echo -en "AGTCAGTCAGTGAGAAAGTCAGTGAGAAAGTGAGTGAGAAAGTGAGTCAGTGAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTTAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTGAGAAAGTC" | codext -d dna-2 morse reverse
test string

$ codext add-macro my-encoding-chain gzip base63 lzma base64

$ codext list macros
example-macro, my-encoding-chain

$ echo -en "Test string" | codext encode my-encoding-chain
CQQFAF0AAIAAABuTgySPa7WaZC5Sunt6FS0ko71BdrYE8zHqg91qaqadZIR2LafUzpeYDBalvE///ug4AA==

$ codext remove-macro my-encoding-chain

$ codext list macros
example-macro

Playing with base encodings.

$ echo "Test string !" | base122
*.7!ft9�-f9Â

$ echo "Test string !" | base91 
"ONK;WDZM%Z%xE7L

$ echo "Test string !" | base91 | base85
B2P|BJ6A+nO(j|-cttl%

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr
QVx5tvgjvCAkXaMSuKoQmCnjeCV1YyyR3WErUUErFf

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr | base58-flickr -d | base36 -d | base85 -d | base91 -d
Test string !

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr | unbase -m 3
Test string !

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr | unbase -f Test
Test string !

Listing codecs.

$ codext list encodings
a1z26                      adler32               affine             alternative-rot        ascii           
atbash                     autoclave             bacon              barbie                 base            
base1                      base2                 base3              base4                  base8           
<<snipped>>

Finding a codec based on a name.

$ codext search bitcoin
base58

Encoding a string.

$ echo -en "This is a test" | codext encode polybius
44232443 2443 11 44154344

Encoding a file.

$ echo -en "this is a test" > to_be_encoded.txt
$ codext encode base64 < to_be_encoded.txt > text.b64
$ cat text.b64 
dGhpcyBpcyBhIHRlc3Q=

Chaining codecs.

$ echo -en "mrdvm6teie6t2cq=" | codext encode upper | codext decode base32 | codext decode base64
test

Iteratively guessing decodings.

$ echo -en "test" | codext encode base64 gzip | codext guess
Codecs: gzip
dGVzdA==
$ echo -en "test" | codext encode base64 gzip | codext guess gzip -i base
Codecs: gzip, base64
test

Getting the list of available codecs.

>>> import codext

>>> codext.list()
['ascii85', 'base85', 'base100', 'base122', ..., 'tomtom', 'dna', 'html', 'markdown', 'url', 'resistor', 'sms', 'whitespace', 'whitespace-after-before']

Playing with some base encodings.

```python
>>> codext.encode("this is a test", "base58-bitcoin")
'jo91waLQA1NNeBmZKUF'

>>> codext.encode("this is a test", "base58-ripple")
'jo9rA2LQwr44eBmZK7E'

>>> codext.encode("this is a test", "base58-url")
'JN91Wzkpa1nnDbLyjtf'

>>> codecs.encode("this is a test", "base100")
'👫👟👠👪🐗👠👪🐗👘🐗👫👜👪👫'

>>> codecs.decode("👫👟👠👪🐗👠👪🐗👘🐗👫👜👪👫", "base100")
'this is a test'

Playing with some cryptography-based codecs.

>>> codext.encode("This is a test !", "vigenere-MYSECRETKET")
'Ffaw kj e mowm !'

>>> codext.encode("This is a test !", "autoclave-SECRET")
'Llkj ml t amkb !'

Encoding/decoding with various other codecs.

>>> for i in range(8):
        print(codext.encode("this is a test", "dna-%d" % (i + 1)))
GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA
CTCACGGACGGCCTATAGAACGGCCTATAGAACGACAGAACTCACGCCCTATCTCA
ACAGATTGATTAACGCGTGGATTAACGCGTGGATGAGTGGACAGATAAACGCACAG
AGACATTCATTAAGCGCTCCATTAAGCGCTCCATCACTCCAGACATAAAGCGAGAC
TCTGTAAGTAATTCGCGAGGTAATTCGCGAGGTAGTGAGGTCTGTATTTCGCTCTG
TGTCTAACTAATTGCGCACCTAATTGCGCACCTACTCACCTGTCTATTTGCGTGTC
GAGTGCCTGCCGGATATCTTGCCGGATATCTTGCTGTCTTGAGTGCGGGATAGAGT
CACTCGGTCGGCCATATGTTCGGCCATATGTTCGTCTGTTCACTCGCCCATACACT
>>> codext.decode("GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA", "dna-1")
'this is a test'

>>> codecs.encode("this is a test", "morse")
'- .... .. ... / .. ... / .- / - . ... -'

>>> codecs.decode("- .... .. ... / .. ... / .- / - . ... -", "morse")
'this is a test'

>>> with open("morse.txt", 'w', encoding="morse") as f:
	f.write("this is a test")
14

>>> with open("morse.txt",encoding="morse") as f:
	f.read()
'this is a test'

>>> print(codext.encode("An example test string", "baudot-tape"))
***.**
   . *
***.* 
*  .  
   .* 
*  .* 
   . *
** .* 
***.**
** .**
   .* 
*  .  
* *. *
   .* 
* *.  
* *. *
*  .  
* *.  
* *. *
***.  
  *.* 
***.* 
 * .* 

• base1: useless, but for the sake of completeness
• base2: simple conversion to binary (with a variant with a reversed alphabet)
• base3: conversion to ternary (with a variant with a reversed alphabet)
• base4: conversion to quarternary (with a variant with a reversed alphabet)
• base8: simple conversion to octal (with a variant with a reversed alphabet)
• base10: simple conversion to decimal
• base11: conversion to digits with a "a"
• base16: simple conversion to hexadecimal (with a variant holding an alphabet with digits and letters inverted)
• base26: conversion to alphabet letters
• base32: classical conversion according to the RFC4648 with all its variants (zbase32, extended hexadecimal, geohash, Crockford)
• base36: Base36 conversion to letters and digits (with a variant inverting both groups)
• base38: Base38 encoding of the Matter setup QR codes (chunks of 3 bytes)
• base45: Base45 DRAFT algorithm (with a variant inverting letters and digits)
• base58: multiple versions of Base58 (bitcoin, flickr, ripple)
• base62: Base62 conversion to lower- and uppercase letters and digits (with a variant with letters and digits inverted)
• base63: similar to base62 with the "_" added
• base64: classical conversion according to RFC4648 with its variant URL (or file) (it also holds a variant with letters and digits inverted)
• base67: custom conversion using some more special characters (also with a variant with letters and digits inverted)
• base69: Base69 encoding (chunks of 7 bytes, 2 characters per 7 bits)
• base85: all variants of Base85 (Ascii85, z85, Adobe, (x)btoa, RFC1924, XML)
• base91: Base91 custom conversion
• base100 (or emoji): Base100 custom conversion
• base122: Base122 custom conversion
• base128: conversion to the ASCII charset (with control characters)
• base128-latin1: Base128 with printable characters (digits, letters and Latin-1)
• base256: Base256 with one printable character per byte
• base1024 (or ecoji): Ecoji custom conversion (10 bits per emoji, versions 1 and 2)
• base2048: Base2048 custom conversion (11 bits per character, optimised for Twitter)
• base32768: Base32768 custom conversion (15 bits per character, optimised for UTF-16)
• base65536: Base65536 custom conversion (16 bits per character, optimised for UTF-32)
• base131072: Base131072 conversion (17 bits per character), as implemented by the base131072 Rust crate
• bech32: Bech32 encoding with a human-readable part ; bech32m for its Bech32m variant
• leb128 (or varint): LEB128 encoding of lists of integers ; sleb128 for its signed variant
• base-genericN: see base encodings ; supports any possible base

This category also contains ascii85, adobe, [x]btoa, zeromq with the base85 codec.

• baudot: supports CCITT-1, CCITT-2, EU/FR, ITA1, ITA2, MTK-2 (Python3 only), UK, ...
• baudot-spaced: variant of baudot ; groups of 5 bits are whitespace-separated
• baudot-tape: variant of baudot ; outputs a string that looks like a perforated tape
• bcd: Binary Coded Decimal, encodes characters from their (zero-left-padded) ordinals
• bcd-extended0: variant of bcd ; encodes characters from their (zero-left-padded) ordinals using prefix bits 0000
• bcd-extended1: variant of bcd ; encodes characters from their (zero-left-padded) ordinals using prefix bits 1111
• chuck-norris: Chuck Norris unary code (runs of the 7-bit ASCII bits written with blocks of zeros)
• differential-manchester: line code writing the bits as transitions at the start of their periods (also inverted)
• elias-gamma, elias-delta: Elias universal codes, encoding characters from their ordinals (or lists of integers with -integers)
• excess3: uses Excess-3 (aka Stibitz code) binary encoding to convert characters from their ordinals
• fibonacci: Fibonacci universal code, encoding characters from their ordinals (or lists of integers with -integers)
• gray: aka reflected binary code
• hamming: Hamming(7,4) error-correcting code, correcting single-bit errors (and detecting double-bit errors with hamming84)
• manchester: XORes each bit of the input with 01
• manchester-inverted: variant of manchester ; XORes each bit of the input with 10
• nrzi: Non-Return-to-Zero Inverted line code, writing 1 (or 0 with nrzi-space) as a transition
• rotateN: rotates characters by the specified number of bits (N belongs to [1, 7] ; Python 3 only)

• adler: Adler32 algorithm (relies on zlib)
• crc: CRC of lengths 8, 10-17, 21, 24, 30-32, 40, 64, 82 with a variety of polynoms
• fletcher: Fletcher-16, -32 and -64 checksums
• luhn: Luhn mod N algorithm

• a1z26: keeps words whitespace-separated and uses a custom character separator
• cases: set of case-related encodings (including camel-, kebab-, lower-, pascal-, upper-, snake- and swap-case, slugify, capitalize, title)
• dummy: set of simple encodings (including integer, replace, reverse, word-reverse, substite and strip-spaces)
• octal: dummy octal conversion (converts to 3-digits groups)
• octal-spaced: variant of octal ; dummy octal conversion, handling whitespace separators
• ordinal: dummy character ordinals conversion (converts to 3-digits groups)
• ordinal-spaced: variant of ordinal ; dummy character ordinals conversion, handling whitespace separators

• gzip: standard Gzip compression/decompression
• huffman: canonical Huffman coding of the bytes, with a self-describing header
• lz77: compresses the given data with the algorithm of Lempel and Ziv of 1977
• lz78: compresses the given data with the algorithm of Lempel and Ziv of 1978
• pkzip_deflate: standard Zip-deflate compression/decompression
• pkzip_bzip2: standard BZip2 compression/decompression
• pkzip_lzma: standard LZMA compression/decompression
• rle: compresses runs of identical characters with Run-Length Encoding (count before or after each character, compact, bytes or repeated-byte escape variants)
• zstd: Zstandard compression/decompression (relies on compression.zstd from Python 3.14, on backports.zstd before)

⚠️ Compression functions are of course definitely NOT encoding functions ; they are implemented for leveraging the .encode(...) API from codecs.

• adfgx: aka ADFGX Cipher (Polybius square and columnar transposition) ; adfgvx for its 6x6 variant
• aer256: aka AER-256 (toy cipher with float arithmetic, from online JavaScript tools)
• affine: aka Affine Cipher
• armon64: aka ARMON-64 (variant of AER-256 with a fourth operation)
• atbash: aka Atbash Cipher
• atom128: aka ATOM-128 (Base64 with a shuffled alphabet, from online JavaScript tools)
• autoclave: aka Autoclave/Autokey Cipher (variant of Vigenere Cipher)
• bacon: aka Baconian Cipher
• barbie-N: aka Barbie Typewriter (N belongs to [1, 4])
• bazeries: aka Bazeries Cipher (two Polybius grids and a transposition keyed by a number)
• beaufort: aka Beaufort Cipher (variant of Vigenere Cipher)
• bifid: aka Bifid Cipher (Delastelle's fractionation with a 5x5 Polybius square) ; trifid for its 3x3x3 cube variant
• chaocipher: aka Chaocipher (two alphabets permuted after each letter)
• cisco-type7: aka Cisco IOS type 7 password encoding
• citrix: aka Citrix CTX1 password encoding
• columnar: aka Columnar Transposition Cipher (keyword-ordered columns ; a second keyword gives the double transposition)
• enigma: aka Enigma machine (Enigma I, M3 and M4, with reflector, rotors, rings, positions and plugboard)
• esab46: aka ESAB-46 (Base64 with a shuffled alphabet, from online JavaScript tools)
• ezip64: aka EZIP-64 (variant of ARMON-64 with smaller blocks)
• feron74: aka FERON-74 (Base64 with a shuffled alphabet, from online JavaScript tools)
• foursquare: aka Four-square Cipher (bigrams substituted with two keyed squares and two plain ones)
• fractionated-morse: aka Fractionated Morse Cipher (Morse code split into groups of three symbols, each replaced by a letter of a keyed alphabet)
• gila7: aka GILA7 (Base64 with a shuffled alphabet, from online JavaScript tools)
• gronsfeld: aka Gronsfeld Cipher (variant of Vigenere Cipher with a numeric key)
• hill: aka Hill Cipher (blocks of letters multiplied by a key matrix modulo 26)
• nihilist: aka Nihilist Cipher (sums of the Polybius coordinates of the text and of a repeated key)
• playfair: aka Playfair Cipher
• phillips: aka Phillips Cipher (polyalphabetic block cipher with 8 key squares)
• polybius: aka Polybius Square Cipher
• porta: aka Porta Cipher (reciprocal variant of Vigenere Cipher with 13 alphabets)
• railfence: aka Rail Fence Cipher
• rotN: aka Caesar cipher (N belongs to [1,25])
• scytaleN: encrypts using the number of letters on the rod (N belongs to [1,[)
• shiftN: shift ordinals (N belongs to [1,255])
• skip: aka Skip Cipher (transposition reading every N-th character, looping on the text)
• trithemius: aka Trithemius Cipher (variant of Vigenere Cipher)
• twosquare: aka Two-square Cipher or Double Playfair (bigrams substituted with two keyed squares, vertically or horizontally)
• vernam: aka Vernam Cipher (XOR with a repeating key)
• vic: aka VIC Cipher
• vigenere: aka Vigenere Cipher
• xorN: XOR with a single byte (N belongs to [1,255])
• zara128: aka ZARA-128 (variant of AER-256 with additions only and smaller blocks)

⚠️ Crypto functions are of course definitely NOT encoding functions ; they are implemented for leveraging the .encode(...) API from codecs.

• befunge: Befunge-93, two-dimensional esoteric language (encoding generates a program printing the text)
• brainfuck: well-known esoteric programming language (encoding generates a program printing the text)
• church: Church encoding in the lambda calculus (encoding generates a lambda term of the list of the numerals of the bytes)
• cow: esoteric language made of variations of "moo" (Brainfuck-like, with a register and integer output)
• lolcode: esoteric language inspired by lolcats (encoding generates a program printing the text), also with lowercase keywords
• malbolge: esoteric language designed to be as difficult to program in as possible (encoding generates a program printing the text)
• ook: Brainfuck derivative for orang-utans (Ook., Ook? and Ook!), also in its short form
• piet: esoteric language whose programs are images looking like abstract paintings (encoding generates a PNG image of a program printing the text)
• pikalang: Brainfuck derivative using Pikachu's vocabulary

• coquelet: Coquelet 13 and 8 (aka Mk 1 and Mk 2), radio teleprinter systems sending each ITA-2 character as two audio tones (gives their frequencies in Hz)
• dtmf: Dual-Tone Multi-Frequency telephone keypad signaling (gives the frequencies of the tones in Hz, or a WAV recording with dtmf-wav, decoded with the Goertzel algorithm)
• piccolo: Piccolo I, II (aka Mk 6) and III, radio teleprinter systems sending each ITA-2 character as one or two audio tones (gives their frequencies in Hz)

• bcrypt: aka bcrypt password hash (relies on bcrypt when available, e.g. with pip install codext[bcrypt], pure Python otherwise)
• blake: includes BLAKE2b and BLAKE2s (Python 3 only ; relies on hashlib)
• crypt: Unix's crypt hash for passwords (Python 3 and Unix only ; relies on crypt)
• fnv: aka Fowler-Noll-Vo ; includes FNV-1 and FNV-1a in 32, 64, 128, 256, 512 and 1024 bits
• keccak: original Keccak (pre-SHA-3 padding, as used e.g. by Ethereum) in 224, 256, 384 and 512 bits
• lm: aka LAN Manager hash, the legacy Windows password hash (with a pure Python DES)
• md: aka Message Digest ; includes MD2, MD4 (relies on hashlib when available, pure Python otherwise), MD5 (relies on hashlib) and MD6 (digest sizes multiple of 8, up to 512 bits)
• mysql: MySQL password hashes ; includes the MySQL 3.23 hash and the MySQL 4.1+ native password hash
• ntlm: aka NT hash, the Windows password hash (MD4 of the UTF-16LE password)
• ripemd160: aka RIPEMD-160 (relies on hashlib when available, pure Python otherwise)
• sha: aka Secure Hash Algorithms ; includes SHA0, SHA1, 224, 256, 384, 512 (Python2/3) but also SHA3-224, -256, -384 and -512 (Python 3 only ; relies on hashlib)
• shake: aka SHAKE hashing (Python 3 only ; relies on hashlib)
• snefru: aka Snefru (Merkle's hash function) in 32 to 480 bits, with 2 to 8 passes
• whirlpool: aka Whirlpool (relies on hashlib when available, pure Python otherwise)

⚠️ Hash functions are of course definitely NOT encoding functions ; they are implemented for convenience with the .encode(...) API from codecs and useful for chaning codecs.

• braille: well-known braille language (Python 3 only)
• fancy: Unicode styled alphabets (mathematical bold, italic, script, fraktur, double-struck, ..., circled and parenthesized letters), also fullwidth and smallcaps
• ipsum: aka lorem ipsum
• galactic: aka Standard Galactic Alphabet or Minecraft enchantment language (as written by most online translators)
• leetspeak: based on minimalistic elite speaking rules
• morse: uses whitespace as a separator
• navajo: only handles letters (not full words from the Navajo dictionary)
• radio: aka NATO or radio phonetic alphabet
• southpark: converts letters to Kenny's language from Southpark (whitespace is also handled)
• southpark-icase: case insensitive variant of southpark
• tap: converts text to tap/knock code, commonly used by prisoners (as written by cryptii)
• tomtom: similar to morse, using slashes and backslashes
• upsidedown: upside-down (flip) text, with turned Unicode characters in reverse order

• asn1: readable dump of ASN.1 BER/DER elements (e.g. certificates and keys), encoded back to DER
• ber: text as an ASN.1 BER string element (decoding indefinite lengths and segmented strings)
• binhex: BinHex 4.0 (.hqx), the binary-to-text encoding of the classic Mac OS
• der: text as an ASN.1 DER string element (UTF8String)
• dna: implements the 8 rules of DNA sequences (N belongs to [1,8])
• dvorak: text typed with the Dvorak keyboard layout instead of QWERTY
• letter-indices: encodes consonants and/or vowels with their corresponding indices
• markdown: unidirectional encoding from Markdown to HTML
• soundex: Soundex phonetic codes (decoding gives English words with the same codes)
• telegraph: Chinese telegraph code, with the codebooks of Mainland China and Taiwan
• xx: XXencode, a UUencode variant with an alphanumeric alphabet (+, -, digits and letters)
• yenc: Usenet binary-to-text encoding (raw, wrapped or framed with header, trailer and CRC32)

• hexagram: uses Base64 and encodes the result to a charset of I Ching hexagrams (as implemented here)
• klopf: aka Klopf code ; Polybius square with trivial alphabetical distribution
• resistor: aka resistor color codes
• rick: aka Rick cipher (in reference to Rick Astley's song "Never gonna give you up")
• sms: also called T9 code ; uses "-" as a separator for encoding, "-" or "_" or whitespace for decoding
• unicode-tags: aka ASCII smuggling ; hides ASCII text in invisible Unicode tag characters
• whitespace: replaces bits with whitespaces and tabs
• whitespace_after_before: variant of whitespace ; encodes characters as new characters with whitespaces before and after according to an equation described in the codec name (e.g. "whitespace+2*after-3*before")
• zero-width: hides text in invisible zero-width characters (binary, ternary and quinary variants, compatible with zwsp-steg)

• aaencode: JavaScript obfuscation with Japanese-style emoticons (encoding generates a program running the code)
• html: implements entities according to this reference
• jjencode: JavaScript obfuscation with only symbols (encoding generates a program running the code)
• jsfuck: JavaScript with only six characters ([]()!+) ; encoding generates an expression giving the text, or a program running the code with jsfuck-eval
• url: aka URL encoding