LLM Steganography Demo

This demo lets you hide a secret message inside ordinary-looking AI-generated text. The output reads like a normal sentence a language model might produce, but it secretly encodes your message bit by bit.

The practical point is covert communication: two people who share a key can exchange messages that, to anyone watching, look like innocuous GPT-2 text. There's no obvious ciphertext, no encrypted file, and no metadata screaming "this is encrypted." The secret is hidden in plain sight.

It's also a neat demonstration of how much information is packed into every token a language model emits — each token can carry a full secret bit while still looking natural.

This demo uses a half-splitting entropy coding scheme built on top of GPT-2's next-token probability distribution.

1. The secret message is converted to a bit string (UTF-8 → bits).
2. For each secret bit, GPT-2 produces a probability distribution over the entire vocabulary for the next token.
3. Tokens are sorted by probability (descending) and split into two halves at the 50% cumulative probability mark.
4. Bit 0 → the next token is sampled from the first (higher-probability) half; bit 1 → from the second half.
5. A shared-key PRNG (mulberry32) selects the exact token within the chosen half, so the decoder can reproduce the same random draws.
6. The decoder re-runs GPT-2 on the same context, observes which half each cover token fell into, and recovers the bits → original message.

Because both sides share the same model, context, and PRNG seed, the decoder can perfectly reconstruct the hidden bits. The resulting cover text reads like normal GPT-2 output, hiding the secret in plain sight.

Suppose the context is "I like to eat" and your secret message is "HI". The letter H is byte 0x48 = bits 01001000, and I is 0x49 = bits 01001001. So the encoder needs to hide 16 bits total.

For the first bit (0), GPT-2 looks at "I like to eat" and ranks every possible next token by probability. It splits that ranked list at the 50% cumulative-probability mark into a "high-prob" half and a "low-prob" half. Since the bit is 0, the encoder samples a token from the high-prob half — say " pizza". The cover text is now "I like to eat pizza".

For the second bit (1), GPT-2 now looks at "I like to eat pizza" and produces a fresh distribution. The bit is 1, so the encoder samples from the low-prob half — maybe " and". Cover text: "I like to eat pizza and".

This continues for all 16 bits, then a few padding tokens are added so the text ends naturally. The final cover text might read something like:

"I like to eat pizza and pasta with my friends on weekends."

To anyone else, that's just a normal sentence. But the decoder — who knows the same context, key, and model — re-runs GPT-2 at each step, checks which half each token fell into, and recovers the bits 01001000 01001001 → "HI".

This project is inspired by research into linguistic steganography — the art of hiding information within natural language. Traditional steganography hides data in images, audio, or file metadata, but text-based steganography is harder because text is compact and every character is visible.

The breakthrough idea is to use a language model's own probability distribution as the carrier channel. Instead of tweaking pixels or LSBs, you steer which token the model emits next, encoding one secret bit per token. Key inspirations include:

• Entropy coding approaches (e.g. Cachin's information-theoretic steganography) that treat the cover as a probability distribution and map secret bits onto it.
• "Dissociated Press" / Markov-chain steganography — early experiments that used statistical text generation to hide messages, limited by the poor quality of pre-LLM language models.
• Modern LLM steganography research — papers that use GPT-style models to achieve near-perfect cover text quality while encoding multiple bits per token via arithmetic or half-splitting codes.

The half-splitting scheme used here is one of the simplest robust approaches: it guarantees exactly one bit per token, is symmetric (encoder and decoder run the same logic), and produces text that's indistinguishable from normal model output to a casual reader.