In the span of just weeks, the US government has experienced what may be the most consequential security breach in its history—not through a sophisticated cyberattack or an act of foreign espionage, but through official orders by a billionaire with a poorly defined government role. And the implications for national security are profound.

First, it was reported that people associated with the newly created Department of Government Efficiency (DOGE) had accessed the US Treasury computer system, giving them the ability to collect data on and potentially control the department’s roughly $5.45 trillion in annual federal payments.

Then, we learned that uncleared DOGE personnel had gained access to classified data from the US Agency for International Development, possibly copying it onto their own systems. Next, the Office of Personnel Management—which holds detailed personal data on millions of federal employees, including those with security clearances—was compromised. After that, Medicaid and Medicare records were compromised.

Meanwhile, only partially redacted names of CIA employees were sent over an unclassified email account. DOGE personnel are also reported to be feeding Education Department data into artificial intelligence software, and they have also started working at the Department of Energy.

This story is moving very fast. On Feb. 8, a federal judge blocked the DOGE team from accessing the Treasury Department systems any further. But given that DOGE workers have already copied data and possibly installed and modified software, it’s unclear how this fixes anything.

In any case, breaches of other critical government systems are likely to follow unless federal employees stand firm on the protocols protecting national security.

The systems that DOGE is accessing are not esoteric pieces of our nation’s infrastructure—they are the sinews of government.

For example, the Treasury Department systems contain the technical blueprints for how the federal government moves money, while the Office of Personnel Management (OPM) network contains information on who and what organizations the government employs and contracts with.

What makes this situation unprecedented isn’t just the scope, but also the method of attack. Foreign adversaries typically spend years attempting to penetrate government systems such as these, using stealth to avoid being seen and carefully hiding any tells or tracks. The Chinese government’s 2015 breach of OPM was a significant US security failure, and it illustrated how personnel data could be used to identify intelligence officers and compromise national security.

In this case, external operators with limited experience and minimal oversight are doing their work in plain sight and under massive public scrutiny: gaining the highest levels of administrative access and making changes to the United States’ most sensitive networks, potentially introducing new security vulnerabilities in the process.

But the most alarming aspect isn’t just the access being granted. It’s the systematic dismantling of security measures that would detect and prevent misuse—including standard incident response protocols, auditing, and change-tracking mechanisms—by removing the career officials in charge of those security measures and replacing them with inexperienced operators.

The Treasury’s computer systems have such an impact on national security that they were designed with the same principle that guides nuclear launch protocols: No single person should have unlimited power. Just as launching a nuclear missile requires two separate officers turning their keys simultaneously, making changes to critical financial systems traditionally requires multiple authorized personnel working in concert.

This approach, known as “separation of duties,” isn’t just bureaucratic red tape; it’s a fundamental security principle as old as banking itself. When your local bank processes a large transfer, it requires two different employees to verify the transaction. When a company issues a major financial report, separate teams must review and approve it. These aren’t just formalities—they’re essential safeguards against corruption and error. These measures have been bypassed or ignored. It’s as if someone found a way to rob Fort Knox by simply declaring that the new official policy is to fire all the guards and allow unescorted visits to the vault.

The implications for national security are staggering. Sen. Ron Wyden said his office had learned that the attackers gained privileges that allow them to modify core programs in Treasury Department computers that verify federal payments, access encrypted keys that secure financial transactions, and alter audit logs that record system changes. Over at OPM, reports indicate that individuals associated with DOGE connected an unauthorized server into the network. They are also reportedly training AI software on all of this sensitive data.

This is much more critical than the initial unauthorized access. These new servers have unknown capabilities and configurations, and there’s no evidence that this new code has gone through any rigorous security testing protocols. The AIs being trained are certainly not secure enough for this kind of data. All are ideal targets for any adversary, foreign or domestic, also seeking access to federal data.

There’s a reason why every modification—hardware or software—to these systems goes through a complex planning process and includes sophisticated access-control mechanisms. The national security crisis is that these systems are now much more vulnerable to dangerous attacks at the same time that the legitimate system administrators trained to protect them have been locked out.

By modifying core systems, the attackers have not only compromised current operations, but have also left behind vulnerabilities that could be exploited in future attacks—giving adversaries such as Russia and China an unprecedented opportunity. These countries have long targeted these systems. And they don’t just want to gather intelligence—they also want to understand how to disrupt these systems in a crisis.

Now, the technical details of how these systems operate, their security protocols, and their vulnerabilities are now potentially exposed to unknown parties without any of the usual safeguards. Instead of having to breach heavily fortified digital walls, these parties  can simply walk through doors that are being propped open—and then erase evidence of their actions.

The security implications span three critical areas.

First, system manipulation: External operators can now modify operations while also altering audit trails that would track their changes. Second, data exposure: Beyond accessing personal information and transaction records, these operators can copy entire system architectures and security configurations—in one case, the technical blueprint of the country’s federal payment infrastructure. Third, and most critically, is the issue of system control: These operators can alter core systems and authentication mechanisms while disabling the very tools designed to detect such changes. This is more than modifying operations; it is modifying the infrastructure that those operations use.

To address these vulnerabilities, three immediate steps are essential. First, unauthorized access must be revoked and proper authentication protocols restored. Next, comprehensive system monitoring and change management must be reinstated—which, given the difficulty of cleaning a compromised system, will likely require a complete system reset. Finally, thorough audits must be conducted of all system changes made during this period.

This is beyond politics—this is a matter of national security. Foreign national intelligence organizations will be quick to take advantage of both the chaos and the new insecurities to steal US data and install backdoors to allow for future access.

Each day of continued unrestricted access makes the eventual recovery more difficult and increases the risk of irreversible damage to these critical systems. While the full impact may take time to assess, these steps represent the minimum necessary actions to begin restoring system integrity and security protocols.

Assuming that anyone in the government still cares.

This essay was written with Davi Ottenheimer, and originally appeared in Foreign Policy.

Steel rulers are a mainstay in many a workshop. From a design standpoint, you'd think there isn't much to them: Add a satin finish so they don't glare under shop lights, and make the numbers easy to read. With their "Pick Up" line of rulers, Japanese manufacturer Shinwa has met these first two criteria…

…and they've also added a brilliant third feature we've all needed, but never thought much about.

If you've ever struggled to pick up a ruler that isn't at the edge of your workbench, you'll appreciate it. The ruler has a slightly kicked-up tail, like on a skateboard. Once it's tilted back, you can easily grab the thing, even if you've just cut your nails.

To me, that's a designer or engineer going the extra mile. (Er, kilometer.)

The days that PDFs were the granny-proof Swiss Army knives of document sharing are definitely over, according to [vk6]. He has managed to pull off the ultimate mind-bender: running Linux inside a PDF file. Yep, you read that right. A full Linux distro chugging along in a virtual machine all encapsulated within a document. Just when you thought running DOOM was the epitome of it. You can even try it out in your own browser, right here. Mind-boggling, or downright Pandora’s box?

Let’s unpack how this black magic works. The humble PDF file format supports JavaScript – with a limited standard library, mind you. By leveraging this, [vk6] managed to compile a RISC-V emulator (TinyEMU) into JavaScript using an old version of Emscripten targeting asm.js instead of WebAssembly. The emulator, embedded within the PDF, interfaces with virtual input through a keyboard and text box.

The graphical output is ingeniously rendered as ASCII characters – each line displayed in a separate text field. It’s a wild solution but works astonishingly well for something so unconventional.

Security-wise, this definitely raises eyebrows. PDFs have long been vectors for malware, but this pushes things further: PDFs with computational power. We know not to trust Word documents, whether they just capable of running Doom, or trash your entire system in a blink. This PDF anomaly unfolds a complete, powerful operating system in front of your very eyes. Should we think lightly, and hope it’ll lead to smarter, more interactive PDFs – or will it bring us innocent looking files weaponized for chaos?

Curious minds, go take a look for yourself. The project’s code is available on GitHub.

Once, I got to look into a vault with billions of dollars inside. That’s enough money to power the entire economy of Latvia, Iceland, or El Salvador for an entire year. The vault itself was buried underneath the ground in Uptown Charlotte. It stood three stories high and was the width of two basketball courts. To get there, I had to pass through layers upon layers of security. Cameras were everywhere. The walls were made from thick steel. I was not allowed to step into this vault. Few humans are.

Before we go on, I want you to stop for a moment. Close your eyes. Think about that last paragraph. Imagine what this room might look like. (I’m not saying that you’re me, but every time I do this, my brain conjures up Scrooge McDuck’s vault from the opening of DuckTales. It also conjures up the DuckTales song to go along with it. Music is powerful.)

Okay. Got it?

Now open your eyes.

And take a look at this boring-ass room:

Look at it! Not a single piece of gold! Not one damn sack with a dollar sign on it! No glittering doubloons or bearer bonds! No field of Tom Cruise-proof lasers! Not one buff dude in Oakleys holding a machine gun! Still, there are tens of billions of dollars in that room. It’s possible that you could crack one of those bins open and find $28 million. That’s enough money to support you, your family, and your descendants forever. Hell, that might be true for your whole neighborhood. And yet, look at how beige everything is. Ew.

Subscribe now

What you’re looking at is the Federal Reserve’s cash vault in Charlotte. Back in 2018, I did a whole podcast episode about it. And from time to time, I bust out a picture of this place just to impress people. The most recent victim was my son. He was not impressed. Every time I look at this picture, a picture that literally depicts enough money to buy more than a hundred fighter jets with a few billion dollars left over, I want to yawn. It makes me sleepy.

But that’s okay. This vault is not there to impress you. It’s not there to make you cower in awe. It’s there because it needs to be. It’s there to make stuff work.

I don’t know about you, but I’m a bit paralyzed right now. There is so much news coming out of Washington, and almost all of it is bad. Where do you even start? What can I possibly address right now that won’t be replaced by another, more depressing story tomorrow?

I haven’t been able to come up with some thoughtful way to express what I’m feeling lately. I have hope. But also dread. There’s a lot on my mind, but if there’s one broad worry that I have at the moment, it’s this: What happens if stuff stops working?

I’m not naive enough to think that everything works now. I know that even though it’s on the menu, ordering a McFlurry is a dicey proposition. I know that mechanics can’t seem to make the check engine light on my Chevy turn off, even though it seems to be running just fine. I know that no matter how much I recycle, I can’t keep all trash out of the ocean. And I know that, as a white man who lives in a suburban spot of North Carolina with his family, I’m privileged. A lot of things that work for me may not work for you.

Share

But take, for instance, the amount of things that work without you thinking about them. Order something from Amazon. Try and think about the process of getting an individual package from some far flung warehouse to you. How much computing power it requires. How many people are involved. How it changes the calculus about how many things you buy, the convenience of finding products you once couldn’t find, and the stores you no longer visit as a result.

Or, go to a park, and ponder the effort necessary to carve out and maintain a space that’s free for you to enjoy. Maybe get in the car, and consider the logistics of building a smooth stretch of pavement between where you currently are to anywhere you might want to go. Turn on your water. Flip on a light. Eat a sandwich. Send a text message. Take an aspirin. If you had to stop and think about how anything worked before you used it, you’d never get anything done. Or maybe you’d be overly grateful. After years of trying to garden, I can’t help but see the produce section at the grocery story as a miracle. Do you know how hard it was for me to grow one stupid watermelon a few years ago? Really, really hard! And I don’t even like watermelon!

It’s easy to take the sheer volume of this stuff for granted until it’s gone. In 2018, Hurricane Michael washed out a bridge over the creek that separates Oak Ridge from Summerfield. It took months to replace. In an instant, a three-mile, five minute drive to my in-laws’ house became a 10-mile, 15 minute trip. It changed all sorts of routines and plans. I never thought about that bridge until one day, it wasn’t there. I was grateful when it finally re-opened. Then I stopped thinking about it. Why? Because it’s supposed to work.

It’s important to understand how things work, even if you don’t think about them a lot. At the moment, we’re all learning a lot about how our federal government functions. About the people and processes and protections that are supposed to make everything keep working. If there’s any silver lining to what’s happening, it’s that we’re beginning to understand just how big of an effort it is, imperfect or not.

Which brings me back to the vault. When I was there in 2018, I got a tour from Kelly Stewart, who ran the cash operations at the Charlotte Fed. She was very nice, but I got the impression that she wasn’t all that impressed with my DuckTales references and my questions about aesthetics. Instead, she said, you have to understand the purpose of this place.

For example: Let’s say there’s a hurricane coming. After it hits, the power’s going to be out. ATMs won’t work. Debit and credit cards would be useless. People are still going to need money to live their lives. So in those instances, the Charlotte Fed would make sure that banks in North and South Carolina would have enough physical cash on hand to meet their customers’ demand (The Fed, basically, is a bank for banks). After a storm, people are going to have wet, soggy, or ripped bills. Those bills get deposited at banks, and the banks send them to the Fed, where they’re fed into a machine. If a bill is in good enough shape, it comes out the other side. If it’s not, it’s shredded on the spot. Some of the resulting currency confetti is given away as tchotchkes:

The Fed does all sort of stuff. It sets interest rates, monitors inflation, and checks on the health of the overall economy. It also (still) clears paper checks. (If you’d like to know more about how the Charlotte Fed works, Mark Washburn wrote a great series of stories for The Charlotte Ledger last year.) But mostly, it helps provide part of the full faith and credit behind every single $1, $5, $10, $20, $50, and $100 bill out there. Cash has to go where’s needed. It takes a lot of people, planning, and trust to make it all happen. A lot of work happens behind the scenes to make that you don’t have to stop and consider the logistics of money every time you want to use it.

Stewart explained it to me this way:

The great thing about really a lot of payment options is that we, as consumers, don't have to think about them when we're using them. So, if we go to an ATM, we expect cash to come out. And we know what cash is going to do when we use it. We don't think about it when we swipe a debit card. We don't really think about how the transaction flows, or the inner workings behind it. We just trust that it's going to work when our paycheck gets deposited into our accounts. You don't have to think about your payment choices. You trust that they'll work, but they work because we work. That's key.

After my tour, I walked out of the Federal Reserve building in Charlotte (which is the most ‘90s-looking building in a city full of ‘90s-looking buildings) and found myself standing on a sidewalk on East Trade Street. Cars drove by. A few people walked past, heading to lunch or back to the office. Most of them probably were not thinking, or blissfully unaware, of the billions of dollars beneath their feet. I was thinking of all of the people down there, just out of sight, making sure that I don’t have to think about them. Things work because they work.

Leave a comment

Hello there! If you follow tech news, you might have heard about the Okta security incident that was reported on 1st of November. The TLDR of the incident was this:

The Bcrypt algorithm was used to generate the cache key where we hash a combined string of userId + username + password. Under a specific set of conditions, listed below, this could allow users to authenticate by providing the username with the stored cache key of a previous successful authentication.

This means that if the user had a username above 52 chars, any password would suffice to log in. Also, if the username is, let’s say, 50 chars long, it means that the bad actor needs to guess only 3 first chars to get in, which is quite a trivial task for the computers these days. Too bad, isn’t it?

On the other hand, such long usernames are not very usual, which I agree with. However, some companies like using the entire name of the employee as the email address. So, let’s say, Albus Percival Wulfric Brian Dumbledore, a headmaster of Hogwarts, should be concerned, as [email protected] is 55 chars. Ooops!

This was possible due to the nature of Bcrypt hashing algorithm that has a maximum supported input length of 72 characters (read more here), so in Okta case the characters above the limit were ignored while computing the hash, and therefore, not used in the comparison operation. We can reverse engineer that:

• 72 - 53 = 19 - user id with separators if any
• this way, the password will be outside the 72 chars limit, and, therefore, ignored by the Bcrypt algorithm

However, there was one thing that made me wonder: if there is a known limit of the algorithm, why is it not enforced by the crypto libraries as a form of input validation? A simple if input length > 72 -> return error will do the trick. I assumed that they might have used some custom library for Bcrypt implementation and simply forgotten about the input validation, which can happen. So, I decided to check how other programming languages behave.

Go and Bcrypt

Let’s start with Go, and implement the Okta incident-like case with the help of the official golang.org/x/crypto/bcrypt library:

package main

import (
	"crypto/rand"
	"encoding/base64"
	"fmt"
	"golang.org/x/crypto/bcrypt"
)

func main() {
	// 18 + 55 + 1 = 74, so above 72 characters' limit of BCrypt
	userId := randomString(18)
	username := randomString(55)
	password := "super-duper-secure-password"

	combinedString := fmt.Sprintf("%s:%s:%s", userId, username, password)

	combinedHash, err := bcrypt.GenerateFromPassword([]byte(combinedString), bcrypt.DefaultCost)
	if err != nil {
		panic(err)
	}

	// let's try to break it
	wrongPassword := "wrong-password"
	wrongCombinedString := fmt.Sprintf("%s:%s:%s", userId, username, wrongPassword)

	err = bcrypt.CompareHashAndPassword(combinedHash, []byte(wrongCombinedString))
	if err != nil {
		fmt.Println("Password is incorrect")
	} else {
		fmt.Println("Password is correct")
	}
}

func randomString(length int) string {
	bytes := make([]byte, length)
	_, err := rand.Read(bytes)
	if err != nil {
		panic(err)
	}
	return base64.URLEncoding.EncodeToString(bytes)[:length]
}

All the code samples can be found here

What this code does is:

• generates 18-chars long userId
• generates 55-chars long username
• concatenates them with each other and a dummy password super-duper-secure-password with the use of : as a separator
• computes Bcrypt hash from the concatenated string
• then concatenates the same userId and username with a different password wrong-password
• uses bcrypt API to compare whether the 2nd concatenated string matches the hash of the 1st one

Let’s run the code and see the result:

panic: bcrypt: password length exceeds 72 bytes

goroutine 1 [running]:
main.main()
	/n0rdy-blog-code-samples/20250121-bcrypt-api/01-bcrypt-in-go/main.go:20 +0x2d1

Good job, Go! If we check the source code of the bcrypt.GenerateFromPassword(...) function, we’ll see this piece of code at the very beginning:

if len(password) > 72 {
	return nil, ErrPasswordTooLong
}

Perfect! At this point, I became even more suspicious about the tool Okta used, as it seemed like the industry figured that out based on this example. Spoiler alert: it’s not that simple.

Let’s proceed with Java.

Btw, if you like my blog and don’t want to miss out on new posts, consider subscribing to my newsletter here. You’ll receive an email once I publish a new post.

Java and Bcrypt

Java doesn’t support Bcrypt from its core API, but my simple Google search showed that Spring Security library has implemented it. For those who are not into Java ecosystem, Spring is the most used and battle-tested frameworks out there, that has libraries for almost anything: Web, DBs, Cloud, Security, AI, etc. Pretty powerful tool, that I’ve used a lot in the past, and still sometimes use for my side projects.

Spring Security

So, I added the latest version of Spring Security to the project and reproduced the same scenario, as in Go example above:

import org.apache.commons.lang3.RandomStringUtils;
import org.springframework.security.crypto.bcrypt.BCrypt;

public class BcriptSpringSecurity {
    public static void main(String[] args) {
        // 18 + 55 + 1 = 74, so above 72 characters' limit of BCrypt
        var userId = RandomStringUtils.randomAlphanumeric(18);
        var username = RandomStringUtils.randomAlphanumeric(55);
        var password = "super-duper-secure-password";

        var combinedString = String.format("%s:%s:%s", userId, username, password);

        var combinedHash = BCrypt.hashpw(combinedString, BCrypt.gensalt());

        // let's try to break it
        var wrongPassword = "wrong-password";
        var wrongCombinedString = String.format("%s:%s:%s", userId, username, wrongPassword);

        if (BCrypt.checkpw(wrongCombinedString, combinedHash)) {
            System.out.println("Password is correct");
        } else {
            System.out.println("Password is incorrect");
        }
    }
}

I ran the code, and to my great surprise, saw this outcome:

Password is correct

I took a peak at the implementation code, and was disappointed: even though there are a bunch of checks on salt:

if (saltLength < 28) {
	throw new IllegalArgumentException("Invalid salt");
}
...
if (salt.charAt(0) != '$' || salt.charAt(1) != '2') {
	throw new IllegalArgumentException("Invalid salt version");
}
...
minor = salt.charAt(2);
if ((minor != 'a' && minor != 'x' && minor != 'y' && minor != 'b') || salt.charAt(3) != '$') {
	throw new IllegalArgumentException("Invalid salt revision");
}
...

I didn’t see any validation of the input that will be hashed. Hm…

I decided to check other Google results, and the next Java library in the list was bcrypt from Patrick Favre (link to GitHub repo) with 513 starts and the last release version 0.10.2 (so, not stable) from 12th of February 2023 (almost 2 years old). This suggested that I’d not use it in production, but why not to run our tests.

Bcrypt from Patrick Favre

import at.favre.lib.crypto.bcrypt.BCrypt;
import org.apache.commons.lang3.RandomStringUtils;

public class BcryptAtFavre {

    public static void main(String[] args) {
        // 18 + 1 + 55 = 74, so above 72 characters' limit of BCrypt
        var userId = RandomStringUtils.randomAlphanumeric(18);
        var username = RandomStringUtils.randomAlphanumeric(55);
        var password = "super-duper-secure-password";

        var combinedString = String.format("%s:%s:%s", userId, username, password);

        var combinedHash = BCrypt.withDefaults().hashToString(12, combinedString.toCharArray());

        // let's try to break it
        var wrongPassword = "wrong-password";
        var wrongCombinedString = String.format("%s:%s:%s", userId, username, wrongPassword);

        var result = BCrypt.verifyer().verify(combinedHash.toCharArray(), wrongCombinedString);
        if (result.verified) {
            System.out.println("Password is correct");
        } else {
            System.out.println("Password is incorrect");
        }
    }
}

Let’s run it:

Exception in thread "main" java.lang.IllegalArgumentException: password must not be longer than 72 bytes plus null terminator encoded in utf-8, was 102
	at at.favre.lib.crypto.bcrypt.LongPasswordStrategy$StrictMaxPasswordLengthStrategy.innerDerive(LongPasswordStrategy.java:50)
	at at.favre.lib.crypto.bcrypt.LongPasswordStrategy$BaseLongPasswordStrategy.derive(LongPasswordStrategy.java:34)
	at at.favre.lib.crypto.bcrypt.BCrypt$Hasher.hashRaw(BCrypt.java:303)
	at at.favre.lib.crypto.bcrypt.BCrypt$Hasher.hash(BCrypt.java:267)
	at at.favre.lib.crypto.bcrypt.BCrypt$Hasher.hash(BCrypt.java:229)
	at at.favre.lib.crypto.bcrypt.BCrypt$Hasher.hashToString(BCrypt.java:205)
	at BcryptAtFavre.main(BcryptAtFavre.java:14)

Nice, good job, Patrick, you saved the day for Java!

After checking the source code, I found this piece:

@Override
public byte[] derive(byte[] rawPassword) {
    if (rawPassword.length >= maxLength) {
        return innerDerive(rawPassword);
    }
    return rawPassword;
}

and the strict strategy that threw the exception we’ve seen:

final class StrictMaxPasswordLengthStrategy extends BaseLongPasswordStrategy {
    StrictMaxPasswordLengthStrategy(int maxLength) {
        super(maxLength);
    }

    @Override
    public byte[] innerDerive(byte[] rawPassword) {
        throw new IllegalArgumentException("password must not be longer than " + maxLength + " bytes plus null terminator encoded in utf-8, was " + rawPassword.length);
    }
}

We can see that this strict strategy is used as a part of the default configs:

public static Hasher withDefaults() {
    return new Hasher(Version.VERSION_2A, new SecureRandom(), LongPasswordStrategies.strict(Version.VERSION_2A));
}

Cool!

Let’s switch to JavaScript.

JavaScript and Bcrypt

Here I used the bcryptjs which has over 2 million weekly downloads based on the NPM stats.

const bcrypt = require('bcryptjs')

function randomString (length) {
  const chars = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
  let result = ''
  for (let i = length; i > 0; --i) {
    result += chars[Math.floor(Math.random() * chars.length)]
  }
  return result
}

function runTest () {
  // 18 + 55 + 1 = 74, so above 72 characters' limit of BCrypt
  const userId = randomString(18)
  const username = randomString(55)
  const password = 'super-duper-secure-password'

  const combinedString = `${userId}:${username}:${password}`

  const combinedHash = bcrypt.hashSync(combinedString)

  // let's try to break it
  const wrongPassword = 'wrong-password'
  const wrongCombinedString = `${userId}:${username}:${wrongPassword}`

  if (bcrypt.compareSync(wrongCombinedString, combinedHash)) {
    console.log('Password is correct')
  } else {
    console.log('Password is wrong')
  }
}

runTest()

The output is:

Password is correct

Not great. The source code reveals that similar to Spring Security, the library validates the salt

if (salt.charAt(0) !== '$' || salt.charAt(1) !== '2') {
     err = Error("Invalid salt version: "+salt.substring(0,2));
     if (callback) {
         nextTick(callback.bind(this, err));
         return;
     }
     else
         throw err;
}
...

but not the input length.

Let’s try if Python can do any better.

Python and Bcrypt

Using bcrypt library with 1.3k starts and the latest release in November.

import random
import string

import bcrypt

def random_string(length):
    return ''.join(random.choice(string.ascii_letters) for i in range(length))

if __name__ == '__main__':
    # 18 + 55 + 1 = 74, so above 72 characters' limit of BCrypt
    user_id = random_string(18)
    username = random_string(55)
    password = "super-duper-secure-password"

    combined_string = "{0}:{1}:{2}".format(user_id, username, password)

    combined_hash = bcrypt.hashpw(combined_string.encode('utf-8'), bcrypt.gensalt())

    # let's try to break it
    wrong_password = "wrong-password"
    wrong_combined_string = "{0}:{1}:{2}".format(user_id, username, wrong_password)

    if bcrypt.checkpw(wrong_combined_string.encode('utf-8'), combined_hash):
        print("Password is correct")
    else:
        print("Password is incorrect")

The result is same as we observed for most of our test subjects:

Password is correct

All right, but what about some newer and more safety-oriented language - let’s try Rust.

Rust and Bcrypt

Here I need to be honest: since I’m not a Rust expert at all, I used a help of a Claude AI to write this code. So, if you see any issues there, please, let me know in the comments section, so I can fix that.

As a library, I used rust-bcrypt based on my AI friend advice.

use rand::RngCore;
use base64::{Engine as _, engine::general_purpose::URL_SAFE};
use std::error::Error;

fn random_string(length: usize) -> String {
    let mut bytes = vec![0u8; length];
    rand::thread_rng().fill_bytes(&mut bytes);
    URL_SAFE.encode(&bytes)[..length].to_string()
}

fn main() -> Result<(), Box<dyn Error>> {
    // 18 + 55 + 1 = 74, so above 72 characters' limit of BCrypt
    let user_id = random_string(18);
    let username = random_string(55);
    let password = "super-duper-secure-password";

    let combined_string = format!("{}:{}:{}", user_id, username, password);
    let combined_hash = bcrypt::hash(combined_string.as_bytes(), bcrypt::DEFAULT_COST)?;

    // let's try to break it
    let wrong_password = "wrong-password";
    let wrong_combined_string = format!("{}:{}:{}", user_id, username, wrong_password);

    match bcrypt::verify(wrong_combined_string.as_bytes(), &combined_hash) {
        Ok(true) => println!("Password is correct"),
        Ok(false) => println!("Password is incorrect"),
        Err(e) => println!("{}", e),
    }

    Ok(())
}

The output is:

Password is correct

I can see the validation of the cost:

if !(MIN_COST..=MAX_COST).contains(&cost) {
    return Err(BcryptError::CostNotAllowed(cost));
}

but not of the input. And here is the place where the explicit truncation of 72 chars happens (the comment is from the library source code):

// We only consider the first 72 chars; truncate if necessary.
// `bcrypt` below will panic if len > 72
let truncated = if vec.len() > 72 {
    if err_on_truncation {
        return Err(BcryptError::Truncation(vec.len()));
    }
    &vec[..72]
} else {
    &vec
};

let output = bcrypt::bcrypt(cost, salt, truncated);

Why?

That was my first question after seeing that the majority of the tools follow the pattern that leads to the vulnerability. Wikipedia article about Bcrypt gave a hint:

Many implementations of bcrypt truncate the password to the first 72 bytes, following the OpenBSD implementation

Interesting! Let’s check the OpenBSD implementation of this algorithm, and here is the link to it. The first point of interest lies here:

/* strlen() returns a size_t, but the function calls
 * below result in implicit casts to a narrower integer
 * type, so cap key_len at the actual maximum supported
 * length here to avoid integer wraparound */
key_len = strlen(key);
if (key_len > 72)
	 key_len = 72;
key_len++;

And from that moment on, key_len is used as a limit to iterate over the input string within, for example:

u_int32_t
Blowfish_stream2word(const u_int8_t *data, u_int16_t databytes,
    u_int16_t *current)
{
	u_int8_t i;
	u_int16_t j;
	u_int32_t temp;

	temp = 0x00000000;
	j = *current;

	for (i = 0; i < 4; i++, j++) {
		if (j >= databytes)
			j = 0;
		temp = (temp << 8) | data[j];
	}

	*current = j;
	return temp;
}

Where key_length is passed as a databytes parameter. So this piece of code:

if (j >= databytes)
	j = 0;

will make sure that no chars over the limit (72) will end up being processed.

Git blame shows that the if (key_len > 72) line is 11 years old

while the if (j >= databytes) j = 0; is 28 years old (what were you busy with in 1997, ah?)

So, it’s been a while since the API has been reiterated.

Some thoughts on that

Disclaimer

Let me start with a short disclaimer: I have a huge respect for people who spend their free time and mental capacity on maintaining open-source projects. That’s a large amount of work, that is not paid, and, unfortunately, quite often not appreciated by the users of the tools. That’s why they have all the legal and ethical rights to build the project the way they see them. My opinions below are not targeted towards anyone in particular.

My initial goal was to create issues for each of the mentioned library, but I noticed that this behavior has been already reported to each of them:

Check the discussions and their outcomes by following those links.

Thoughts and lessons

As a guy who spent a few years of my career on building tools and solutions to be used by other software engineers, I understand the frustration: you invested your time and effort into writing a clear documentation and guides, but a certain number of your users don’t bother checking it at all, and just use the tool the way they think it should be used. However, that’s the reality that I had to accept and started thinking about how can I make my tools handle those use cases. Here are a few principles I came up with in that process.

Don’t let the people use your API incorrectly

In my opinion, from the API perspective, the approach when the tool silently cuts the part of the input and processes the remaining one only, it is an extremely poor design choice. What makes things worse is the fact that Bcrypt is used in the domain of security and sensitive data, and, as we can see, most of the tools mentioned above, use password as the name of the input parameter of the hashing method. The good design should explicitly reject the invalid input with the error / exception / any other mechanism the platform uses. So, basically, exactly what Go and Patrick’s Java library did. This way, incidents like Okta one would be impossible by design (btw, I’m not shifting the blame away from Okta, considering the domain they operate in).

It is ok, though, to offer the non-default unsafe option, that will let the users pass longer input that will be truncated if the user explicitly asks for that. A prefix/suffix like unsafe, truncated, etc. can be a good addition to the names of the method that expose these options.

Be predictable

If we take a step back from the Bcrypt case, imagine other examples, if such a pattern becomes common in the industry:

• We created a new user account on HBO to watch a new season of Rick and Morty, and there is a warning that the max size of the password should not exceed 18 chars. However, the password generator of your password manager tool uses 25 chars as a default length of the produced password. So, the password manager inserts that password while creating an account, but the server cuts the last 7 chars, hashes the rest, and saves the hash to the DB. How easy would it be for us to be able to log in to HBO next time and watch a new episode?
• The tech lead of the new project configured a linter tool, and set the max line length as 100 chars. While performing a check, linter removes the chars above the defined limit, and informs that the check has passed. How useful would it be?

A good API design should remember that when it comes to tech, nobody likes surprises.

No ego

While following a few online discussions about the Bcrypt Okta incident, I noticed something else: while the majority of comments agreed that we should design APIs like these better, there were a few folks that took a very defensive stance and exposed their ego: “Read a paper before using anything!”, “APIs are only correcting the input after the stupid users!”, etc. Based on my experience, ego is a big enemy of engineering. And I wouldn’t be surprised if you have a story or two in that regard as well. So, yeah, let’s not bring our egos to our APIs.

Be helpful

Don’t get me wrong, I do understand the gist that the users should have some basic knowledge before using any tool. But let’s get back to the reality: how many different tools, programming languages, databases, protocols, frameworks, libraries, algorithms, data structures, clouds, AI models, etc. does a software engineer use per week these days? I tried to count for my use case, but stopped after the number had reached 30. Is it possible to know all of them deep? To know all the edge cases and limits? For some of them and to some degree is a reasonable ask, as well as having an expertise in 1 or 2, but definitely not all. The hard truth is that on average, the industry today requires the wide spectrum of knowledge over the deep one (check any job opening to verify that claim). Therefore, while designing the tools, why not to help our fellow colleagues? For example, if our tool accepts only positive numbers, let’s add if num < 1 -> return error to our solution, and make the life simpler for somebody out there.

Especially, if the tool might be used in the security-sensitive context, where humans are usually the weak point in the thread modelling. The good API can help there.

Be brave

It’s not so often that the API we design is something completely new to the world. Most likely, there are other solutions like ours out there. And the chances are that they’ve been already doing certain things the particular way. However, that doesn’t mean that we need to follow the same path. Kudos to the Go team and Patrick’s Java library for being brave to do things the different way as the industry does in the Bcrypt example. Let’s learn from them.

Reiterate

Regardless of the original design choices and intentions, it’s never too late to reiterate on some of them if we see a need or have discovered new information. That’s, actually, a place where a lot of us fail due to different reasons, with some of them listed above.

Instead of a conclusion

The Okta incident exposed large security issues out there. Our test showed, even 3 months after the incident, the industry is still vulnerable to the same outcome, so the chances are that more to come. However, we, as software engineers, can learn from that, and apply these lessons while designing APIs to make them predictable and easier to use.

I hope that was useful, and triggered some thoughts. Thanks a lot for reading my post, and see you in the following ones, there are plenty of topics to discuss. Have fun! =)

Adblock test (Why?)