Clowntown flags

The first approach I’ve been experimenting with is gating dangerous behavior behind flags that sound dangerous. I’ve observed that an AI agent will perform relatively little verification before using the -f flag associated with a command - it will quite happily run docker system prune -af with little care about what damage this might do.

To avoid this, I’ve been experimenting with an approach that was initially described by Rachel Kroll. The essential idea is that some commands or flags are so dangerous, that they should have names that communicate that danger.

One of them from Facebook was “clown town” or just “clowntown”. While there might be several uses, I tended to think of it as an adjective sometimes, almost like calling something “goofy” without calling it straight-up “stupid”. Of course, there was also “clowny”, with its trailing “y” suggesting adjective use (downy, cushy, wobbly), but that’s language for you.

In particular, “clowntown” made it out of the spoken realm and back into the computers in the form of command-line arguments you could pass to certain tools. By using them, you were affirming that whatever you were asking it to do was in fact broken, crazy, goofy, wacky, or maybe just plain stupid, but you needed it to happen anyway. […] Basically, you COULD run with scissors, but other people would find out about it, and if the organization was doing its job, there would be follow-up to find out why. […]

I should also mention that we had other flags which were even scarier than the sort that you might hide behind the --clowntown flag. I think one of them was something like --i-want-to-cause-a-sev.

_{Sharp tools for emergencies and the --clowntown flag, October 27, 2020}

I had a similar problem, in that I had a shell script responsible for setting up the agent’s SBX environment, which was named something like ./sbx-setup. I didn’t want this script to run outside of SBX, because it would do things like try to control the globally installed version of Python.

After I created this script, I was worried the agent would try to use this set up script for setting up a non-sandboxed environment, so I tried disincentivising it by making the script refuse to run unless it was either running in a sandbox or the user had set the environment variable PLEASE_FUCK_UP_MY_COMPUTER. I reasoned that such a strange looking flag would give the agent more cause to investigate exactly what the flag did and if the user intended such an outcome. Also, when manually approving permission prompts, such an environment variable would stick out and make it more apparent that I was approving something unusual.

 1#!/usr/bin/env bash
 2set -euo pipefail
 3
 4if [[ -z "${PLEASE_FUCK_UP_MY_COMPUTER:-}" && -z "${IS_SANDBOX:-}" ]]; then
 5    echo "Warning: this script runs with no safety checks and should not" >&2
 6    echo "be run outside of a sandbox." >&2
 7    echo >&2
 8    echo "If you're running this in a sandbox/VM/container you don't" >&2
 9    echo "care about, set IS_SANDBOX=1. Otherwise, to acknowledge the" >&2
10    echo "risk and run it anyway:" >&2
11    echo >&2
12    echo "    PLEASE_FUCK_UP_MY_COMPUTER=1 $0" >&2
13    exit 1
14fi
15
16echo "Continuing..."

(In this case, the variable IS_SANDBOX=1 is set automatically by the sbx template. If your template doesn’t provide this, you can use SBX kits to add an environment variable.)

I’ve had mixed success with this. In some cases, it will try to supply the PLEASE_FUCK_UP_MY_COMPUTER variable, even when agent wasn’t explicitly asked to override this, and supplying the flag wasn’t required to accomplish its stask. It seems that having a scary name is not sufficient disincentive to prevent the AI from running these commands.

PreToolUse Hooks

Another approach I’ve been experimenting with is PreToolUse hooks. These allow you to write pieces of code which act on individual command invocations, even when --dangerously-skip-permissions is set. This allows you to deny commands based upon them matching some pattern.

The specific pattern I was interested in matching was docker compose down -v. I find that Claude is a little too gung-ho about deleting Docker volumes: it doesn’t give the user enough warning that the command it’s asking to run is about to delete the database volume. (In the specific case where this happened, it happened inside a sandbox to data I didn’t need, but I treated it as a near miss.)

One way to address this is by restricting what permissions the agent has, by adding docker to its deny permission set. However, a lot of docker commands are quite useful, and don’t pose any particular danger. The solution I landed on was to use hooks to identify and deny dangerous commands.

Here’s an example of how to write a hook. First, register the hook in your .claude/settings.json

 1{
 2  "hooks": {
 3    "PreToolUse": [
 4      {
 5        "matcher": "Bash",
 6        "hooks": [
 7          {
 8            "type": "command",
 9            "if": "Bash(docker compose *)",
10            "command": "python3 \"$CLAUDE_PROJECT_DIR/.claude/hooks/docker-filter.py\""
11          }
12        ]
13      }
14    ]
15  }
16}

Next, provide a hook that acts on the command, and detects some dangerous pattern.

 1"""PreToolUse hook (matcher: Bash). Blocks `docker compose ... -v/--volumes`."""
 2import json
 3import shlex
 4import sys
 5
 6
 7def main() -> int:
 8    command = json.load(sys.stdin)["tool_input"]["command"]
 9    tokens = shlex.split(command)
10    if "-v" in tokens or "--volumes" in tokens:
11        REASON = (
12            "Command blocked by a local PreToolUse hook. "
13            "That looks like a command to delete all local volumes, including the database. "
14            "Are you sure the user asked you to do that?"
15        )
16        json.dump(
17            {"hookSpecificOutput": {
18                "hookEventName": "PreToolUse",
19                "permissionDecision": "deny",
20                "permissionDecisionReason": REASON,
21            }},
22            sys.stdout,
23        )
24    return 0
25
26
27if __name__ == "__main__":
28    try:
29        sys.exit(main())
30    except Exception as exc:
31        # Fail closed: if the hook can't do its job, block the command.
32        print(f"hook failed, blocking command: {exc}", file=sys.stderr)
33        sys.exit(2)

This is a simplified example, but it does demonstrate some key points. (Note: this example is simple enough to solve using permissions with globs, but I wanted a way to demonstrate PreToolUse. PreToolUse also has some other interesting applications, like steering the model away from incorrect coding patterns.)

Hooks do not fail-secure by default

By default, Claude will only interpret your hook as denying the tool call if it either emits a valid "permissionDecision": "deny" JSON, or if it exits with exit code 2. Other kinds of errors, like Python not being installed, are not treated as a deny.

What I would suggest in the scenario where you get an unexpected exception, is to exit with exit code 2, to immediately stop the tool use. If you have an unexplored corner case in the code responsible for making this decision, the safest thing to do is to immediately deny the tool use.

Also, you should err on the side of having false positives in your rule rather than false negatives. In the above rule, there are innocent reasons to have both docker compose plus -v in the same command. (For example, the flag -v can also be used to apply bind mounts or request more debug information.)

Bash is a complex language

There are a variety of ways in Bash to obfuscate a command so that it never contains a string like --volumes, but expands to that full string at run-time. Therefore, you should view this restriction as more of a guideline than an actual rule. Of course, it’s possible to lex and parse Bash more precisely, but this ends up being more complex, and I’m not sure it’s more secure in practice than a simple substring or regex match.

Use factual language

According to the Claude docs, it’s a good idea to avoid writing tool hooks which contain imperative language, as it can trigger Claude’s prompt-injection defenses.

Write the text as factual statements rather than imperative system instructions. Phrasing such as “The deployment target is production” or “This repo uses bun test” reads as project information. Text framed as out-of-band system commands can trigger Claude’s prompt-injection defenses, which causes Claude to surface the text to you instead of treating it as context.

For this reason, the denial reason doesn’t use imperative mood. It’s only a factual observation about the command that was just attempted. It also avoids conclusory language, because the hook’s matcher is over-broad, and the agent might have been running a completely innocent command. In practice, I’ve found that the agent immediately requests clarification in this situation.

OS Sandboxing

This is the most powerful and most secure technique, but it is also quite hard to apply. It’s a powerful technique because you’re implementing the sandbox in a manner the agent cannot modify. It’s secure because it makes use of sandboxing primitives that are well studied, and unlikely to have weaknesses. However, it’s hard to apply because the sandbox’s rules rarely match your domain.

To give a specific example, SBX provides a very helpful capability: you can allow and deny access to HTTPS hosts based on hostname. This is great, because it a better matches the user’s mental model of permissions. In some cases, it’s also too restrictive. If you want to allow certain read-only methods in a REST API, while denying other methods, SBX cannot do that.

The ultimate goal is to be able to express a precise rule that both matches the concerns in your domain and is enforced by well-audited code the agent cannot change. This is quite hard to achieve in practice, however. I plan to write a post in the future on this topic, because it deserves a longer treatment.

Here’s how I discovered it.

Motivation

Recently, I was working on a project which required me to fit ARIMA models to over ten thousand time series using the Python package pmdarima. This is a handy Python package which automates the selection of ARIMA models by varying the p, d, and q parameters, and measuring model fit for each one, while penalizing more complex models.

Unfortunately, fitting these models is very slow, so I started looking for a way to parallelize it. When I started this work, I ran into something surprising: it’s already using multiple cores!

This contradicts the pmdarima documentation. The documentation says that parallelism is only used when doing a grid search. However, I’m using the stepwise algorithm for fitting the model, which is supposedly not parallelized.

As I’ll show later in this blog post, although it’s running in parallel, it’s not doing anything useful with those extra cores.

Dataset

In order to demonstrate this issue, I’m using a time-series dataset for grocery stores in Ecuador. The way I’m going to frame this problem is to consider each store and category of items separately, and fit an ARIMA model to each one.

Measuring performance

Most Python benchmarking only tools measure elapsed time, but to debug this issue I needed a way to measure both wall-clock time and CPU time. (Wall-clock time is another name for elapsed time. CPU time is wall-clock time multiplied by the average number of cores that were used.)

I wrote a context manager which measures both kinds of time - the full details are in the notebook at the end of the post. The sections marked with with MyTimer() as timer: are being timed this way.

How parallel is auto_arima?

I wrote a test which uses a package called threadpoolctl to restrict parallelism. I tested fitting the model with and without restricting parallelism.

 1for limit_cores in [True, False]:
 2    series = array[0, 0]
 3    with MyTimer() as timer:
 4        if limit_cores:
 5            controller = ThreadpoolController()
 6            with controller.limit(limits=1, user_api='blas'):
 7                fit = pm.auto_arima(series)
 8        else:
 9            fit = pm.auto_arima(series)
10    print(f"lim: {limit_cores} wall: {timer.wall_time:.3f} cpu: {timer.cpu_time:.3f} ")

Here are the results of this test:

lim: True wall: 8.006 cpu: 8.539 
lim: False wall: 8.312 cpu: 33.715 

There are several things we can learn from the output here:

• When limiting the parallelism, the wall-clock time is roughly equal to the CPU time.
• When not limiting the parallelism, the CPU time is about eight times larger than the wall clock time, so something is using every core, even if the pmdarima documentation says otherwise.
• The multicore version takes longer to run than the singlecore version. That’s pretty surprising - I would expect the multicore version to be faster.
• The multicore version falls even further behind in CPU time: although it slightly slower in elapsed time, it is 3.9x slower in CPU time.

Slowdown cause

Why does pmdarima slow down when you give it additional cores?

I’m not exactly sure what causes the slowdown here. My rough theory is that pmdarima uses statsmodels internally, which uses scipy internally, which uses numpy internally, which uses OpenBLAS internally.

OpenBLAS is a linear algebra library which provides various matrix and vector operations, and can use multiple threads to process a large matrix operation. However, for some small operations, the overhead associated with giving work to a different thread will be larger than the gain from parallelism.

I’m guessing that the threshold for where it switches from a singlecore operation to a multicore operation is set too low, and that’s why restricting the parallelism makes it faster.

As evidence for this, note that with controller.limit(limits=1, user_api='blas'): restricts parallelism, but only for libraries that implement the BLAS api.

Adding parallelism back in

My original purpose for looking into this was that I needed to fit over ten thousand ARIMA models to different time series. Rather than trying to parallelize within a single ARIMA model, I can run multiple different ARIMA models in parallel. This is more efficient, because the units of work are larger.

I set up a very similar problem, except that instead of fitting a single model, it fits one model for each category in a certain store. It uses multiprocessing to distribute each time series to a different process. (You cannot use ThreadPool here because auto_arima() holds the GIL most of the time.)

For the first test, I put no limit on parallelism, so within each ARIMA model, the calculation is also parallelized. For the second test, each process is limited to a single thread.

 1controller = ThreadpoolController()
 2
 3def attach_limit(func, limit, *args, **kwargs):
 4    """Call func() using a limited number of cores, or no limit if limit is None."""
 5    if limit:
 6        return func(*args, **kwargs)
 7    else:
 8        with controller.limit(limits=1, user_api='blas'):
 9            return func(*args, **kwargs)
10
11def predict(x):
12    return pm.auto_arima(x, error_action="ignore")
13
14for limit in [True, False]:
15    with multiprocessing.Pool() as p:
16        # Get one store
17        store_array = array[1]
18        with MyTimer() as timer:
19            predict_restrict = functools.partial(attach_limit, predict, limit)
20            p.map(predict_restrict, store_array)
21        print(f"lim: {str(limit).ljust(4)} time: {timer.wall_time:.3f}")

Result:

lim: True time: 144.185
lim: False time: 534.951

The result of this is a 3.7x speedup for disabling BLAS parallelism, with no changes to how the model is fit. (This is on a 4 core computer - you may get different numbers on a computer with more or less cores.)

Notebook

A jupyter notebook demonstrating this technique can be downloaded here.

Summary

• OpenBLAS does some things in parallel even if you don’t ask for it.
• You can turn this behavior off with the threadpoolctl library.
• Turning it off results in a 1.8x speedup, or a 3.7x speedup if you’re also fitting multiple ARIMA models.

The process you noticed is the master process, the process that starts all other nginx processes. This process is started by the init script that starts nginx. The reason this process is running as root is simply because you started it as root! […]
Most importantly; Only root processes can listen to ports below 1024. A webserver typically runs at port 80 and/or 443. That means it needs to be started as root.
In conclusion, the master process being run by root is completely normal and in most cases necessary for normal operation.

However, Linux has a feature called capabilities, which allow a process to do one privileged operation without being able to do any kind of privileged operation. If you look through that manual page, you’ll find one capability which is exactly what we need: CAP_NET_BIND_SERVICE. This allows a process to bind to a low-numbered port, despite not being root. Perfect!

Editing the systemd service file

Now we need a way to start nginx as an unprivileged user, with this one additional capability. You can do this with systemd. We just need to change a few configuration files.

First, stop the nginx process.

sudo systemctl stop nginx

Now, copy the system-provided nginx service file into the local configuration area.

sudo cp /lib/systemd/system/nginx.service \
    /etc/systemd/system/nginx.service

Now, use your favorite text editor to edit /etc/systemd/system/nginx.service. When we make edits to this file, it will override the system-provided service file.

Go down to the [Service] section, and add these two lines:

User=www-data
Group=www-data

This will start nginx as an unprivileged user. However, to make this work, we need to give nginx the CAP_NET_BIND_SERVICE capability. Add this line:

AmbientCapabilities=CAP_NET_BIND_SERVICE

Next, we need to create a place for nginx to write its PID file. Currently, it writes to /run/nginx.pid, which is a directory owned by root. We need to create a directory called /run/nginx which is owned by www-data. To do this, add this line:

RuntimeDirectory=nginx

systemd will automatically create this directory with the correct ownership.

Now, we need to move the PID file. Edit the line starting with PIDFile to read:

PIDFile=/run/nginx/nginx.pid

We’ll also need to tell nginx about this new PID file.

Edit the file /etc/nginx/nginx.conf. Change the line starting with pid to read:

pid /run/nginx/nginx.pid;

Now restart nginx. Run

sudo systemctl daemon-reload
sudo systemctl restart nginx

If you get an error, run this command to see a detailed error message.

sudo journalctl -u nginx

Additional sandboxing

Note: the following section assumes you have a systemd version greater than 235. To see your systemd version, run systemctl --version .

Running nginx as a non-root user is a good first step, but what else can we do to make this more secure? Linux has many built-in sandboxing features which systemd can make use of.

I added the following to my systemd configuration for nginx.service:

# Process may not gain any capabilities besides the one we just gave it
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
# Process is not allowed to gain new privileges using SUID binaries such as sudo
NoNewPrivileges=true
# Disables use of the personality(2) system call, which may have security bugs
LockPersonality=true
# Allows only common service-related system calls
SystemCallFilter=@system-service
# When system call is disallowed, return error code instead of killing process 
SystemCallErrorNumber=EPERM

You can download my full systemd service file and my nginx configuration here.

Using systemd-analyze

systemd ships with a tool to analyze how much each of the services on your system make use of systemd-related security features. (Note: this report doesn’t consider non-systemd methods of sandboxing, such as a service dropping privileges using setuid.) Run this command to see the report:

SYSTEMD_EMOJI=0 systemd-analyze security

You can also get detailed information about a single unit by running

systemd-analyze security nginx

By following this guide, you can reduce the systemd’s risk score for nginx from 9.5 (UNSAFE) to 5.0 (MEDIUM.)

Further work

There are several other things you could do to improve this sandbox:

• Make the syscall filter more restrictive. The @system-service filter is very broad and over-inclusive. Using perf, you can record exactly which syscalls a service makes, and allow only those syscalls. However, keep in mind that loading new plugins into nginx, or changing its configuration, may cause your syscall list to become out-of-date. For example, an nginx configuration which serves static files will use different syscalls than one which proxies traffic to another service. Here’s a writeup on how to do this: https://prefetch.net/blog/2017/11/27/securing-systemd-services-with-seccomp-profiles/
• Disallow nginx from changing kernel tunables and modules.
• Disallow nginx from connecting to unix domain sockets, netlink sockets, or opening raw sockets.
• Whitelist which devices in /dev nginx is allowed to read/write.
• Blacklist namespace-altering syscalls.

However, I chose to not include these things. First, many of them would require an attacker to have root privilege anyway, so once the service is no longer running as root, they have little value. Second, they have some possibility of breaking someone’s configuration. The sandbox settings I show are intended to be general-purpose and work in a variety of contexts.

Testing notes

I have tested this configuration on recent versions of Debian, Fedora, and Ubuntu. Here’s what I’ve found:

• Works on Debian Buster
• Partially works on Debian Stretch (Note: You must comment out LockPersonality and SystemCallFilter.)
• Doesn’t work on Fedora 32. The use of NoNewPrivileges interferes with SELinux somehow. If you skip the “Additional sandboxing” step, and substitute ’nginx’ for ‘www-data’, it will work. This is possibly fixable, but I don’t have much knowledge of SELinux.
• Works on Ubuntu 20.04
• Partially works on Ubuntu 18.04. (Note: You must comment out SystemCallFilter.)