Posted on by NSO Research group

Devmaster 8000 and devmaster 8001

Setup

This is a docker container that runs a “job server”, that can be reached over the Internet.

A job description contains:

  1. Set of input files that are sent to the server, and saved in a sandboxed directory
  2. The command to run in the sandbox (for example: gcc ./inputfile.c -o binary)
  3. Set of output files to download from the server after the job has run

The interesting part of the direcory structure of the docker image is:

/etc/                             
/home/                            
/home/user/
          challenge.sh      Startup script that runs the job server  
          server            The job server that accepts and runs jobs  
          admin             An admin server, requires a password to access  
          admin.cc          The source code of the admin server  
          target_loop       Rebuilds the admin server every 30 seconds from admin.cc  
          drop_privs        Drops privileiges and executes as requested uid:gid  
          flag              Owned by admin, readonly

/home/user/builds/build-workdir-XXXXX    A unique directory per job

The (interesting) users in /etc/password
root
admin
sanbox-runner-0
sanbox-runner-1
.
.
sandox-runner-7

Interesting: The admin server is built every 30 seconds from admin.cc via the job server as a regular job.

Observations

The sandbox includes:

  1. The uid:gid are set to sandox-runner-X, and no 2 jobs are run simultaniously with the same user. I.E., if sandbox-runner-0 is running, then the new job will be assigned to the next user id, sandbox-runner-1
  2. The job process enters the following namespaces: mount, ipc.
  3. All filesystems are mounted r/o except :
    1. /home/user/builds/build-workdir-XXXXXX of that job is mounted r/w
    2. /proc mounted r/w
    3. /tmp mounted r/w
  4. Notably:
    1. The sandboxing code was copied from the Bazel project, but CLONE_PID and CLONE_USER were removed.
    2. ptrace is ensured to be available, in the challenge.sh startup script
    3. /home/user/drop_privs is owned by root and suid (!)

Devmaster 8000

The challenge is solved by submitting a job that runs

/home/user/drop_privs admin admin cat /home/user/flag

This works since drop_privs is owned by root and suid and sgid.

DevMaster 8001

drop_privs is marked non-executable for non-root users, which precludes the previous solution.

The solution may be an overkill 😉

Our goal was to run an executable that has the same uid and gid as the admin-build job.

Attempt #1

Trying to double fork() and reparent ourselves to the init process, thus, the job server would think we were done running, while we are still running.

The job server set prctl(PR_SET_SUBREAPER,1) on the job’s parent and we were unable to reparent ourselves to init.

Attempt #2

Main idea: Copy a suid file as sandbox-runner-1 to /tmp/patcher and then run it from sandbox-runner-0, thus obtaining the uid:gid of sandbox-runner-1 while the job server thinks we running as sandbox-runner-0.

Note that the admin build job runs

bash -c “sleep 1; g++ –std=C++11 admin.cc -o admin; sleep 1”

and since the job server thinks we are running as sanbox-runner-0, the admin build will run as sandobx-runner-1.

As sandbox-runner-1:

  1. Copy a modified admin.cc to → /tmp/.cc – see step 5 to understand the odd filename
  2. Wait for admin build to happen (monitor this with ps)
  3. ptrace attach to the bash process
  4. find bash’s heap (grep heap /proc/pid/maps)
  5. Search for the string “admin.cc” in bash’s heap and replace it with “/tmp/.cc” (same number of chars)
  6. ptrace detach from the bash process

The result will be that bash will compile our own admin server that will output the flag file without any checks.

Below is the timeline of the attack:

Situation at Time: T

  • user sandbox-runner-0 > job bash -c sleep(10)

Situation at Time: T + 5 seconds

  • user sandbox-runner-0 > job bash -c sleep(10)
  • user sandbox-runner-1 > job bash -c cp new-admin.cc /tmp/.cc; cp ./patcher /tmp/patcher; chmod ug+s /tmp/patcher

Situation at Time: T + 10 seconds

  • user sandbox-runner-0 > job /tmp/patcher

note: /tmp/patcher will actually run as sandbox-runner-1 due to the suid permission)

Situation at Time: T + 10 seconds + sometime (<30)

  • user sandbox-runner-0 > job /tmp/patcher
  • user sandbox-runner-1 > job build admin server (run by the system) – will compile /tmp/.cc instead of admin.cc

Situation at Time: T + 40 seconds

  • connect as new-and-improved admin and get the flag