How to prohibit system calls, GNU/Linux

Go To


I'm currently working on the back-end of ACM-like public programming contest system. In such system, any user can submit a code source, which will be compiled and run automatically (which means, no human-eye pre-moderation is performed) in attempt to solve some computational problem.

Back-end is a GNU/Linux dedicated machine, where a user will be created for each contestant, all such users being part of users group. Sources sent by any particular user will be stored at the user's home directory, then compiled and executed to be verified against various test cases.

What I want is to prohibit usage of Linux system calls for the sources. That's because problems require platform-independent solutions, while enabling system calls for insecure source is a potential security breach. Such sources may be successfully placed in the FS, even compiled, but never run. I also want to be notified whenever source containing system calls was sent.

By now, I see the following places where such checker may be placed:

  • Front-end/pre-compilation analysis - source already checked in the system, but not yet compiled. Simple text checker against system calls names. Platform-dependent, compiler-independent, language-dependent solution.
  • Compiler patch - crash GCC (or any other compiler included in the tool-chain) whenever system call is encountered. Platform-dependent, compiler-dependent, language-independent solution (if we place checker "far enough"). Compatibility may also be lost. In fact, I dislike this alternative most.
  • Run-time checker - whenever system call is invoked from the process, terminate this process and report. This solution is compiler and language independent, but depends on the platform - I'm OK with that, since I will deploy the back-end on similar platforms in short- and mid-terms.

So the question is: does GNU/Linux provide an opportunity for administrator to prohibit system calls usage for a usergroup, user or particular process? It may be a security policy or a lightweight GNU utility.

I tried to Google, but Google disliked me today.

2012-04-03 21:32
by iehrlich
Front-end/pre-compilation analysis ← preprocessor tricks can evade this easily. There is seccomp which is a mode in which a process can only read/write to a pre-opened pipe. It's enabled via a prctl() call. There are similar questions here on SO: - ninjalj 2012-04-03 21:36
@ninjalj tried, but didn't find. Mind to share a link - iehrlich 2012-04-03 21:38
Just added a link to my previous comment - ninjalj 2012-04-03 21:38
If you actually disallowed all system calls, the process could not event exit or provide any output. seccomp allows a minimal set - mark4o 2012-04-03 21:40
So this means that I have to include prctl invocation at the beginning of any code source - iehrlich 2012-04-03 21:42
This is all security configuration stuff, no coding. would be an ideal place to ask, SF wouldn't be bad - Ben Voigt 2012-04-03 21:43


mode 1 seccomp allows a process to limit itself to exactly four syscalls: read, write, sigreturn, and _exit. This can be used to severely sandbox code, as seccomp-nurse does.

mode 2 seccomp (at the time of writing, found in Ubuntu 12.04 or patch your own kernel) provides more flexibility in filtering syscalls. You can, for example, first set up filters, then exec the program under test. Appropriate use of chroot or unshare can be used to prevent it from re-execing anything else "interesting".

2012-04-03 21:57
by ephemient
@suddnely_me I just thought of another alternative, if seccomp doesn't pan out: a NaCL-based sandbox such as ZeroVM - ephemient 2012-04-03 22:18


I think you need to define system call better. I mean,

cat <<EOF > hello.c
#include <stdio.h>
int main(int argc,char** argv) {
  fprintf(stdout,"Hello world!\n");
  return 0;
gcc hello.c
strace -q ./a.out

demonstrates that even an apparently trivial program makes ~27 system calls. You (I assume) want to allow calls to the "standard C library", but those in turn will be implemented in terms of system calls. I guess what I'm trying to say is that run-time checking is less feasible than you might think (using strace or similar anyway).

2012-04-03 21:53
by timday