Digital Adventures

ArchLinux, Ruby, Rails, OpenSource

Build a Secure, Jailed Sftp Filesharing System

Setting up a modern filesharing system on a linux server is fairly easy and convenient. However, there are some caveats and a bit of research involved so I decided to write a bit about it.

SFTP

SFTP stands for SSH File Transfer Protocol aka. Secure File Transfer Protocol and is not a modification of FTP as one might think. FTP is old, moldy and super insecure and I would strongly advise anyone against using it and to switch to SFTP instead.

Basically, as soon as you have sshd up and running on your server you have SFTP up and running too. Try it out: SFTP user@yourserver.com. You will notice that you have access to the whole directory structure of your server. Not a problem if you’re planning to be the only one using SFTP. Since this post is about filesharing though and you’re still reading it I assume you’re not.

Chroot jail

We will be following the principle of least privilege, so exposing the entire filesystem is not an option. Luckily sshd ships with an option called ChrootDirectory which will help us stick to that principle.

What ChrootDirectory basically does is to “jail” a user to a specified directory upon login. For the user this looks like / and there is no way to get out of it.

Forcing SFTP

Because SFTP relies on the SSH infrastructure every user getting SFTP access needs a login on the server. This is crucial to understand because it has major security concerns. We don’t want to allow an SFTP user to, say, run executables. A chroot jail isn’t enough, we need to restrict users to SFTP only. Again sshd comes to the rescue with an option called ForceCommand.

The setup

This setup covers setting up the chroot, adding users and making the chroot directory accessible via a simple web interface (or basic directory index).

Create the chroot directory and assign correct permissions

1
2
3
4
mkdir -p /path/to/sftp-chroot
chown root /path{,/to{,/sftp-chroot}}
chgrp http /path{,/to{,/sftp-chroot}} # Need this for web access
chmod 755 /path{,/to{,/sftp-chroot}}

It is very important that the owner of the chroot directory is root and no one else has write access to it.

In case you’re wondering about the braces above: This is one of the many awesome features of zsh. Basically what it does is:

1
2
a_common_part{_different1,_different2{,_nested}}
# => a_common_part_different1 a_common_part_different2 a_common_part_different2_nested

I use it for renaming files a lot:

1
2
mv /some/file/in/a/deeply/nested/directory{,.bak}
# => mv /some/file/in/a/deeply/nested/directory /some/file/in/a/deeply/nested/directory.bak

Create a group for SFTP users

1
groupadd sftpusers

Create a public directory every SFTP user has (write) access to

1
2
3
mkdir /path/to/sftp-chroot/public
chgrp sftpusers /path/to/sftp-chroot/public
chmod 3775 /path/to/sftp-chroot/public

Line #3 gives read and write permissions to the group and sets the sticky and sgid bits for the directory.

For your webserver to be able to access the public directory add its user to the sftpusers group:

1
usermod -a -G sftpusers http

Set up sshd

Add the following lines to your sshd_config:

1
2
3
4
5
6
7
8
9
10
11
# /etc/ssh/sshd_config

# override default of no subsystems
Subsystem       sftp    internal-sftp -u 027

Match Group sftpusers, User *,!yourusername
        ChrootDirectory /path/to/sftp-chroot
        ForceCommand internal-sftp
        AuthorizedKeysFile /path/to/sftp-chroot/%u/.ssh/authorized_keys
        AllowTcpForwarding no
        X11Forwarding no

Remember to replace the paths and yourusername.

Setting User, *,!yourusername is important so that you can still log in on your server with ssh after you’ve added yourself to the group sftpusers. It excludes your user from the match.

sshd will substitute %u with the user that tries to log in and search for a public key in /path/to/sftp-chroot/username/.ssh/authorized_keys. (You should have set PasswordAuthentication no in sshd_config for security reasons)

With this config every user in the group sftpusers (except for the excluded one) is thrown into the chroot jail and is only allowed to use SFTP.

Add yourself to the sftpusers group

Add yourself to the group and create a “home-directory” in the SFTP chroot:

1
2
3
4
usermod -a -G sftpusers yourusername
mkdir -p /path/to/sftp-chroot/yourusername/.ssh
chown yourusername:http /path/to/sftp-chroot/yourusername
chmod 2750 /path/to/sftp-chroot/yourusername

Set up the webserver

Ideally every SFTP user should only have access to his/her “home-directory” as well as to the public directory. This is already ensured with the permissions set before. However, the permissions don’t apply for the web interface.

We added the user http to the group sftpusers and made sure http has read rights for the user’s home directory. To make sure that a user that accesses the sftp-chroot via web interface can only access his/her personal home directory and the public directory we need to introduce individual .htpasswd-files per user.

Let’s take a look at the nginx-config I wrote for the system:

/etc/nginx/sites-available/drop.conf (drop.conf) download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
# redirect http to https.
server {
  listen 80;
  server_name drop.myserver.com;
  return 301 https://$server_name$request_uri;  # enforce https
}

# paste (ssl/tls)
server {
  listen 443 ssl;
  ssl_certificate /path/to/ssl.crt;
  ssl_certificate_key /path/to/ssl.key;
  server_name drop.myserver.com;
  root /path/to/sftp-chroot;

  access_log  /var/log/nginx/drop_access.log;
  error_log   /var/log/nginx/drop_error.log;

  index /.h5ai/server/php/index.php; # nice directory indexing

  auth_basic "Restricted";
  auth_basic_user_file /path/to/sftp-chroot/.htpasswd;

  location /robots.txt {
    allow all;
  }

  location ~ /\.ht.* {
    deny all;
  }

  location ~ ^/\.h5ai/(client|cache) {
  }

  location /.h5ai {
    internal;
  }

  location ~ ^(?<script_name>/\.h5ai/.+?\.php)(?<path_info>/.*)?$ {
    try_files $script_name = 404;
    fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;

    fastcgi_param HTTPS $fe_https;
    include fastcgi_params;
  }

  location ~ ^/public/ {
  }

  location ~ ^/(?<user>[^/]+)/ {
    auth_basic "Restricted per user";
    auth_basic_user_file /home/sftp-chroot/$user/.htpasswd;
  }

}

You can ignore everything containing h5ai. H5ai is a great web server index with some really cool features but I don’t want to get into too much detail here. I left the parts in the config for those that are interested in setting it up with h5ai.

The most relevant parts here are lines 21 and 22 and 51-54. As you can see a default .htpasswd file is used for everything except for the users' private directories. For those we use a bit of regex magic to define the path to their individual .htpasswd files.

The location block for the public directory in line 48 is also important. Without it nginx would look for a .htpasswd file inside the public directory.

Syncing .htpasswd files

Because it would be cumbersome to manually sync the individual files with the global one I wrote a ruby script using inotify to track changes on the per-user-files and reflect them on the global one:

/usr/local/bin/htpasswd-sync (htpasswd-sync) download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
#!/usr/bin/env ruby

require 'rb-inotify'
require 'etc'

WATCHER_PATH = ARGV[0]
GLOBAL_HTPASSWD_FILE = File.join(WATCHER_PATH, '.htpasswd')
notifier = INotify::Notifier.new

class Htpasswd < Hash
  def self.parse(file)
    lines = File.read(file).split("\n")
    lines.reject! {|line| line.match(/^\s*(#|$)/)}
    self[lines.map {|line| line.split(':')}]
  end

  def serialize
    map {|user,hash| "#{user}:#{hash}"}.join("\n")
  end

  alias :has_user? :has_key?
  alias :users :keys
end

class INotify::Event
  def relevant?
    absolute_name =~ %r{#{WATCHER_PATH}/.+/\.htpasswd$} &&
      name !~ %r{(~$)|(\.sw[px])} # swap files
  end
end

def sync_htpasswd!(file)
  global_htpasswd = Htpasswd.parse(GLOBAL_HTPASSWD_FILE)

  owner = Etc.getpwuid(File.stat(file).uid).name
  user_htpasswd = Htpasswd.parse(file)

  return if user_htpasswd.empty?

  if user_htpasswd.has_user?(owner)
    global_htpasswd[owner] = user_htpasswd[owner]
  else
    $stderr.puts "Bad file owner #{owner} for file #{file}: users are #{user_htpasswd.users.join(', ')}"
    return
  end

  File.write(GLOBAL_HTPASSWD_FILE, global_htpasswd.serialize)
end

notifier.watch(WATCHER_PATH, :recursive, :create, :modify, :moved_to) do |event|
  if event.relevant?
    sync_htpasswd!(event.absolute_name)
  end
end

notifier.run

You need to install the gem rb-inotify for this to work.

Enable and start the following systemd service to run the script at boot:

/etc/systemd/system/htpasswd-sync.service (htpasswd-sync.service) download
1
2
3
4
5
6
7
8
9
[Unit]
Description=Syncing htpasswd files of sftp users

[Service]
ExecStart=/usr/bin/ruby /usr/local/bin/htpasswd-sync /path/to/sftp-root
Restart=always

[Install]
WantedBy=default.target

Adding more users

I wrote a script to quickly add a user to the filesharing system. It creates the user and its home directory, sets the correct permissions, asks for a password for the web access and and for a public ssh key.

/usr/local/bin/add-sftp-user (add-sftp-user) download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
#!/usr/bin/env ruby
require 'fileutils'

CHROOT_DIR = '/path/to/sftp-chroot'

user = ARGV[0] || ENV['USERNAME']

if user.nil?
  puts "Usage: #{__FILE__} <username>"
  exit 1
end

user_home = File.join(CHROOT_DIR, user)
ssh_dir = File.join(user_home, '.ssh')
authorized_keys = File.join(ssh_dir, 'authorized_keys')
htpasswd = File.join(user_home, '.htpasswd')

system "useradd -m -G sftpusers -b #{CHROOT_DIR} -s /sbin/nologin #{user}"
system "usermod -d /#{user} #{user}" # This sets the home relative to the chroot

system "htpasswd -c #{htpasswd} #{user}"
FileUtils.mkdir(ssh_dir)

puts "Paste the user's public key and hit return"
File.write(authorized_keys, gets)

FileUtils.chown_R(user, 'http', user_home)
FileUtils.chmod_R('g+rs,o-rwx', user_home)

htpasswd is a tool commonly found in a package called apache-tools or httpd-tools on most distros. On Arch-Linux it’s available in the AUR.

Comments