Install Mathesar from source on Linux¶

For experienced Linux sysadmins

To follow this guide you need be experienced with Linux server administration, including the command line interface and some common utilities.

Requirements¶

System¶

We recommend having at least 60 GB disk space and 4 GB of RAM.

Operating System¶

We’ve tested this on Ubuntu, but we expect that it can be adapted for other Linux distributions as well.

Access¶

You should have root access to the machine you’re installing Mathesar on.

Software¶

You’ll need to install the following system packages before you install Mathesar:

Python 3.9

Python version

Python older than 3.9 will not run Mathesar.

Python newer than 3.9 will run Mathesar, but will require some slightly modified installation steps which we have not yet documented.
PostgreSQL 13 or newer (Verify with psql --version)
NodeJS 14 or newer (Verify with node --version)

(This is required for installation only and will eventually be relaxed)
npm 6 or newer (Verify with npm --version)

(This is required for installation only and will eventually be relaxed)
Caddy (Verify with caddy version)
git (Verify with git --version)

Domain (optional)¶

If you want Mathesar to be accessible over the internet, you’ll probably want to set up a domain or sub-domain to use. If you don’t need a domain, you can skip this section.

Before you start installation, ensure that the DNS for your sub-domain or domain is pointing to the machine that you’re installing Mathesar on.

Customizing this Guide¶

Type your domain name into the box below. Do not include a trailing slash.

Then press Enter to customize this guide with your domain name.

Installation Steps¶

Set up the database¶

Open a psql shell.
```
sudo -u postgres psql
```
Mathesar needs a Postgres superuser to function correctly. Let’s create a superuser.
```
CREATE USER mathesar WITH SUPERUSER ENCRYPTED PASSWORD '1234';
```
Customize your password

Be sure to change the password 1234 in the command above to something more secure and private. Record your custom password somewhere safe. You will need to reference it later.
Next, we have to create a database for storing Mathesar metadata.
```
CREATE DATABASE mathesar_django;
```
Now we let us create a database for storing your data.
```
CREATE DATABASE your_db_name;
```
Press Ctrl+D to exit the psql shell.

Set up your installation directory¶

Choose a directory to store the Mathesar application files.
Examples
- /home/my_user_name/mathesar
- /etc/mathesar
Type your installation directory into the box below. Do not include a trailing slash.

Then press Enter to customize this guide with your installation directory.
Create your installation directory.
```
mkdir -p xMATHESAR_INSTALLATION_DIRx
```
When installing outside your home folder

If you choose a directory outside your home folder, then you’ll need to create it with sudo and choose an appropriate owner for the directory (i.e. root or a custom user of your choosing).

The remainder of this guide requires you to run commands with full permissions inside your installation directory. You can do this, for example via:
- chown my_user_name: xMATHESAR_INSTALLATION_DIRx
  
  Or
- sudo su
Navigate into your installation directory.
```
cd xMATHESAR_INSTALLATION_DIRx
```
The remaining commands in this guide should be run from within your installation directory.

Set up the environment¶

Clone the git repo into the installation directory.

git clone https://github.com/centerofci/mathesar.git .

Checkout the tag of the latest stable release, 0.1.3.
```
git checkout 0.1.3
```
Important

If you don’t run the above command you’ll end up installing the latest development version of Mathesar, which will be less stable.

Tip

You can install a specific Mathesar release by running commands like git checkout 0.1.1 (to install version 0.1.1, for example). You can see all available versions by running git tag.

We need to create a python virtual environment for the Mathesar application.

<path-to-python-binary> -m venv ./mathesar-venv
# /usr/bin/python3.9 -m venv ./mathesar-venv

Next we will activate our virtual environment:
```
source ./mathesar-venv/bin/activate
```
Important

You need to activate the environment each time you restart the shell as they don’t persist across sessions.

Install the Mathesar application¶

Install Python dependencies
```
pip install -r requirements.txt
```
Set the environment variables
1. Create .env file
```
touch .env
```
2. Edit your .env file, making the following changes:
  - Add the Backend Configuration environment variables
  - Customize the values of the environment variables to suit your needs.
  Example
  
  Your .env file should look something like this
```
ALLOWED_HOSTS='xDOMAIN_NAMEx'
SECRET_KEY='dee551f449ce300ee457d339dcee9682eb1d6f96b8f28feda5283aaa1a21'
DJANGO_DATABASE_URL=postgresql://mathesar:1234@localhost:5432/mathesar_django
MATHESAR_DATABASES=(your_db_name|postgresql://mathesar:1234@localhost:5432/your_db_name)
```
3. Add the environment variables to the shell
  
  You need to export the environment variables listed in the .env file to your shell. The easiest way would be to run the below command.
```
export $(sudo cat .env)
```
  Important
  
  You need to export the environment variables each time you restart the shell as they don’t persist across sessions.
Install the frontend dependencies
```
npm install --prefix mathesar_ui
```

Compile the Mathesar Frontend App

npm run --prefix mathesar_ui build --max_old_space_size=4096

Install Mathesar functions on the database:

python install.py --skip-confirm | tee /tmp/install.py.log

Create a media directory for storing user-uploaded media
```
mkdir .media
```

Set up Gunicorn¶

We will use systemd to run the gunicorn service as it lets you use easily start and manage the service.

Create a user for running Gunicorn

sudo groupadd gunicorn && \
sudo useradd gunicorn -g gunicorn

Make the gunicorn user the owner of the .media directory
```
sudo chown -R gunicorn:gunicorn .media/
```

Create the Gunicorn systemd service file.

sudo touch /lib/systemd/system/gunicorn.service

and copy the following code into it.

[Unit]
Description=gunicorn daemon
After=network.target network-online.target
Requires=network-online.target

[Service]
Type=notify
User=gunicorn
Group=gunicorn
RuntimeDirectory=gunicorn
WorkingDirectory=xMATHESAR_INSTALLATION_DIRx
ExecStart=/bin/bash -c 'xMATHESAR_INSTALLATION_DIRx/mathesar-venv/bin/gunicorn config.wsgi:application'
EnvironmentFile=xMATHESAR_INSTALLATION_DIRx/.env

[Install]
WantedBy=multi-user.target

Reload the systemctl and Start the Gunicorn socket

sudo systemctl daemon-reload && \
sudo systemctl start gunicorn.service && \
sudo systemctl enable gunicorn.service

Check the logs to verify if Gunicorn is running without any errors
```
sudo journalctl --priority=notice --unit=gunicorn.service
```

Set up the Caddy reverse proxy¶

We will be using the Caddy Reverse proxy to serve the static files and set up SSL certificates.

Create the CaddyFile
```
sudo touch /etc/caddy/Caddyfile
```

Add the configuration details to the CaddyFile

https://xDOMAIN_NAMEx {
    log {
        output stdout
    }
    respond /caddy-health-check 200
    encode zstd gzip
    handle_path /media/* {
        @downloads {
            query dl=*
        }
        header @downloads Content-disposition "attachment; filename={query.dl}"

        file_server {
            precompressed br zstd gzip
            root {$MEDIA_ROOT:xMATHESAR_INSTALLATION_DIRx/.media/}
        }
    }
    handle_path /static/* {
        file_server {
            precompressed br zstd gzip
            root {$STATIC_ROOT:xMATHESAR_INSTALLATION_DIRx/static/}
        }
    }
    reverse_proxy localhost:8000
}

Create a user for running Caddy

sudo groupadd caddy && \
sudo useradd caddy -g caddy

Create the Caddy systemd service file.

sudo touch /lib/systemd/system/caddy.service

and copy the following code into it.

[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target network-online.target
Requires=network-online.target

[Service]
Type=notify
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --force
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

Reload the systemctl and start the Caddy socket

sudo systemctl daemon-reload && \
sudo systemctl start caddy.service && \
sudo systemctl enable caddy.service

Check the logs to verify if Caddy is running without any errors
```
sudo journalctl --priority=notice --unit=caddy.service
```

Set up your user account¶

Mathesar is now installed! You can use it by visiting the URL xDOMAIN_NAMEx.

You’ll be prompted to set up an admin user account the first time you open Mathesar. Just follow the instructions on screen.

Administration¶

Upgrade¶

Go to your Mathesar installation directory.
```
cd xMATHESAR_INSTALLATION_DIRx
```
Note

Your installation directory may be different from above if you used a different directory when installing Mathesar.

Pull the latest version from the repository

git pull https://github.com/centerofci/mathesar.git

Update Python dependencies
```
pip install -r requirements.txt
```
Next we will activate our virtual environment:
```
source ./mathesar-venv/bin/activate
```
Add the environment variables to the shell before running Django commands
```
export $(sudo cat .env)
```
Run the latest Django migrations
```
python manage.py migrate
```
Install the frontend dependencies
```
npm install --prefix mathesar_ui
```

Build the Mathesar frontend app

npm run --prefix mathesar_ui build --max_old_space_size=4096

Update Mathesar functions on the database:

python install.py --skip-confirm >> /tmp/install.py.log

Restart the gunicorn server
```
sudo systemctl restart gunicorn
```

Uninstalling Mathesar¶

Stop Caddy service

sudo systemctl disable caddy.service && sudo systemctl stop caddy.service

Remove Caddy service file and Caddyfile

sudo rm /lib/systemd/system/caddy.service && sudo rm /etc/caddy/Caddyfile

Stop Gunicorn

sudo systemctl disable gunicorn.service && sudo systemctl stop gunicorn.service

Remove Gunicorn service file

sudo rm /lib/systemd/system/gunicorn.service

Remove your Mathesar installation directory
```
sudo rm -r xMATHESAR_INSTALLATION_DIRx
```
Your installation directory might be customized

It’s possible that Mathesar could have been installed into a different directory than shown above. Use caution when deleting this directory.
Remove Django database
1. Connect to the psql terminal.
```
sudo -u postgres psql
```
2. Drop the Django database.
```
DROP DATABASE mathesar_django;
```
Remove Mathesar internal schemas.

If you’d like to continue using your PostgreSQL database, you’ll need to remove the schemas created for Mathesar’s use during installation. You can remove them from the database as follows:
1. Connect to the database.
```
psql -h <DB HOSTNAME> -p <DB PORT> -U <DB_USER> <DB_NAME>
```
2. Delete the types schema.
```
DROP SCHEMA mathesar_types CASCADE;
```
  Deleting this schema will also delete any database objects that depend on it. This should not be an issue if you don’t have any data using Mathesar’s custom data types.
3. Delete the function schemas.
```
DROP SCHEMA msar CASCADE;
DROP SCHEMA __msar CASCADE;
```