This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Getting Started

Getting and installing OpenDataBio

OpenDataBio is a web-based software supported in Debian, Ubuntu and Arch-Linux distributions of Linux and may be implemented in any Linux based machine. We have no plans for Windows support, but it may be easy to install on a Windows machine using Docker.

OpenDataBio is written in PHP and developed with the Laravel framework. It requires a web server (Apache or nginx), PHP and a SQL database – tested only with MySQL and MariaDB.

You may install OpenDataBio easily using the Docker files included in the distribution. The repository now includes a docker/prod profile and docker-compose.prod.yml so Docker can also be used for production deployments (with server-specific tuning and secrets management).

If you just want to test OpenDataBio locally, follow the Docker Installation.


Next steps

  1. Apache Installation
  2. Nginx Installation
  3. Docker Installation
  4. Upgrade OpenDataBio
  5. Administrator configuration
  6. Backup and restore

Prep for installation

  1. You may want to request a Tropicos.org API key for OpenDataBio to be able to retrieve taxonomic data from the Tropicos.org database. If not provided, mainly the GBIF nomenclatural service will be used;
  2. Decide how Pl@ntNet image identification will be authorized. A Pl@ntNet developer API key in PLANTNET_API_KEY provides a quota shared by the installation. Alternatively, leave the server key empty, enable personal keys, and require users to add their own key in their profile. With PLANTNET_ALLOW_USER_KEYS=true, personal keys always take precedence; PLANTNET_SERVER_FALLBACK controls whether users without one may use the server key. See Administrator configuration.
  3. OpenDataBio sends email for completed Jobs, dataset requests and password recovery. Configure a dedicated SMTP account or transactional email provider and store its credentials only in .env. Google no longer supports “Less secure app access”; when Gmail SMTP is appropriate, use an account with two-step verification and an app password, subject to your organization’s policy. See config/mail.php and .env.example for the supported settings.

1 - First time users

Tips to first time users!

OpenDataBio is software to be used online. Local installations are for testing or development, although it could be used for a single-user production localhost environment.

Choose your starting point

  • To browse public data, no account is required.
  • A registered user can access content released to registered users but cannot contribute records.
  • A full user can create and import data and participate in Datasets, Projects, and Biocollections.
  • A full user designated as a Trait curator maintains the installation-wide public Trait library; this ability is independent of Dataset roles.
  • Installation super administrators manage global configuration and access.

User roles

  • If you are installing, the first login to an OpenDataBio installation must be done with the default super-admin user: admin@example.org and password1. These settings should be changed or the installation will be open to anyone reading the docs;
  • Self-registrations only grant access to datasets with privacy set to registered users and allows user do download data of open-access, but do not allow the user to edit nor add data;
  • Only full users can contribute with data.
  • A super administrator can promote registered users. A full user explicitly authorized to manage access may also promote registered users to full users and reverse that promotion. Delegated managers cannot alter super administrators, other access managers, or delegate their own permission.

See also User Model.

Trait curator

A super administrator may grant the Trait curator ability to a full user. Curators review duplicate suggestions, scan and consolidate the public Trait library, record non-duplicate decisions, and archive or restore eligible Traits. Ordinary full users can inspect an individual Trait and suggest a duplicate but cannot execute a merge. See Trait governance and curation.

Prep your full-user account

  1. Register yourself as Person and assign it as your user default person, creating a link between your user and yourself as collector.
  2. You need at least a dataset to enter your own data
  3. When becoming a full-user, a restricted-access Dataset and Project will be automatically created for you (your Workspaces). You may modify these entities to fit your personal needs.
  4. You may create as many Projects and Datasets as needed. So, understand how they work and which data they control access to.

Entering data

There three main ways to import data into OpenDataBio:

  1. One by one through the web Interface
  2. Using the OpenDataBio POST API services:
    1. importing from a spreadsheet file (CSV, XLSX or ODS) using the web interface
    2. using the OpenDataBio R package client
  3. When using the OpenDataBio API services you must prep your data or file to import according to the field options of the POST verb for the specific ’endpoint’ your are trying to import.

Follow the Data import workflow to prepare dependencies, validate coordinates, reconcile UserJob results, and preserve IDs.

Tips for entering data

  1. If first time entering data, you should use the web interface and create at least one record for each model needed to fit your needs. Then play with the privacy settings of your Workspace Dataset, and check whether you can access the data when logged in and when not logged in.
  2. Use Dataset for a self-contained set of data that should be distributed as a group. Datasets are dynamic publications, have author, data, and title.
  3. Although ODB attempt to minimize redundancy, giving users flexibility comes with a cost, and some definitions, like that of Traits or Persons may receive duplicated entries. So, care must be taken when creating such records. Administrators may create a ‘code of conduct’ for the users of an ODB installation to minimize such redundancy.
  4. Follow an order for importation of new data, starting from the libraries of common use. For example, you should first register Locations, Taxons, Persons, Traits and any other common library before importing Individuals or Measurements
  5. There is no need to import POINT locations before importing Individuals because ODB creates the location for you when you inform latitude and longitude, and will detect for you to which parent location your individual belongs to. However, if you want to validate your points (understand where such point location will placed), you may use the Location API with querytype parameter specified for this.
  6. There are different ways to create PLOT and TRANSECT locations - see here Locations if that is your case
  7. Creating Taxons require only the specification of a name - ODB will search nomenclature services for you, find the name, metadata and parents and import all of the them if needed. If you are importing published names, just inform this single attribute. Else, if the name is unpublished, you need to inform additional fields. So, separate the batch importation of published and unpublished names into two sets.
  8. The notes field of any model is for both plain text or JSON object string formatted data. The Json option allows you to store custom structured data any model having the notes field. You may, for example, store as notes some secondary fields from original sources when importing data, but may store any additional data that is not provided by the ODB database structure. Such data will not be validate by ODB and standardization of both tags and values depends on you. Json notes will be imported and exported as a JSON string, and will be presented in the interface as a formatted table; URLs in your Json will be presented as links.

Before a large batch, import representative pilot rows, follow the UserJob, review row-level warnings and errors, retrieve affected IDs, GET the records, and only then proceed to dependent objects.

2 - Apache Installation

How to install OpenDataBio

These instructions are for an apache-based installation. For nginx, use Nginx Installation.

Server requirements

  1. The supported PHP version >= 8.2 (8.3 recommended)
  2. Web server: apache for this guide. For nginx, use Nginx Installation.
  3. It requires a SQL database, MySQL and MariaDB have been tested, but may also work with Postgres. Tested with MySQL 8.0 and MariaDB 10.6+.
  4. PHP extensions required: openssl, pdo, pdo_mysql, mbstring, tokenizer, xml, dom, gd, exif, bcmath, zip, curl, redis.
  5. Redis Server is required for queues and cache.
  6. Tectonic is used for LaTeX/PDF label generation.
  7. Pandoc is used to translate LaTeX code used in bibliographic references. It is not necessary for installation, but suggested for a better user experience.
  8. Requires Supervisor, which is needed background jobs
  9. Node.js 22 with npm is required to build the frontend during installation and upgrades.

Create Dedicated User

The recommended way to install OpenDataBio for production is using a dedicated system user. In this instructions this user is odbserver.

Download OpenDataBio

Login as your Dedicated User and download or clone this software to where you want to install it. Here we assume this is /home/odbserver/opendatabio so that the installation files will reside in this directory. If this is not your path, change below whenever it applies.


Download OpenDataBio

Prep the Server

First, install the prerequisite software: Apache, MySQL, PHP, Redis, Tectonic, Pandoc and Supervisor. On a Debian system, you need to install some PHP extensions as well and enable them:

sudo apt-get install software-properties-common
sudo add-apt-repository ppa:ondrej/php
sudo add-apt-repository ppa:ondrej/apache2

sudo apt-get install mysql-server redis-server tectonic php8.3 libapache2-mod-php8.3 php8.3-intl \
 php8.3-mysql php8.3-sqlite3 php8.3-gd php8.3-cli pandoc \
 php8.3-mbstring php8.3-xml php8.3-bcmath php8.3-zip php8.3-curl php8.3-redis \
 supervisor

sudo a2enmod php8.3
sudo phpenmod mbstring
sudo phpenmod xml
sudo phpenmod dom
sudo phpenmod gd
sudo a2enmod rewrite
sudo a2enmod alias
sudo a2enmod headers
sudo systemctl restart apache2.service



#To check if they are installed:
php -m | grep -E 'mbstring|cli|xml|gd|mysql|redis|bcmath|pcntl|zip'
tectonic --version
redis-server --version

Add a dedicated virtual host to your Apache configuration.

  • Change /home/odbserver/opendatabio to your path (the files must be accessible by apache)
  • Create /etc/apache2/sites-available/opendatabio.conf with the following content.
  • This example installs the application under /opendatabio. Replace localhost with the real host name in a public deployment.
<VirtualHost *:80>
    ServerName localhost
    ServerAdmin webmaster@localhost
    DocumentRoot /var/www/html

    RedirectMatch 302 ^/$ /opendatabio/
    RedirectMatch 301 ^/opendatabio$ /opendatabio/

    Alias /opendatabio/ "/home/odbserver/opendatabio/public/"

    <Directory "/home/odbserver/opendatabio/public">
        Options FollowSymLinks
        AllowOverride All
        Require all granted
        DirectoryIndex index.php
    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/opendatabio-error.log
    CustomLog ${APACHE_LOG_DIR}/opendatabio-access.log combined
</VirtualHost>

The main alias serves all public assets, including build, images, fonts, and Livewire assets. Separate aliases for those directories are not required.

echo 'ServerName localhost' | sudo tee /etc/apache2/conf-available/servername.conf
sudo a2enconf servername
sudo a2enmod alias rewrite headers php8.3
sudo a2dissite 000-default
sudo a2ensite opendatabio
sudo apache2ctl configtest
sudo systemctl reload apache2

Do not reload Apache unless apache2ctl configtest reports Syntax OK.

Content Security Policy (CSP) for Apache

Configure CSP at the web server layer (not in Laravel files). Apply it first in report-only mode, inspect logs, then switch to enforced mode.

For nginx standalone installs, use Nginx Installation.

Apache: where to put it

  1. Enable required module:
sudo a2enmod headers
sudo systemctl restart apache2
  1. Edit your active vhost file (example):
sudo nano /etc/apache2/sites-available/opendatabio.conf
  1. Inside the correct <VirtualHost ...> block (HTTP and/or HTTPS), add the header as a single directive:
Header always set Content-Security-Policy-Report-Only "default-src 'self'; base-uri 'self'; form-action 'self'; frame-ancestors 'self'; object-src 'none'; script-src 'self' 'unsafe-eval' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; img-src 'self' data: blob: https://server.arcgisonline.com https://*.tile.openstreetmap.org; font-src 'self' data:; connect-src 'self'; media-src 'self' blob:; worker-src 'self' blob:;"
  1. Reload Apache:
sudo apachectl configtest
sudo systemctl reload apache2

Subpath installs (/opendatabio)

If your installation runs under a subpath (for example http://localhost/opendatabio), set in .env:

APP_URL=http://localhost/opendatabio
ASSET_URL=http://localhost/opendatabio

Then rebuild all generated assets:

sh scripts/build-assets.sh
php artisan optimize:clear

Notes

  1. https://server.arcgisonline.com and https://*.tile.openstreetmap.org are needed for map tiles.
  2. unsafe-inline / unsafe-eval are temporary compatibility flags; remove after hardening templates/assets.
  3. Keep Report-Only while tuning policy in production.

Configure your php.ini files. With libapache2-mod-php8.3, the relevant files are /etc/php/8.3/cli/php.ini and /etc/php/8.3/apache2/php.ini. An FPM installation uses /etc/php/8.3/fpm/php.ini instead.

For 200 MiB media and 1 GiB ZIPs, configure the website PHP:

upload_max_filesize = 1024M
post_max_size = 1100M
max_input_time = 1800

Add this directive to the application VirtualHost:

LimitRequestBody 1153433600

php --ini identifies CLI configuration only. Check the effective Apache or FPM configuration too. Keep allow_url_fopen=1; 512M is a starting point for memory_limit, subject to image dimensions and concurrency. After changing PHP/Apache settings, run sudo apache2ctl configtest and restart Apache; when using FPM, reload the corresponding PHP-FPM service too. See Administrator configuration for matching .env limits and verification.

Mysql Charset and Collation

  1. You should add the following to your configuration file (mariadb.cnf or my.cnf), i.e. the Charset and Collation you choose for your installation must match that in the ‘config/database.php’
[mysqld]
character-set-client-handshake = FALSE  #without this, there is no effect of the init_connect
collation-server      = utf8mb4_unicode_ci
init-connect          = "SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci"
character-set-server  = utf8mb4
log-bin-trust-function-creators = 1
sort_buffer_size = 256M  #large enough for geometry sort operations; do not use multi-gigabyte global values
max_allowed_packet=100M

# MariaDB only:
[mariadb]
innodb_log_file_size=300M
  1. If using MariaDB and you still have problems of type #1267 Illegal mix of collations, then check here on how to fix that,

Configure supervisord

Configure Supervisor, which is required for jobs. Create a file name opendatabio-worker.conf in the Supervisor configuration folder /etc/supervisor/conf.d/opendatabio-worker.conf with the following content:

touch /etc/supervisor/conf.d/opendatabio-worker.conf
echo ";--------------
[program:opendatabio-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /home/odbserver/opendatabio/artisan queue:work --sleep=3 --tries=1 --timeout=0 --memory=512
autostart=true
autorestart=true
user=odbserver
numprocs=8
redirect_stderr=true
stdout_logfile=/home/odbserver/opendatabio/storage/logs/supervisor.log
;--------------" > /etc/supervisor/conf.d/opendatabio-worker.conf

Folder permissions

  • Folders storage and bootstrap/cache must be writable by the Server user (usually www-data). Use group write permission (0775) rather than making them world-writable.
  • Config .env file requires 0640 permission.
  • This link has different ways to set up permissions for files and folders of a Laravel application. Below the preferred method:
cd /home/odbserver

#give write permissions to odbserver user and the apache user
sudo chown -R odbserver:www-data opendatabio
sudo find ./opendatabio -type f -exec chmod 644 {} \;
sudo find ./opendatabio -type d -exec chmod 755 {} \;  

#in these folders the server stores data and files.
#Make sure their permission is correct
cd /home/odbserver/opendatabio
sudo chgrp -R www-data storage bootstrap/cache
sudo chmod -R ug+rwx storage bootstrap/cache
sudo find storage bootstrap/cache -type d -exec chmod g+s {} \;

#make sure media folder has the correct permissions
sudo find ./storage/app/public/media  -type f -exec chmod 664 {} \;
sudo find ./storage/app/public/media  -type d -exec chmod 775 {} \;

#make sure the .env file has 640 permission
sudo chmod 640 ./.env

#verify that Apache can write to Laravel's writable directories
sudo -u www-data test -w storage
sudo -u www-data test -w bootstrap/cache

Install OpenDataBio

  1. Many Linux distributions (most notably Ubuntu and Debian) have different php.ini files for the command line interface and the Apache plugin. It is recommended to use the configuration file for Apache when running the install script, so it will be able to correctly point out missing extensions or configurations. To do so, find the correct path to the .ini file, and export it before using the php install command.

For example,

export PHPRC=/etc/php/8.3/apache2/php.ini
  1. The installation script will download the Composer dependency manager and all required PHP libraries listed in the composer.json file. However, if your server is behind a proxy, you should install and configure Composer independently. We have implemented PROXY configuration, but we are not using it anymore and have not tested properly (if you require adjustments, place an issue on GitLab).

  2. The script will prompt you configurations options, which are stored in the environment .env file in the application root folder.

You may, optionally, configure this file before running the installer:

  • Create a .env file with the contents of the provided cp .env.example .env
  • Read the comments in this file and adjust accordingly.
  • Make sure ASSET_URL is correct for your deployment URL/subpath.
  • Decide whether Pl@ntNet will use a shared PLANTNET_API_KEY, personal user keys, or both. A server key is optional when personal keys are enabled. See Administrator configuration.
  1. Run the installer. Select the Apache profile explicitly:
cd /home/odbserver/opendatabio
php install apache

The installer builds the Vite frontend and publishes the Livewire assets after .env is configured. These generated files are intentionally not versioned. Node.js 22 and npm must therefore be installed on the server.

  1. Seed data - the script above will ask if you want to install seed data for Locations and Taxons - seed data is version specific. Check the seed data repository version notes.

Validate the completed installation:

php artisan migrate:status
php artisan locales:audit
composer check-platform-reqs
sudo supervisorctl status
redis-cli ping
curl -I http://localhost/opendatabio/
curl -I http://localhost/opendatabio/build/manifest.json

Optional assisted translation

User-maintained names and descriptions must contain their essential fields in the primary locale. Other enabled content locales are optional, and any enabled locale containing text may be used as the source of an assisted translation. A translation is always presented for review and is never saved automatically.

Assisted translation is disabled by default. Google Cloud Translation v3 is the only currently supported provider:

USER_TRANSLATION_PROVIDER=google
GOOGLE_CLOUD_PROJECT=your-project-id
GOOGLE_TRANSLATION_LOCATION=global
GOOGLE_APPLICATION_CREDENTIALS=/secure/path/service-account.json

Enable the Cloud Translation API and billing, grant the service account only the required translation permission, keep its JSON file outside the repository, and configure billing quotas/alerts. Google currently applies a monthly free usage credit to the first 500,000 NMT characters; billing is still required and usage beyond the credit is charged. Check current Google pricing before enabling the feature. When the server provides Application Default Credentials, GOOGLE_APPLICATION_CREDENTIALS may remain empty.

After editing .env, verify without sending text, then make one live request:

php artisan optimize:clear
php artisan translations:check --source=en --target=es
php artisan translations:check --source=en --target=es --live

The live command sends the short phrase shown by the command to the selected provider. Provider failure does not prevent OpenDataBio from operating; leave USER_TRANSLATION_PROVIDER empty to disable the feature.

Installation issues

There are countless possible ways to install the application, but they may involve more steps and configurations.

  • if you browser return 500|SERVER ERROR you should look to the last error in storage/logs/laravel.log. If you have ERROR: No application encryption key has been specified run:
php artisan key:generate
php artisan config:cache
  • If you receive the error “failed to open stream: Connection timed out” while running the installer, this indicates a misconfiguration of your IPv6 routing. The easiest fix is to disable IPv6 routing on the server.
  • If you receive errors during the random seeding of the database, you may attempt to remove the database entirely and rebuild it. Of course, do not run this on a production installation.
php artisan migrate:fresh
  • You may also replace the Locations and Taxons tables with seed data after a fresh migration using:
php seedodb

Post-install configs

  • If your import/export jobs are not being processed, make sure Supervisor is running with sudo systemctl enable --now supervisor, and check storage/logs/supervisor.log.
  • You can change several configuration variables for the application. The most important of those are probably set by the installer, and include database configuration and proxy settings, but many more exist in the .env and config/app.php files. In particular, you may want to change the language, timezone and e-mail settings. Run php artisan config:cache after updating the config files.
  • In order to stop search engine crawlers from indexing your database, add the following to your “robots.txt” in your server root folder (in Debian, /var/www/html):
User-agent: *
Disallow: /

Updating an existing Apache installation

Before updating, back up your database, .env, and storage/app/public/media. Before running commands, review config diffs for the target version:

  • Compare .env with .env.example (including ASSET_URL)
  • Check PHP settings (php.ini in CLI and FPM/Apache)
  • Check Supervisor worker settings
  1. Put the application in maintenance mode:
cd /home/odbserver/opendatabio
php artisan down
  1. Update source code to the target version:
git fetch --tags
git checkout <target-tag-or-branch>
  1. Update dependencies and apply database migrations:
composer install --no-dev --optimize-autoloader
php artisan migrate:status
php artisan migrate --force
  1. Rebuild frontend and Livewire assets after .env updates:
sh scripts/build-assets.sh
  1. Refresh caches and restart queue workers:
php artisan optimize:clear
php artisan config:cache
php artisan queue:restart
echo "" > storage/logs/laravel.log
  1. Bring the application back online:
php artisan up

If the target version includes new environment variables (compare yours with the contents of .env.example), add them to .env before running asset/cache commands.

Storage & Backups

You may change storage configurations in config/filesystem.php, where you may define cloud based storage, which may be needed if have many users submitting media files, requiring lots of drive space.

  1. Data downloads are queue as jobs and a file is written in a temporary folder, and the file is deleted when the job is deleted by the user. This folder is defined as the download disk in filesystem.php config file, which point to storage/app/public/downloads. UserJobs web interface difficult navigation will force users to delete old jobs, but a cron cleaning job may be advisable to implement in your installation;
  2. Media files are by default stored in the media disk, which place files in folder storage/app/public/media;
  3. For regular configuration create both directories storage/app/public/downloads and storage/app/public/media with writable permissions by the Server user, see below topic;
  4. Remember to include media folder in a backup plan;

Use the supported database snapshot and incremental storage workflow described in Backup and restore.

Scheduler and private import storage

Alongside Supervisor workers, configure the scheduler in the application user’s crontab (odbserver in this example), once per minute:

* * * * * cd /home/odbserver/opendatabio && php artisan schedule:run >> storage/logs/scheduler.log 2>&1

It recovers pending queue deliveries and cleans expired temporary files. Apache, workers and scheduler must be able to read, write and remove files under storage/app/imports. When they use different users, share a group:

cd /home/odbserver/opendatabio
sudo usermod -aG www-data odbserver
sudo mkdir -p storage/app/imports
sudo chgrp -R www-data storage/app/imports
sudo find storage/app/imports -type d -exec chmod 2770 {} +
sudo find storage/app/imports -type f -exec chmod 0660 {} +

Adapt the user and group to the server. Restart worker processes after changing user groups; queue:restart alone may not refresh groups inherited from an already-running Supervisor. Verify effective access for both users. The setgid bit (2) preserves the group on new directories; the application’s private disk uses group permissions for new files. Keep this directory outside public, without an HTTP alias or public symlink, and do not use 777.

3 - Docker Installation

How to install OpenDataBio with Docker

The easiest way to install and run OpenDataBio is using Docker and the docker configuration files provided, which contain the required configuration to run OpenDataBio. It uses nginx, MySQL, and Supervisor for queues

Choose a profile

OpenDataBio provides two Compose profiles:

  1. docker-compose.yml: development and local testing, with source bind mounts, phpMyAdmin, and ports 8081/8082.
  2. docker-compose.prod.yml: production-oriented, immutable application images, no source bind mounts or phpMyAdmin, a dedicated database user, healthchecks, and named volumes.

The Makefile does not ask which profile you want. The target selects it:

  • make docker-init starts the development profile and uses .env;
  • make init-prod starts the production profile and uses .env.production.

For a production installation, or a production-profile test alongside an existing Apache installation, use make init-prod.

If you accidentally start make docker-init, interrupt it with Ctrl+C and stop only the development Compose project:

docker compose -p odb down

Do not add -v, because that option deletes the selected Docker project’s volumes.

Production installation

1. Prepare the environment

cd opendatabio
cp .env.production.example .env.production
nano .env.production
chmod 600 .env.production

Compose reads .env.production on the host and injects its values into the application containers. The production image intentionally does not contain a /var/www/html/.env file; Laravel reads the injected environment variables.

At minimum, replace:

APP_URL=https://data.example.org
ASSET_URL=https://data.example.org
APP_FORCE_HTTPS=true
APP_HTTP_PORT=80

DB_DATABASE=opendatabio
DB_USERNAME=opendatabio
DB_PASSWORD=a-strong-application-password
DB_ROOT_PASSWORD=a-different-strong-root-password

Also decide whether Pl@ntNet will use a server key shared by the installation, personal keys stored by users, or both. An empty PLANTNET_API_KEY is valid when PLANTNET_ALLOW_USER_KEYS=true; PLANTNET_SERVER_FALLBACK determines whether users without a personal key may use the server key. See Administrator configuration before starting the containers.

If TLS terminates in an external reverse proxy, keep the application containers on a private HTTP network/port and configure the proxy to forward the original host and protocol.

For a local production-profile test while Apache already uses port 80:

APP_URL=http://localhost:8083
ASSET_URL=http://localhost:8083
APP_FORCE_HTTPS=false
APP_HTTP_PORT=8083

2. Build and initialize

The initialization script:

  1. generates APP_KEY only when it is empty;
  2. builds the frontend once and copies the same generated assets into the self-contained PHP and nginx images;
  3. waits for MySQL and Redis healthchecks;
  4. runs migrations;
  5. configures interface and user-content locales;
  6. caches Laravel configuration/routes/views;
  7. starts nginx and queue workers;
  8. runs the locale audit.
make init-prod

To initialize a new production database and then optionally import the version-compatible reference data for locations and taxons:

make init-prod SEED=1

The seed step is interactive and requires typing PROCEED. It replaces the current location and taxon reference tables, so use it only for a new installation or when the version-specific upgrade notes explicitly instruct you to replace them. To run it later against an initialized production installation:

make seed-prod

The production seed runs entirely inside the odb-prod containers and does not read, modify, or remove files from an Apache installation’s local storage/.

The default locale selection is:

interface: en,es,pt-br
user-entered content: pt-br

Override it for the initialization command when required:

ODB_INTERFACE_LOCALES=en,es,pt-br \
ODB_CONTENT_LOCALES=pt-br,en \
make init-prod

ODB_CONTENT_LOCALES initializes the content locale selection. The primary locale is always enabled and essential translated fields remain required in that locale; translations in other enabled locales are optional.

Assisted translation is disabled by default. To enable the only currently supported provider, configure Google Cloud Translation v3 before make init-prod. Google currently applies a monthly free usage credit to the first 500,000 NMT characters, but billing is required and excess usage is charged. Configure API quotas and billing alerts and verify current pricing.

mkdir -p docker-secrets
cp /secure/source/google-translation.json docker-secrets/
chmod 700 docker-secrets
chmod 600 docker-secrets/google-translation.json
USER_TRANSLATION_PROVIDER=google
GOOGLE_CLOUD_PROJECT=your-project-id
GOOGLE_TRANSLATION_LOCATION=global
GOOGLE_APPLICATION_CREDENTIALS=/run/secrets/google-translation.json

The ignored docker-secrets directory is mounted read-only at /run/secrets inside both Laravel and queue containers. Never add its files to the image or repository.

Test the selected provider:

docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml exec -T -u www-data laravel php artisan translations:check --source=en --target=es
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml exec -T -u www-data laravel php artisan translations:check --source=en --target=es --live

Do not regenerate APP_KEY after data has been stored.

3. Validate production

docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml ps
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml exec -T redis redis-cli ping
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml exec -T -u www-data laravel php artisan migrate:status
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml exec -T -u www-data laravel php artisan locales:audit
curl -I "${APP_URL:-http://localhost:8083}/"

Inspect logs:

docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml logs --tail=200 nginx
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml logs --tail=200 laravel
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml logs --tail=200 supervisord
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml logs --tail=200 mysql

CSP is initially sent as Content-Security-Policy-Report-Only. Test the complete UI and review browser reports before enforcing it in docker/prod/nginx.conf.

Development quick start

This section is only for a development checkout. Do not follow it in the same checkout that currently serves an Apache installation: the development profile uses .env, bind-mounts the source tree, and may write to local application directories.

Prerequisites:

  1. Docker with the Compose v2 plugin (docker compose).
  2. Linux/macOS: a user allowed to access the Docker socket, or use a rootless Docker installation.
  3. Windows: Docker Desktop with WSL2/Hyper-V.
  4. make for the short commands below.
  5. Node.js 22 and npm on the host. The development source bind mount replaces the image’s application tree, so make docker-init builds the unversioned frontend and Livewire assets in the checkout.

Preserve any non-Docker environment file first:

cp .env .env.backup.apache
cp .env.docker .env
make docker-init

For a new development database, import the optional location and taxon reference data after initialization:

make seed-odb

Alternatively, run both steps with make docker-init SEED=1. The seed replaces the current location and taxon reference tables and asks for explicit confirmation.

The development application is available at http://localhost:8081 and phpMyAdmin at http://localhost:8082.

Default login:

user: admin@example.org
password: password1

Change the password after installation.

The initialization command is for a new installation. It does not overwrite an existing APP_KEY, but migrations and seed operations must still be treated as database changes.

Make commands

Build and database

  1. make docker-init - copy .env.docker if .env is absent, build/start containers, install dependencies, generate a missing key, migrate, and create the storage link
  2. make build - build containers
  3. make key-generate - generate an app key only if one does not already exist
  4. make composer-install - install PHP dependencies
  5. make composer-update - update php dependencies
  6. make composer-dump-autoload - execute composer dump-autoload within container
  7. make migrate - create or update the database
  8. make drop-migrate - delete and recreate the database
  9. make seed-odb - seed the database with locations and taxons
  10. make seed-prod - seed a production Docker database without touching host storage
  11. make init-prod - build and initialize the production profile
  12. make start-prod / make stop-prod - start or stop the production profile

Container access

  1. make start - start all containers
  2. make stop - stop all containers
  3. make restart - restart all containers
  4. make ssh - enter the main laravel app container
  5. make ssh-mysql - enter the mysql container, so you may the log to the database using mysql -uUSER -pPWD
  6. make mysql - enter the docker mysql console
  7. make ssh-nginx - enter the nginx container
  8. make ssh-supervisord - enter the supervisord container

Maintenance commands

  1. make optimize - clean caches and log files
  2. make info - show app info
  3. make logs - show laravel logs
  4. make logs-mysql - show mysql logs
  5. make logs-nginx - show nginx logs
  6. make logs-supervisord - show supervisor logs

Data persistence and clean resets

MySQL, Redis, and production media use named volumes. Rebuilding an image does not delete these volumes.

docker volume ls

To reset only a development test project, including its database:

docker compose -p odb down -v --remove-orphans

For the production profile:

docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml down -v --remove-orphans

Updating an existing Docker installation

Before updating, follow Backup and restore to back up the database, media, and persistent dataset versions. For new production deployments, that page also explains how ODB_STORAGE_SOURCE can expose storage as a host bind mount for efficient incremental rsync backups. Before running commands, review config diffs for the target version:

  • Compare .env.production with .env.production.example (including APP_URL and ASSET_URL)
  • Check PHP settings from the target profile (docker/prod/php.ini or your custom PHP config)
  • Check Supervisor settings (docker/general/supervisord.conf or your deployment equivalent)
  1. Update source code to the target version:
cd opendatabio
git fetch --tags
git checkout <target-tag-or-branch>
  1. Build the new immutable images:
make build-prod
  1. Put the existing application in maintenance mode and run migrations with the new PHP image:
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml exec -T -u www-data laravel php artisan down
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml up -d mysql redis laravel
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml exec -T -u www-data laravel php artisan migrate --force
  1. Refresh Laravel caches and replace the web/worker containers:
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml exec -T -u www-data laravel php artisan optimize
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml up -d --force-recreate nginx supervisord scheduler
  1. Return the application to service and validate:
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml exec -T -u www-data laravel php artisan up
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml exec -T -u www-data laravel php artisan locales:audit
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml ps

Composer dependencies and frontend assets are built into the production images; do not run composer update or npm run build interactively inside production containers. If the new version introduces .env.production keys, add them before building/recreating containers.

Media imports: limits, volumes and scheduler

Both profiles include scheduler, running php artisan schedule:work. Keep it active alongside workers for delivery recovery and temporary-file cleanup. Production mounts the private odbimports volume at /var/www/html/storage/app/imports in the application, workers and scheduler; Nginx does not expose it. Development shares this path through the project bind mount. Preserve this storage across restarts to support resuming imports. Distributed installations must share these files and the Redis lock store between all participating processes.

See Administrator configuration for variables and limits. PHP and Nginx configuration are built into images: changes to docker/prod/php.ini, docker/dev/php.ini or Nginx configuration require rebuilding affected images and recreating containers. A restart alone does not pick up image changes or Compose environment changes. After editing the active environment file, recreate application, worker and scheduler services and rebuild Laravel configuration cache in the updated environment. Recreate Nginx when its image/configuration changes. Do not remove volumes to apply settings. If an external proxy is present, configure its upload limit and trusted proxies as explained in the Nginx installation guide.

4 - Nginx Installation

How to install OpenDataBio with nginx

These instructions are for an nginx-based installation. If you prefer Apache, use the Apache installation page.

Server requirements

  1. Supported PHP version >= 8.2 (8.3 recommended).
  2. Web server: nginx.
  3. SQL database: MySQL or MariaDB (tested with MySQL 8.0 and MariaDB 10.6+).
  4. Required PHP extensions: openssl, pdo, pdo_mysql, mbstring, tokenizer, xml, dom, gd, exif, bcmath, zip, curl, redis.
  5. Redis for queues/cache.
  6. Tectonic for label PDF generation.
  7. Pandoc for bibliographic rendering (recommended).
  8. Supervisor for background jobs.

Prepare the server

The example below uses Ubuntu/Debian packages and PHP 8.3. Install nginx, PHP-FPM and the same application services and extensions required by the Apache installation:

sudo apt-get install software-properties-common
sudo add-apt-repository ppa:ondrej/php
sudo apt-get update
sudo apt-get install nginx mysql-server redis-server tectonic pandoc supervisor \
 php8.3-fpm php8.3-cli php8.3-intl php8.3-mysql php8.3-sqlite3 php8.3-gd \
 php8.3-mbstring php8.3-xml php8.3-bcmath php8.3-zip php8.3-curl php8.3-redis

sudo systemctl enable --now nginx php8.3-fpm redis-server supervisor

php -m | grep -E 'mbstring|xml|gd|mysql|redis|bcmath|pcntl|zip'
systemctl status php8.3-fpm --no-pager

Follow the dedicated-user, download, MySQL, Supervisor and permissions sections of the Apache installation. For nginx, configure both /etc/php/8.3/cli/php.ini and /etc/php/8.3/fpm/php.ini, then restart php8.3-fpm.

Nginx site config

Create your site config file (example):

sudo nano /etc/nginx/sites-available/opendatabio

Use this base server block (adjust paths/domain):

server {
    listen 80;
    server_name your-domain.example;

    root /home/odbserver/opendatabio/public;
    index index.php index.html;

    charset utf-8;
    client_max_body_size 1100M;

    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass unix:/var/run/php/php8.3-fpm.sock;
        fastcgi_index index.php;
        include fastcgi.conf;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
        fastcgi_read_timeout 300s;
        fastcgi_send_timeout 300s;
    }

    location ~ /\. {
        deny all;
    }
}

Enable and reload:

sudo ln -s /etc/nginx/sites-available/opendatabio /etc/nginx/sites-enabled/opendatabio
sudo rm -f /etc/nginx/sites-enabled/default
sudo nginx -t
sudo systemctl reload nginx

If the symlink already exists, do not recreate it. Never reload nginx unless nginx -t succeeds.

HTTPS

The port 80 block is suitable for initial validation. A public production installation must use HTTPS. Configure a certificate directly in nginx (for example with your distribution’s Certbot integration) or terminate TLS in a trusted reverse proxy. Redirect HTTP to HTTPS only after the HTTPS virtual host has been tested.

Set the public URL consistently:

APP_URL=https://your-domain.example
ASSET_URL=https://your-domain.example
APP_FORCE_HTTPS=true

When TLS terminates at a reverse proxy, forward the original Host and X-Forwarded-Proto headers and restrict direct access to the backend.

Content Security Policy (CSP)

Edit the same nginx site file and add inside the server { ... } block:

add_header Content-Security-Policy-Report-Only "default-src 'self'; base-uri 'self'; form-action 'self'; frame-ancestors 'self'; object-src 'none'; script-src 'self' 'unsafe-eval' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; img-src 'self' data: blob: https://server.arcgisonline.com https://*.tile.openstreetmap.org; font-src 'self' data:; connect-src 'self'; media-src 'self' blob:; worker-src 'self' blob:;" always;

Then reload:

sudo nginx -t
sudo systemctl reload nginx

Notes:

  1. Start with Report-Only, then move to enforced CSP after validating logs.
  2. https://server.arcgisonline.com and https://*.tile.openstreetmap.org are required for map tiles.

Deployment URL

The server block above publishes OpenDataBio at the root of a dedicated host. Use matching values in .env:

APP_URL=https://your-domain.example
ASSET_URL=https://your-domain.example

Then rebuild all generated assets:

sh scripts/build-assets.sh
php artisan optimize:clear

Installing nginx under a subpath such as /opendatabio also requires subpath-aware location, alias and FastCGI rules; changing only .env is not sufficient. Prefer a dedicated host or subdomain. If a subpath is mandatory, use the tested Apache configuration or provide and test a deployment-specific nginx configuration before exposing it publicly.

Shared application setup

To avoid repeating the same instructions, use these sections from Apache installation (they also apply to nginx deployments):

  1. PHP settings in Apache Installation, using the FPM path stated above
  2. Configure supervisord in Apache Installation
  3. Folder permissions in Apache Installation
  4. Install OpenDataBio as described in Apache Installation, but run php install nginx
  5. Post-install configs in Apache Installation

Import limits and proxies

The 1100 MiB limit above accommodates a 1 GiB ZIP. Also configure PHP-FPM and .env as described in Administrator configuration. Use the scheduler and private storage permissions described in the standard installation. Run sudo nginx -t, then reload Nginx and the corresponding PHP-FPM service after changing their configuration.

TrustProxies::$proxies = null is suitable when PHP receives the original protocol and host correctly. If an external proxy terminates HTTPS and forwards HTTP, configure trusted proxy IPs/networks and forwarded headers; otherwise URLs and signed uploads may fail. Do not use "*" just to enable imports. APP_URL does not replace trusted proxy configuration.

5 - Customize Installation

How to customize the web interface!

Simple changes that can be implemented in the layout of a OpenDataBio web site

Logo and BackGround Image

To replace the Navigation bar logo and the image of the landing page, just put your image files replacing the files in /public/custom/ without changing their names.

Texts and Info

To change the welcome text of the landing page, change the values of the array keys in the following files:

  • /lang/en/customs.php
  • /lang/pt-br/customs.php
  • /lang/es/customs.php
  • Do not remove the entry keys. Set to null to suppress from appearing in the footer and landing page.

Local Documentation

You can add documentation in *.md format to the repository in files located in the following folders:

  • /resources/docs/en/*
  • /resources/docs/pt/*

This space is reserved for administrators to set documentation and custom directives for users of a specific OpenDataBio installation. For example, this is a place to include a code of conduct for users, information on who to contact to become a full user, specific tutorials, and so on.

  1. If you want to change the color of the top navigation bar and the footer, replace the Bootstrap 5 CSS class in the corresponding templates under /resources/views/layouts.
  2. You may add additional html to the footer and navbar, change logo size, etc… as you wish.

6 - Upgrade OpenDataBio

Safe upgrade instructions for OpenDataBio installations

Use this page for the common deployment sequence. Before upgrading, read UPGRADES_NOTES.md from the target version of OpenDataBio. It is the authoritative source for version-specific prerequisites, storage permissions, migrations, backfills and repair commands; those instructions are intentionally not duplicated here.

Before you start

  1. Read the target version’s UPGRADES_NOTES.md and release notes. Record every required preflight check and post-migration command before starting.
  2. Back up at least:
    • Database dump
    • .env
    • the complete storage/app tree, including media, generated exports and persisted dataset-version archives Follow Backup and restore to create and verify the database dump and incremental storage copy.
  3. Compare current config files against target-version templates/settings:
    • .env against .env.example (including ASSET_URL)
    • Supervisor worker config (/etc/supervisor/conf.d/opendatabio-worker.conf or container equivalent)
    • PHP config (php.ini for CLI and FPM/Apache)
  4. Plan a maintenance window for production.

Upgrade (Apache or nginx installation)

  1. Put application in maintenance mode:
cd /home/odbserver/opendatabio
php artisan down
  1. Update source code:
git fetch --tags
git checkout <target-tag-or-branch>
  1. Install dependencies and run migrations:
composer install --no-dev --optimize-autoloader
php artisan migrate:status
php artisan migrate --force
  1. Run the post-migration commands listed in the target version’s UPGRADES_NOTES.md, in the documented order. Some commands submit background UserJobs; keep the application in maintenance mode and monitor them until completion when the notes require it.
  2. Rebuild frontend and Livewire assets after .env changes (required when ASSET_URL changes):
sh scripts/build-assets.sh
  1. Refresh caches and restart workers:
php artisan optimize:clear
php artisan config:cache
php artisan queue:restart
systemctl restart supervisor.service
# Restart the web server and PHP-FPM as appropriate for your installation.
  1. Verify the application, queue workers, logs and the operations named in the upgrade notes, then bring the application back online:
php artisan up

Upgrade (Docker installation)

The commands below are for the production profile (docker-compose.prod.yml and .env.production). For the bind-mounted development profile, rebuild with make docker-init only when you intentionally want a development environment.

  1. Update source code and review configuration changes:
cd opendatabio
git fetch --tags
git checkout <target-tag-or-branch>

Compare .env.production with .env.production.example, without replacing the existing application key or secrets.

  1. Put the current application in maintenance mode and build the new images:
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml exec -T -u www-data laravel php artisan down
make build-prod
  1. Start the new application container and run migrations:
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml up -d mysql redis laravel
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml exec -T -u www-data laravel php artisan migrate:status
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml exec -T -u www-data laravel php artisan migrate --force
  1. Run the target version’s UPGRADES_NOTES.md post-migration commands inside the application container, in the documented order, and monitor any UserJobs they submit.
  2. Refresh caches and recreate nginx and queue workers. Composer dependencies and frontend assets are already included in the production images:
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml exec -T -u www-data laravel php artisan optimize
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml up -d --force-recreate nginx supervisord
  1. Validate, then return the application to service:
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml exec -T -u www-data laravel php artisan locales:audit
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml exec -T -u www-data laravel php artisan up
docker compose --env-file .env.production -p odb-prod -f docker-compose.prod.yml ps

Environment variables

For Apache/nginx, compare .env with .env.example. For production Docker, compare .env.production with .env.production.example. Follow UPGRADES_NOTES.md for variables whose value or meaning changed; verify APP_FORCE_HTTPS and ASSET_URL.

Rollback strategy

If something fails after migration:

  1. Keep maintenance mode on.
  2. Restore the database, .env, and the matching storage/app backup. Database rows and persisted dataset-version archives must be restored as one snapshot.
  3. Checkout the previous known-good tag.
  4. Rebuild dependencies/containers and validate logs before php artisan up.

7 - Administrator configuration

Configure external services, e-mail, translation and application locales.

OpenDataBio works without optional external services, but administrators must make explicit decisions about e-mail delivery, taxonomic services, assisted translation and the locales available to users. Keep credentials in the installation environment file; never commit them to the source repository.

The shipped environment templates leave USER_TRANSLATION_PROVIDER and all service credentials empty. Apache, nginx and Docker installation therefore do not silently enable translation or make an external request. During the interactive direct-server installer, answering the optional Google question with its default no keeps the feature disabled. Configure services only after deciding who will manage credentials, quotas and operating costs.

After changing .env or .env.production, refresh the application configuration:

php artisan optimize:clear
php artisan config:cache

For Docker production, run Artisan inside the application container and use paths as seen inside that container.

Media imports and upload limits

Use this reference set in the installation’s .env (.env.production for production Docker). Existing installations must add the variables: updating a template does not change the active environment file.

MEDIA_MAX_FILE_SIZE=209715200
MEDIA_IMPORT_MAX_ARCHIVE_KB=1048576
MEDIA_IMPORT_MAX_ENTRY_BYTES=209715200
MEDIA_IMPORT_MAX_UNCOMPRESSED_BYTES=2147483648
MEDIA_IMPORT_MAX_ENTRIES=5000
MEDIA_IMPORT_MAX_COMPRESSION_RATIO=100
MEDIA_IMPORT_CHUNK_SIZE=50
MEDIA_IMPORT_CHUNK_SECONDS=30
MEDIA_IMPORT_WORKER_TIMEOUT=300
MEDIA_IMPORT_CONCURRENT_CHUNKS=1
MEDIA_IMPORT_RETENTION_DAYS=7
MEDIA_IMPORT_DELIVERY_RETRY_SECONDS=600
MEDIA_IMPORT_LOCK_STORE=redis
QUEUE_CONNECTION=redis
REDIS_DB=0
REDIS_CACHE_DB=1

MEDIA_MAX_FILE_SIZE limits each media file to 200 MiB and also contributes to file limits for other imports. ZIPs allow 1 GiB, each extracted entry 200 MiB, and total extracted content 2 GiB. Sizes are bytes except MEDIA_IMPORT_MAX_ARCHIVE_KB, which is KiB. The 5000-entry ceiling includes metadata and directories; the compression ratio ceiling applies per entry.

Each execution processes up to 50 rows or a 30-second budget checked between files. One file can exceed that budget; the 300-second timeout covers the whole execution, including ZIP preparation. Workers need pcntl to enforce it. One concurrent media execution is the starting point for controlling CPU, memory and disk usage in multiuser installations. Larger upload limits do not require more concurrency.

Redis carries jobs and shared locks; the database stores progress. Keep the queue Redis database (0) separate from cache/locks (1). The scheduler recovers interrupted or lost deliveries; 600 seconds is the reference recovery interval, not the normal delay between chunks. Temporary files for inactive failed or cancelled imports expire after 7 days; an expired import requires a new upload.

Align application, PHP and web server limits

The lowest limit anywhere in the upload path wins. Livewire follows the configured ZIP/media limits, but .env does not configure PHP, Apache, Nginx or external proxies. For this reference set, configure the PHP serving the website:

upload_max_filesize = 1024M
post_max_size = 1100M
max_input_time = 1800

Use client_max_body_size 1100M; in Nginx and LimitRequestBody 1153433600 in Apache. The margin above 1 GiB accommodates the rest of the request. Apply compatible limits to external proxies too. These uploads do not require increasing MySQL’s max_allowed_packet: ZIPs and media are stored on disk.

max_input_time is an initial allowance for slow uploads; buffering and server/proxy timeouts also affect the result. Do not increase every timeout to match the total batch duration: processing runs in the queue. Size memory for decoded images and concurrency, not ZIP size. A 200 MiB file ceiling does not guarantee every image fits in available memory. Allow disk space for temporary uploads, extraction and final media across users; the extracted-content limit is per ZIP, not a global storage quota.

Apply and verify configuration changes

  1. Edit the environment file actually used by the installation.
  2. Run php artisan config:cache and php artisan queue:restart, keeping Supervisor active to relaunch workers.
  3. Check the website’s php.ini: php --ini only describes CLI PHP. Apache mod_php and PHP-FPM use their own configuration; pool/VirtualHost overrides can also change effective values.
  4. Validate and reload Apache/Nginx; restart or reload PHP-FPM when applicable.
  5. For Docker, follow the installation page’s rebuild/recreation instructions.
  6. Test a representative ZIP and follow its UserJob to completion, including failure recovery. Check the scheduler, storage access and worker logs.

Google Cloud Translation

OpenDataBio uses Cloud Translation Advanced (v3) only to translate user-maintained content in edit forms. Interface strings under lang/ are not sent to Google. Generated text fills missing fields and must be reviewed before the record is saved. Import jobs never invoke Google automatically.

Google requires billing even when usage remains within an applicable free credit. Review current pricing, set a budget alert, and restrict quotas before enabling the service.

  1. Sign in to the Google Cloud console.
  2. Create or select a project and record its project ID.
  3. Link a billing account.
  4. Under APIs & Services, enable Cloud Translation API.
  5. Under IAM & Admin → Service Accounts, create a dedicated service account for OpenDataBio.
  6. Grant that service account Cloud Translation API User (roles/cloudtranslate.user). Do not grant Owner, Editor, Admin, or the Cloud Translation service-agent role.
  7. Create a JSON key for the service account and download it once. Store it outside the repository, readable by the web-server user and not by other system users.
  8. Configure:
USER_TRANSLATION_PROVIDER=google
GOOGLE_CLOUD_PROJECT=your-google-project-id
GOOGLE_TRANSLATION_LOCATION=global
GOOGLE_APPLICATION_CREDENTIALS=/absolute/server/path/google-translation.json
USER_TRANSLATION_MAX_CHARACTERS_PER_REQUEST=10000
USER_TRANSLATION_TIMEOUT=30

For Apache or nginx, the credential file and all parent directories must be accessible to the PHP/web-server user. A typical permission arrangement is:

sudo chgrp www-data /secure/path/google-translation.json
sudo chmod 750 /secure/path
sudo chmod 640 /secure/path/google-translation.json

For Docker production, place the file under the untracked docker-secrets/ directory, mount it read-only, and set the environment value to its container path, for example:

GOOGLE_APPLICATION_CREDENTIALS=/run/secrets/google-translation.json

Verify configuration first without an external request, then with a short live translation:

php artisan translations:check --source=en --target=es
php artisan translations:check --source=en --target=es --live

Disable assisted translation at any time by leaving USER_TRANSLATION_PROVIDER empty.

Tropicos

Tropicos Web Services requires a personal API key in every request.

Configuring it improves taxonomic curation by allowing editors to search and validate published botanical names against Tropicos instead of relying only on the local library or other external sources.

  1. Open the Tropicos API-key request page.
  2. Submit the requested contact and intended-use information.
  3. Store the issued key in the installation environment:
MOBOT_API_KEY=your-tropicos-api-key

If no key is configured, OpenDataBio remains functional and other configured taxonomic services, primarily GBIF, can still be used. Do not expose the key in client-side code or commit it.

Pl@ntNet image identification

OpenDataBio can send one to five images of the same plant to Pl@ntNet and show species, genus and family candidates for human review. Results are cached, an identification is never changed automatically, and applying a candidate still requires the normal OpenDataBio permissions.

Create a free developer account on the Pl@ntNet sign-up page, then generate or manage the key under API-key settings. See the official getting-started guide and API reference for current quotas, terms and request details.

OpenDataBio supports two credential sources:

  • Server key: configured by an administrator and shared by users who do not have a personal key. Its quota is shared by the installation.
  • Personal key: stored by a registered user in Edit profile. It is encrypted at rest, takes precedence over the server key, and uses that Pl@ntNet account’s independent quota.

For an installation that provides a shared server key, configure:

PLANTNET_API_KEY=your-server-plantnet-key
PLANTNET_ALLOW_USER_KEYS=true
PLANTNET_SERVER_FALLBACK=true
PLANTNET_DAILY_REQUEST_LIMIT=500
PLANTNET_DAILY_USER_LIMIT=20

With this configuration, personal keys are preferred. Users without one fall back to PLANTNET_API_KEY. PLANTNET_DAILY_REQUEST_LIMIT is a local safety ceiling per credential; PLANTNET_DAILY_USER_LIMIT limits a non-administrator’s use of the shared server key. Pl@ntNet’s reported remote balance is also respected. Administrators are exempt from the per-user shared-key limit, but not from the credential’s local or remote quota.

To require personal keys and avoid a shared installation quota:

PLANTNET_API_KEY=
PLANTNET_ALLOW_USER_KEYS=true
PLANTNET_SERVER_FALLBACK=false
PLANTNET_DAILY_REQUEST_LIMIT=500

In this mode, Pl@ntNet remains available to every user who stores a valid personal key. Users without one do not see the identification action. Leaving PLANTNET_API_KEY empty does not disable personal keys.

To disable personal credentials while retaining only the installation key, set PLANTNET_ALLOW_USER_KEYS=false. If neither an allowed personal key nor an enabled server fallback is available, Pl@ntNet identification is unavailable; the rest of OpenDataBio continues to work.

Requests are sent by the OpenDataBio server, not directly by the browser. Therefore, normal OpenDataBio use does not require enabling Expose my API key or adding the OpenDataBio URL under Pl@ntNet CORS authorized domains. If you deliberately expose the key in Pl@ntNet settings, follow Pl@ntNet’s current instructions and authorize the server IP for non-CORS requests.

After changing the environment, run the configuration refresh commands shown at the top of this page. Never commit either server or personal keys.

E-mail

E-mail is used for password recovery, optional address verification, dataset requests and job notifications. Production installations should use a dedicated SMTP account or transactional provider.

Without working e-mail, administrators must support account recovery manually, users can miss access-request decisions and long-running background work cannot reliably notify them when attention is required.

MAIL_MAILER=smtp
MAIL_HOST=smtp.example.org
MAIL_PORT=587
MAIL_USERNAME=opendatabio@example.org
MAIL_PASSWORD=replace-with-secret
MAIL_ENCRYPTION=tls
MAIL_FROM_ADDRESS=opendatabio@example.org
MAIL_FROM_NAME="${APP_NAME}"
MAIL_VERIFY_PEER=true
MAIL_VERIFY_PEER_NAME=true
MAIL_ALLOW_SELF_SIGNED=false
EMAIL_VERIFICATION_ENABLED=false

Use port 465 with the encryption required by your provider when applicable. Keep certificate verification enabled in production. Enable EMAIL_VERIFICATION_ENABLED only after outbound delivery and password recovery have been tested. Queue workers must be running for queued notifications.

Locale responsibilities

OpenDataBio keeps three concepts separate:

  • APP_LOCALE is the permanent primary locale and is always required for translatable content.
  • Interface locales have complete application translation files under lang/<code>/.
  • Content locales are languages in which users may maintain UserTranslation values. They do not require an interface translation.

ODB_INTERFACE_LOCALES and ODB_CONTENT_LOCALES initialize a new installation. For an existing installation, use Admin → Application locales or:

php artisan locales:configure --interfaces=en,es,pt-br --content=en,es,pt-br
php artisan locales:audit

Add a content-only locale

  1. In Admin → Application locales, add a normalized code such as fr or es-mx and a human-readable name.
  2. Enable User content.
  3. Do not enable Interface unless lang/<code>/ is complete.
  4. Add a provider mapping in config/user-translation.php if the translation provider does not accept the application code directly.

Existing records are not automatically backfilled. Users may open their edit forms and explicitly generate missing translations. Bulk imports must supply their translations explicitly.

Add a new interface locale

  1. Copy the complete key structure from an existing lang/<code>/ directory into lang/<new-code>/.
  2. Translate every value in application context without changing keys, placeholders, HTML structure, or pluralization syntax.
  3. Add the locale name to config/languages.php.
  4. Run the locale audit and application tests.
  5. Deploy the code containing the translation files.
  6. Add/enable the locale through the admin page or locales:configure.
  7. Clear caches. Rebuild resources/api/odb_param_schema.json with its generator when releasing documentation/schema changes; never edit the generated JSON manually.

When upgrading OpenDataBio, compare the new .env.example with the deployed environment, run database migrations, run php artisan locales:audit, and update every installed lang/<code>/ directory with any newly introduced translation keys before enabling that interface.

8 - Backup and restore

Create verified database backups and synchronize persistent OpenDataBio files without duplicating large storage on the application server.

An OpenDataBio backup has two parts: a logical database dump and the persistent files referenced by that database. The supplied tools deliberately do not compress or duplicate the potentially large media storage on the application server. They create small, verified database snapshots and optionally let an administrator’s workstation, NAS, or backup server pull the current files with incremental rsync.

Keeping only database dumps on the application server is a minimum safeguard, not disaster recovery. If that server or disk is lost, its original storage is lost too. Whenever possible, regularly pull or push a second copy to another machine.

What must be protected

Back up together:

  1. the MySQL/MariaDB database;
  2. storage/app/public/media;
  3. storage/app/public/datasets, including published dataset versions;
  4. the production environment and secret files in a separate secure store;
  5. especially APP_KEY, which is required to decrypt values stored by the application, such as personal Pl@ntNet keys.

Redis, caches, sessions, logs, vendor, node_modules, and regenerable downloads are not part of the durable backup.

Included tools

The repository provides:

scripts/backup/
├── backup-opendatabio.sh
├── backup.env.example
├── pull-opendatabio-backup.sh
├── pull.env.example
├── verify-opendatabio-backup.sh
└── systemd/

backup-opendatabio.sh runs on the application server. It creates a compressed database dump, verifies it, writes checksums and a manifest, marks the snapshot COMPLETE, and atomically updates LATEST. It never copies or deletes storage.

pull-opendatabio-backup.sh runs on another machine. It downloads the latest complete database snapshot and synchronizes media and datasets directly from their original directories.

Configure the application server

Create a root-owned configuration:

sudo install -d -m 700 /etc/opendatabio /var/backups/opendatabio
sudo cp scripts/backup/backup.env.example /etc/opendatabio/backup.env
sudo chmod 600 /etc/opendatabio/backup.env
sudo editor /etc/opendatabio/backup.env

Always set the real checkout and backup paths. The backup destination must be an absolute path other than /.

Docker production

Use:

ODB_DEPLOYMENT=docker
ODB_INSTALL_ROOT=/opt/opendatabio
ODB_ENV_FILE=/opt/opendatabio/.env.production
ODB_COMPOSE_FILE=/opt/opendatabio/docker-compose.prod.yml
ODB_COMPOSE_PROJECT=odb-prod
ODB_BACKUP_ROOT=/var/backups/opendatabio
ODB_BACKUP_GROUP=
ODB_STORAGE_PATH=/srv/opendatabio/storage

Docker production uses a named odbstorage volume by default. A remote rsync client cannot safely address files inside that private Docker volume. For a new production installation that will use pull backups, prepare a host directory:

sudo install -d -m 755 /srv/opendatabio/storage

Then set in .env.production before the first start:

ODB_STORAGE_SOURCE=/srv/opendatabio/storage

The Compose services mount this directory at the normal application storage location, and the container entrypoint applies the image’s configured www-data ownership. Do not hard-code a host UID/GID because image build settings may change it. Existing installations using the named volume must copy its contents to the bind-mounted directory during a planned maintenance window before changing ODB_STORAGE_SOURCE; do not merely change the variable or the files will appear missing.

Native Apache/nginx installation

Use:

ODB_DEPLOYMENT=native
ODB_INSTALL_ROOT=/var/www/opendatabio
ODB_BACKUP_ROOT=/var/backups/opendatabio
ODB_DB_NAME=opendatabio
ODB_DB_DEFAULTS_FILE=/root/.opendatabio-backup.cnf
ODB_BACKUP_GROUP=
ODB_STORAGE_PATH=/var/www/opendatabio/storage/app/public

Store database credentials outside the script:

[client]
host=localhost
port=3306
user=opendatabio_backup
password=REPLACE_WITH_A_PRIVATE_PASSWORD
sudo chmod 600 /root/.opendatabio-backup.cnf

The database account needs enough read access to dump all application tables and triggers. Credentials are not passed as host command-line arguments.

Run and verify a backup

Test manually before scheduling:

sudo /opt/opendatabio/scripts/backup/backup-opendatabio.sh \
  /etc/opendatabio/backup.env

The result is:

/var/backups/opendatabio/
├── LATEST
└── snapshots/20260813T021700Z/
    ├── database.sql.gz
    ├── SHA256SUMS
    ├── manifest.env
    └── COMPLETE

Verify it independently:

sudo scripts/backup/verify-opendatabio-backup.sh \
  /var/backups/opendatabio/snapshots/20260813T021700Z

Only directories with COMPLETE are eligible for automatic retention. The default retains daily database snapshots for seven days. Set ODB_RETENTION_DAYS as appropriate.

Schedule on the server

Copy and review the supplied examples:

sudo cp scripts/backup/systemd/opendatabio-backup.service.example \
  /etc/systemd/system/opendatabio-backup.service
sudo cp scripts/backup/systemd/opendatabio-backup.timer.example \
  /etc/systemd/system/opendatabio-backup.timer
sudo systemctl daemon-reload
sudo systemctl enable --now opendatabio-backup.timer
sudo systemctl list-timers opendatabio-backup.timer

Inspect executions with:

sudo systemctl status opendatabio-backup.service
sudo journalctl -u opendatabio-backup.service

Persistent=true runs a missed backup after the server returns.

cron alternative

Edit root’s crontab with sudo crontab -e:

17 2 * * * /opt/opendatabio/scripts/backup/backup-opendatabio.sh /etc/opendatabio/backup.env >> /var/log/opendatabio-backup.log 2>&1

List scheduled jobs with:

sudo crontab -l
sudo ls -la /etc/cron.d /etc/cron.daily
sudo systemctl list-timers --all

The script uses flock, so overlapping executions fail safely.

Pull to an administrator-controlled machine

Install openssh-client and rsync on the receiving workstation, NAS, or server. Configure an SSH user that can read only the backup and persistent storage paths. Prefer a dedicated SSH key and verify the server host key. On the application server, create the configured reader group and add that SSH user to it:

sudo groupadd --system opendatabio-backup
sudo usermod -aG opendatabio-backup backup-reader

Then set ODB_BACKUP_GROUP=opendatabio-backup in backup.env and run a new backup so the completed snapshot receives the group permissions.

backup-opendatabio.sh grants this group read access only to completed snapshots and LATEST. Separately grant the reader traverse/read access to the configured media and datasets paths using an appropriate group or filesystem ACL; do not make private configuration files world-readable.

Copy the client configuration:

sudo install -d -m 700 /etc/opendatabio /srv/backups/opendatabio
sudo cp scripts/backup/pull.env.example /etc/opendatabio/pull-backup.env
sudo chmod 600 /etc/opendatabio/pull-backup.env
sudo editor /etc/opendatabio/pull-backup.env

Important values:

ODB_REMOTE_HOST=backup-reader@example.org
ODB_REMOTE_BACKUP_ROOT=/var/backups/opendatabio
ODB_REMOTE_STORAGE_ROOT=/srv/opendatabio/storage
ODB_LOCAL_ROOT=/srv/backups/opendatabio
ODB_SSH_IDENTITY=/home/admin/.ssh/opendatabio-backup
ODB_STORAGE_POLICY=archive

Test:

scripts/backup/pull-opendatabio-backup.sh \
  /etc/opendatabio/pull-backup.env

The client reads LATEST from the server instead of guessing a folder from its local date. It reuses one SSH connection, validates the downloaded checksum and gzip stream, and then runs incremental rsync directly against the original storage. It does not use compression because images and many dataset artifacts are already compressed.

Storage policies:

  • archive (default) never deletes a local file that disappeared remotely;
  • mirror makes the current copy match the server, but moves deleted or replaced files into history/SNAPSHOT/ instead of discarding them.

Do not add plain --delete without a snapshot or backup directory. A mistaken deletion on the application server would otherwise propagate to the backup.

The supplied opendatabio-backup-pull.service.example and opendatabio-backup-pull.timer.example can be installed and edited on the receiving machine. Their Persistent=true setting is useful for a workstation that is not always on. Cron is acceptable when the receiving machine runs continuously.

Restore

Test restoration periodically on an isolated installation. At minimum:

  1. verify COMPLETE, SHA256SUMS, and gzip -t;
  2. use the OpenDataBio version recorded in manifest.env;
  3. place the target application in maintenance and stop workers;
  4. create a safety backup of the target;
  5. recreate the target database and import database.sql.gz;
  6. restore media and datasets to storage/app/public;
  7. restore the matching APP_KEY and required secrets;
  8. correct ownership and permissions;
  9. run php artisan migrate --force, php artisan optimize, and php artisan locales:audit;
  10. inspect logs and verify login, record counts, media, dataset versions, and a small queued job before reopening access.

For MariaDB-to-MySQL migration, recent mariadb-dump may prepend a sandbox directive unsupported by Oracle MySQL. Remove only that exact directive; never blindly remove the first line of an SQL dump.

Operational checks

Monitor at least:

  • age and size of LATEST;
  • exit status and logs of the server backup;
  • state/last-success on the pull destination;
  • free space on both machines;
  • periodic checksum verification;
  • a real test restoration at least quarterly.

Backups have only been proven when a restoration succeeds.