Project Overview

Crisis-Monitor is a lightweight incident tracking system built with Flask and SQLAlchemy. It stores emergency tickets in a relational database and provides a simple HTML interface with basic authentication. A small REST API is also included for integrations.

## Features - User accounts with a customizable role system powered by Flask-Principal. The built-in roles are admin, editor, analyst and viewer. - Admin tools to manage users including editing roles and blocking accounts. - Each account stores dashboard settings in a JSON dashboard_config field. - CRUD interface to manage crisis tickets. - Predefined reference tables: regions, emergency types, statuses and others. These are populated on first run. - JSON API endpoints to list tickets or retrieve a single ticket. - Interactive map view to visualize incidents geographically from the ticket list. - Display As options let you view tickets as a table, cards or on a map. - An advanced filter builder encodes queries as JSON for reusable searches. - New ticket forms can automatically populate latitude, longitude and the address using your browser's geolocation API (if permission is granted). - Ticket templates to pre-fill new ticket forms with commonly used details. - Smooth scrolling navigation for in-page links with automatic return to your last scroll position after viewing a ticket or running a search. - Attachments can now be previewed inline in a modal without leaving the page. - A link buffer page lets you store URLs with tags before turning them into tickets. - A keyboard-driven command palette (<kbd>Ctrl</kbd>+<kbd>K</kbd>) quickly searches tickets or users and jumps to buffer or reference sections. - EDXL SituationReport and CAP Alert exports for cross-border interoperability. - Registered GDELT, ReliefWeb, GDACS and QazPipe collection paths with an authenticated feed-check contract that separates probe failures from quiet event streams. - Descriptive source review profiles and a read-only team activation board expose evidence-review composition, staffing and workload without inventing ratings or acknowledgement state. - Online-first field reporting at /field/report creates a normal incident with structured observation metadata and reuses the existing attachment policy; offline drafts and background sync are intentionally separate work. - Narrative velocity is exposed as an observed two-window signal, while the evidence bundle includes a privacy-safe after-action learning summary. - Linked DISARM signals are shown on incident details as observed claims with trust-tier, source and corroboration context; internal review notes stay out of the machine/public projection. - The dashboard includes a read-only operator readiness scorecard with explicit sample-size and not measured states for overdue work, ownership, SitRep, watch-zone and playbook coverage. - The AV DS 4 operator shell has dedicated wide compositions for very large screens, verified at 2560px and 3440px alongside the 1440px, 1024px and 390px route sweeps. See the 2026-08-04 release audit.

Data Model

The main CrisisTicket model contains fields such as title, description, incident time, optional address and location coordinates, and links to reference tables like Region, EmergencyType, RiskLevel and more. Tickets can be related to one another through an association table.

Hidden notes

Users may attach short notes to a ticket. Each note can be marked as public or internal. Viewers only see public notes while editors and admins see all of them. Opening a note records a NoteView entry for auditing. Use the "Добавить" button on the ticket page or visit /tickets/<id>/notes to manage notes. Administrators can search across all notes via /notes/search. An admin audit log at /admin/audit_log lists all ticket actions with old and new values for each change.

Setup

Ensure Python 3.11 and Node.js 18 (or newer) are installed, then create and activate a virtual environment. bash python3 -m venv .venv source .venv/bin/activate On Windows run python -m venv .venv followed by .venv\Scripts\activate.
Install the Python dependencies. You can run the helper script or execute the equivalent pip commands manually: bash ./setup-dev.sh # or pip install --upgrade pip pip install -r requirements.txt -r requirements-dev.txt Install the python-magic library if it is not already present. Windows users must install the underlying libmagic package (for example via pip install python-magic-bin). Debian/Ubuntu users can run sudo apt-get install libmagic1. On macOS with Apple Silicon use Homebrew to install the libraries: brew install libmagic cairo pango gdk-pixbuf libffi go. Re-run the installation whenever the requirement files are updated. The helper only installs Python packages; Node dependencies are handled in the next step.
Install the JavaScript dependencies and build the Vite bundle so static/dist exists before starting Flask: bash npm ci npm run build Run npm run dev in a separate terminal if you want hot reloading during development.
Copy .env.example to .env and configure at least SECRET_KEY and APP_ENV. When deploying to production also set DEFAULT_LOCALE, DEFAULT_TIMEZONE, DEFAULT_CURRENCY and BASE_CURRENCY.
Configure the database connection via environment variables or a .env file. The APP_ENV variable should be set to either development or production. The appropriate configuration class will be loaded automatically. To use MySQL you might create a .env file like:

dotenv DATABASE_URL=mysql+pymysql://user:password@localhost/crisisdb SECRET_KEY=replace_this_key APP_ENV=development Install either the PyMySQL or mysqlclient package for MySQL support.

Optional variables enable email alerts and control the monitoring behaviour:

dotenv MAIL_SERVER=smtp.example.com MAIL_PORT=587 MAIL_USERNAME=your_user MAIL_PASSWORD=your_password MAIL_USE_TLS=true ALERT_EMAIL_TO=alerts@example.com ALERT_EMAIL_FROM=monitor@example.com REMINDER_RECIPIENTS=user1@example.com,user2@example.com TELEGRAM_TOKEN=bot_token UPLOAD_FOLDER=uploads MAX_UPLOAD_SIZE=2097152 ALLOWED_MIME_TYPES=image/png,image/jpeg,image/gif,application/pdf,text/plain HEALTH_CHECK_INTERVAL=60 EXTERNAL_API_URL=https://example.com/data WEATHER_API_KEY=your_weather_api_key The server validates attachments using python-magic (requires libmagic) and rejects files exceeding `MAX_UPLOAD_SIZE` or not listed in `ALLOWED_MIME_TYPES`. ALERTS_ENABLED=true SENTRY_DSN=https://examplePublicKey@o0.ingest.sentry.io/0 BACKUP_DIR=backups LOG_DIR=logs TEMP_DIR=temp # Paths are relative to the project root unless absolute BACKUP_INTERVAL_HOURS=24 CLEANUP_INTERVAL_HOURS=24 CLEANUP_DAYS_THRESHOLD=30 # default: FILE_CLEANUP_DAYS DEFAULT_LOCALE=ru SUPPORTED_LOCALES=ru,kk DEFAULT_TIMEZONE=UTC DEFAULT_CURRENCY=USD BASE_CURRENCY=USD

Setting WEATHER_API_KEY allows the context service to fetch current weather data for incident locations.

When running with config.ProductionConfig the variables MAIL_USERNAME, MAIL_PASSWORD, ALERT_EMAIL_TO and ALERT_EMAIL_FROM fall back to development-friendly defaults. Set them explicitly in production to enable real email delivery.

The MAIL_SERVER, MAIL_PORT, MAIL_USERNAME and MAIL_PASSWORD values configure the SMTP server used for alerts. ALERT_EMAIL_TO defines who receives them while ALERT_EMAIL_FROM sets the sender address. List REMINDER_RECIPIENTS comma-separated to send daily reports and provide a TELEGRAM_TOKEN to enable Telegram notifications.

Backups are created every BACKUP_INTERVAL_HOURS hours in BACKUP_DIR when the built-in scheduler runs in development mode. You can also create one manually by running python scripts/backup.py and schedule it via cron (0 2 * * * python scripts/backup.py) for production deployments. Old log and temporary files older than CLEANUP_DAYS_THRESHOLD days (default: FILE_CLEANUP_DAYS) are removed from LOG_DIR and TEMP_DIR every CLEANUP_INTERVAL_HOURS hours. DEFAULT_LOCALE, SUPPORTED_LOCALES, DEFAULT_TIMEZONE, DEFAULT_CURRENCY and BASE_CURRENCY allow you to tailor the application's language, timezone and currency settings.

Cross-border interoperability

The API exposes endpoints that generate EDXL SituationReport and CAP Alert XML documents. These formats allow neighbouring agencies to consume incident data directly from Crisis‑Monitor.

Running

Start the application with:

python run.py

The first launch seeds the reference data and generates about 50 example crisis tickets so you have some data to explore immediately. These demo tickets include random latitude and longitude coordinates for the map view. Do not use flask run here – custom flags like --seed and --reset are only recognized when running python run.py directly. Run flask --app app:create_app db upgrade before starting the application to apply pending migrations. flask run will still start the server if you prefer that workflow. Run this upgrade command again after pulling updates from the repository so the database schema stays in sync with new migrations. To populate the database with new demo tickets at any time run:

python run.py --seed --no-run

Omit --no-run to seed data and start the development server in one step. By default the development server listens on http://0.0.0.0:8000/. Use the --host and --port flags to change this if needed.

Selecting incidents

Each row in the table has a checkbox so you can choose more than one ticket at a time. Once at least one item is selected the Compare and Export selected buttons become active. Use them to open a comparison page or download a CSV file with the chosen records. Bulk editing or deletion of tickets is not available.

The comparison view lists the chosen tickets in a simple table showing their ID, title and status so you can quickly check differences.

Ticket list views and filtering

Use the Display As buttons on the ticket list to switch between table, card or map layouts. The selected mode is stored in the view URL parameter.

Complex searches can be built with the Add filter button. Each rule has a field, operator and value. Submitting the form adds a query parameter containing JSON:

{
  "group": "AND",
  "rules": [
    {"field": "status_id", "op": "eq", "value": 2},
    {"field": "title", "op": "like", "value": "flood"}
  ]
}

This JSON can be saved as a favorite filter and reused later.

Link buffer

Visit /buffer to store interesting URLs before creating tickets. Each entry records the time it was added and shows a small page preview so you can evaluate the link without leaving the list. Click Create ticket in the table to open the ticket form with the URL and description pre-filled.

Saved filter sets

You can save commonly used search queries from the ticket list. After applying filters click Save search to store the query as a reusable filter set. All saved sets appear on the Favorites page where you can quickly reopen them or rename the stored query. Each set is saved per user so team members can keep their own collections of frequently used filters.

Admin tools

The administration panel exposes JSON endpoints under /admin for managing user accounts. Admins can list all users, assign one of the built‑in roles and block or unblock accounts. Blocked users are unable to log in until re‑enabled.

Escalation rules

Administrators may configure automatic escalation rules under /admin/escalation_rules. Each rule stores filtering conditions and a list of actions to execute when all conditions match an open ticket. Supported condition keys are:

status_name – the ticket status must match this name.
risk_name – the risk level must match this name.
hours_since_update – minimum hours since the ticket was last updated.

Two actions are supported:

"telegram" – send a message to the alert channel.
"notify_team" – message each member of the ticket's assigned team.

A simple rule might look like:

{
  "conditions": {
    "status_name": "In progress",
    "risk_name": "High",
    "hours_since_update": 48
  },
  "actions": ["telegram"]
}

Rules can be enabled or disabled through the admin interface. The scheduler runs the evaluation job every five minutes and records matches in the EscalationLog table.

When saving a rule the admin UI checks for other active rules with the same conditions. If one is found but the actions differ, a warning is displayed to help avoid conflicting automation.

Monitoring

The monitoring panel is available from /admin/monitoring once logged in. It shows live notifications for new tickets and displays recent activity. Email alerts are sent when ALERTS_ENABLED is set and mail credentials are configured. The polling frequency is controlled by the HEALTH_CHECK_INTERVAL variable (in seconds), which sets how often the MonitoringScheduler runs.

System status

Admins can open /admin/status to view results of built-in health checks. The page displays database, queue, email and API status along with the current queue length and average message processing rate. These values are updated every few seconds in the browser to help diagnose issues quickly.

If the models change you can generate a migration using Flask-Migrate and then apply it:

flask --app app:create_app db migrate -m "Describe change"
flask --app app:create_app db upgrade

For a complete reset of the SQLite database pass the --reset flag:

python run.py --reset

Creating tickets via the API

Authenticated services can create new incidents by sending a POST request to /api/v1/tickets with JSON payload. The field names match those of the HTML form; only title is required. A logged in session is needed as the endpoint is protected with login_required.

Example request using curl:

curl -X POST http://localhost:8000/api/v1/tickets \
     -H "Content-Type: application/json" \
     -b cookies.txt \
     -d '{
           "title": "Warehouse fire",
           "description": "Smoke visible from roof",
           "region": 1,
           "type": 1,
           "channel": 3
         }'

On success the API returns HTTP 201 with the created ticket in JSON format:

{
  "id": 42,
  "title": "Warehouse fire",
  "description": "Smoke visible from roof",
  "time_created": "2024-01-01T12:00:00",
  "time_incident": null,
  "region": "Some Region",
  "latitude": null,
  "longitude": null,
  "address": null,
  "type": "Fire",
  "scale": null,
  "risk": null,
  "source": null,
  "channel": "Email",
  "status": null,
  "departments": [],
  "response_type": null,
  "deadline": null,
  "fatalities": 0,
  "injured": 0,
  "damage": 0,
  "attachments": [],
 "source_urls": null,
  "related_tickets": []
}

If required fields are missing or numeric identifiers are invalid, the response will be HTTP 400:

{"error": "Некорректные данные"}

The channel field expects one of the predefined IDs:

1 – 112 2 – Телефон 3 – Email 4 – Соцсети/мессенджер

Listing valid responses

Send a GET request to /api/v1/incident-types/<id>/valid-responses to obtain the allowed response types for a particular incident category. The endpoint returns a JSON array with id and name fields.

Sample forwarding scripts

Below are minimal examples of forwarding incidents from other services.

# telegram_forward.py
import requests
from telegram.ext import Updater, MessageHandler, Filters

API_URL = 'http://localhost:8000/api/v1/tickets'
session = requests.Session()
session.post('http://localhost:8000/login',
             data={'username': 'bot', 'password': 'secret'})

def forward(update, _):
    payload = {
        'title': update.message.text[:100],
        'description': update.message.text,
        'channel': 4
    }
    session.post(API_URL, json=payload)

updater = Updater('TOKEN')
updater.dispatcher.add_handler(MessageHandler(Filters.text, forward))
updater.start_polling()

# email_poll.py
import imaplib, email, requests

API_URL = 'http://localhost:8000/api/v1/tickets'
sess = requests.Session()
sess.post('http://localhost:8000/login',
          data={'username': 'bot', 'password': 'secret'})

mail = imaplib.IMAP4_SSL('imap.example.com')
mail.login('user@example.com', 'password')
mail.select('INBOX')

status, data = mail.search(None, 'UNSEEN')
for num in data[0].split():
    _, msg_data = mail.fetch(num, '(RFC822)')
    msg = email.message_from_bytes(msg_data[0][1])
    body = msg.get_payload(decode=True).decode('utf-8')
    sess.post(API_URL, json={
        'title': msg['Subject'],
        'description': body,
        'channel': 3,
    })

Test utilities

Two helper modules are provided only for the automated tests. utils/telegram.py can send a simple request to the Telegram Bot API while tests/test_notification_capture.py keeps all outgoing messages in memory so the tests can inspect them. The application itself does not import these modules.

Testing

Before running the tests make sure all dependencies are installed. If the requirements.txt file has changed (for example after adding Flask-Principal for role management) install the packages again with pip install -r requirements.txt and pip install -r requirements-dev.txt. The development requirements include pytest-cov so you can measure coverage through the Makefile targets:

make test

This command bootstraps the virtual environment when missing and executes the pytest suite. To produce CI-equivalent coverage reports without browser e2e tests use:

make coverage

The local enforced floor is 60% for app, services and utils. The measured baseline on 2026-07-28 was 767 passed tests and 62.11% total coverage. For the occasional full local measurement including e2e tests, use:

make coverage-full

To trigger the same checks inside the Docker Compose stack instead run:

make compose-test

Deployment

When running in production the application should be placed behind an HTTPS reverse proxy (for example Nginx). The proxy must set the X-Forwarded-Proto header so Flask can detect secure requests. Session cookies are configured with the SESSION_COOKIE_SECURE option, so all external traffic must use HTTPS. In the development and testing configurations this option is disabled, allowing the app to run over plain HTTP when developing locally. Set APP_ENV=production and adjust DEFAULT_LOCALE, DEFAULT_TIMEZONE, DEFAULT_CURRENCY and BASE_CURRENCY in the .env file before launching the application in production.

Production Deployment

Run the application under a WSGI server like Gunicorn when deploying to production. The wsgi.py module exposes the Flask app as app. Start Gunicorn with:

gunicorn --bind 0.0.0.0:8000 wsgi:app

run.py is intended for development and debugging. It enables helper flags such as --seed and --reset. Use wsgi.py as the entry point when serving the app with Gunicorn or another WSGI server.

Recommended cron jobs

Add these cron entries when deploying with APP_ENV=production so periodic maintenance tasks run automatically:

0 2 * * * python scripts/backup.py
15 2 * * * flask --app app:create_app shell -c 'from utils.cleanup_utils import cleanup_old_files; cleanup_old_files()'
30 2 * * * python run.py --run-reminders

Threat monitoring

Crisis-Monitor includes the ThreatMonitor helper (services/threat_monitor.py), which recalculates the threat_level for open tickets by counting notes and matching weighted keywords in the description and note text. When a ticket's score jumps past the configured threshold, the service publishes a threat_spike event to the server-sent events stream so dashboards can react in real time.【F:services/threat_monitor.py†L13-L68】

The ARQ cron job cron_threat_scan in services/worker.py runs threat_monitor.update_threat_levels() every minute. This keeps the stored threat levels in sync without a separate worker process. You can trigger the same scan manually via the CLI:

python cli.py cron-threat-scan

Background automation

The same scheduler powers several maintenance tasks while the application runs in development mode:

Backups: BackupService.create_backup() archives the database and attachment directory on the interval specified by BACKUP_INTERVAL_HOURS.【F:app/init.py†L381-L438】【F:services/backup_service.py†L27-L95】
File and undo cleanup: cleanup_old_files() purges aged log/temp files and cleanup_expired_undo_actions() drops expired undo records.【F:app/init.py†L385-L392】【F:utils/cleanup_utils.py†L36-L45】【F:utils/undo_utils.py†L20-L60】
Ticket metrics: flush_ticket_metrics() reports the number of newly created tickets to METRICS_URL.【F:app/init.py†L393-L395】【F:utils/metrics.py†L9-L43】
External API probe: check_external_api() logs the number of new items returned by EXTERNAL_API_URL.【F:app/init.py†L397-L400】【F:utils/external_api.py†L12-L43】
SLA monitoring: check_overdue_tickets() publishes sla_breached events for missed deadlines and escalate_overdue_tickets() raises alerts for inactive tickets approaching their deadline.【F:app/init.py†L401-L468】【F:services/ticket_service.py†L1430-L1495】
Escalation rules: Active rules are reevaluated by services.escalation_service.check_and_escalate() every five minutes.【F:app/init.py†L427-L472】【F:services/escalation_service.py†L13-L114】
Deadline reminders: send_deadline_reminders() emails and messages upcoming deadlines, recording each notification to prevent duplicates.【F:app/init.py†L419-L482】【F:utils/reminders.py†L93-L147】
Nightly archive: archive_old_tickets() moves aged tickets into the archive at midnight UTC by default.【F:app/init.py†L423-L489】【F:utils/archive_utils.py†L9-L34】

In addition, MonitoringScheduler runs in a background thread to perform periodic database health checks, publish health_check_failed events, and send email alerts when failures occur.【F:app/init.py†L493-L495】【F:utils/monitoring.py†L16-L102】

Extending the command palette

Shortcut entries shown in the palette are defined in static/js/commands.js. Each command is an object with a unique id, a human readable title, a type (navigation or action) and either a url to navigate to or a handler function. Example:

{ id: 'nav_dashboard', title: 'Go: Dashboard', type: 'navigation', url: '/dashboard' }

Add context-specific actions by pushing additional objects to the exported commands array from a template. The snippet below registers two extra commands on a ticket detail page:

{% raw %}<script type="module">
import { commands } from '/static/js/commands.js';
commands.push(
  { id: 'edit_ticket', title: 'Edit current ticket', type: 'navigation', url: '/tickets/{{ ticket.id }}/edit' },
  { id: 'add_note', title: 'Add note', type: 'navigation', url: '/tickets/{{ ticket.id }}/notes/new' }
);
</script>{% endraw %}

Once the page loads the command palette will include these new entries.