Error Tracking

oCore automatically detects Python tracebacks and common error patterns in your Odoo instance logs, groups them by fingerprint, tracks occurrences over time, and optionally creates GitHub Issues when errors cross a threshold. This gives you a centralized view of production errors without installing any Odoo module or third-party APM tool.

How It Works

Error tracking follows a pipeline that runs automatically whenever oCore collects logs from a running instance:

1. Log collection -- oCore reads container logs from your Odoo instances via SSH
2. Error detection -- Log lines are scanned for Python tracebacks (Traceback (most recent call last):) and non-traceback patterns (database errors, access errors, module warnings)
3. Fingerprinting -- Each detected error is hashed using SHA-256 over the exception type, custom addon stack frames (excluding line numbers), and module name. This produces a stable identifier even when the same error occurs on different lines
4. Deduplication -- The fingerprint is matched against existing records. If a match is found, the occurrence count is incremented. If not, a new error record is created
5. Occurrence tracking -- Each occurrence is recorded in hourly histogram buckets, enabling sparkline charts and timeline views
6. GitHub Issue creation -- When configured and the occurrence threshold is met, oCore creates a GitHub Issue with a Copilot-optimized body containing the traceback, affected files, and context lines

Severity Classification

Errors are classified automatically based on the detection type:

Error Lifecycle

Each error fingerprint moves through a status lifecycle:

new -> active -> resolved -> regressed -> active -> ...

Enabling Error Detection

Error detection is controlled per instance. By default, the setting is inherited from the environment type:

• Production environments -- error detection is ON by default
• Development and staging environments -- error detection is OFF by default

Toggling Error Detection

You can override the default for any instance:

# Enable error detection
curl -X PUT https://ocore.example.com/api/instances/{instanceId} \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"errorDetectionEnabled": true}'

# Disable error detection
curl -X PUT https://ocore.example.com/api/instances/{instanceId} \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"errorDetectionEnabled": false}'

You can also toggle this from the instance settings page in the dashboard.

Error Dashboard

The error dashboard shows all detected errors across your organization, with filtering and sorting options.

Summary Stats

The top of the dashboard shows four key metrics:

Filtering

Filter the error list using any combination of:

Sorting

Sort errors by:

• Last seen (default) -- most recently occurring errors first
• Total count -- most frequent errors first
• Severity -- critical first, then high, then medium

Sparkline Charts

Each error in the list includes a 7-day sparkline chart showing hourly occurrence counts. This gives you a quick visual of whether an error is trending up, stable, or fading.

Error Detail

Click any error to open the detail view, which includes:

Traceback

The full Python traceback as captured from the log output. For chained exceptions, the entire chain is preserved.

Stack Frames

Parsed stack frames showing:

• File path -- The Python file where the error occurred
• Line number -- The line in the source file
• Function name -- The function or method containing the error

oCore identifies the Odoo module from the file path by checking known addon directories (/mnt/extra-addons/, /mnt/custom-addons/, etc.).

Context Lines

Up to 10 log lines before the traceback and 3 lines after are captured to provide surrounding context for debugging.

Occurrence Timeline

An hourly histogram of occurrences over the selected period (default 7 days, up to 90 days). Use this to correlate error spikes with deployments or other events.

# Fetch timeline data for the last 30 days
curl "https://ocore.example.com/api/errors/{errorId}/timeline?days=30" \
  -H "Authorization: Bearer $TOKEN"

Affected Instances

A list of all instances where this error has been detected, with instance names and slugs for easy identification. This helps you determine if an error is isolated to one instance or widespread.

Metadata

Additional context displayed on the detail view:

• Odoo version at the time of detection
• Commit SHA if available from the deployment
• First seen / last seen timestamps
• Resolved at timestamp (if the error was previously resolved)
• GitHub Issue link (if an issue was created)

GitHub Issues

oCore can automatically create GitHub Issues when errors are detected, using the repository connected to your project via Git Integration.

How Issue Creation Works

Issue Body Format

The issue body is formatted for readability by both humans and GitHub Copilot:

• Exception type and module as the header
• Severity, first/last seen, occurrence count, and affected instances as metadata
• Full traceback in a Python code block
• File paths with line numbers and function names
• Surrounding context lines in a collapsible <details> section
• A note that sensitive data has been redacted

Automatic Labels

oCore ensures the following labels exist in your repository and applies them:

Auto-Close on Resolution

When an error has no occurrences for 3 days, oCore:

1. Transitions the fingerprint status to resolved
2. Closes the linked GitHub Issue
3. Posts a comment: "No occurrences for 3 days. Auto-resolved by oCore."

Auto-Reopen on Regression

If a resolved error reappears:

1. The fingerprint status transitions to regressed
2. The linked GitHub Issue is reopened
3. The regression label is added
4. A comment is posted with the new occurrence timestamp and affected instance

Sensitive Data Redaction

Before writing tracebacks to GitHub Issues, oCore scrubs sensitive data including:

• Connection strings (://user:password@host)
• DSN password values
• AWS access keys (AKIA...)
• GCP API keys (AIza...)
• Environment variables containing secret, token, key, password, or credential

Authentication

oCore authenticates with GitHub using either:

• GitHub App -- Preferred. Uses the organization's GitHub App installation (see Git Integration)
• Personal Access Token -- Falls back to the encrypted PAT stored on the Git repository configuration

Muting Errors

Muting an error hides it from the default dashboard view and prevents GitHub Issue creation for that fingerprint. The error continues to be tracked (occurrences are still recorded), but it will not appear unless you enable the Show muted filter.

When to Mute

• Known harmless errors you cannot fix (e.g., third-party module bugs)
• Expected errors during migrations or maintenance windows
• Noisy low-severity warnings that clutter the dashboard

Mute / Unmute

# Mute an error
curl -X POST https://ocore.example.com/api/errors/{errorId}/mute \
  -H "Authorization: Bearer $TOKEN"

# Unmute an error
curl -X POST https://ocore.example.com/api/errors/{errorId}/unmute \
  -H "Authorization: Bearer $TOKEN"

You can also mute and unmute from the error detail view in the dashboard.

Custom Ignore Patterns

Each instance can define custom ignore patterns to skip known harmless log messages before they enter the error tracking pipeline. Patterns are matched using simple substring matching (strings.Contains).

Setting Ignore Patterns

curl -X PUT https://ocore.example.com/api/instances/{instanceId} \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "errorIgnorePatterns": [
      "l10n_mx.tax.rate: Tax rate mismatch",
      "mail.thread: Failed to send notification"
    ]
  }'

Default Ignore Patterns

oCore ships with built-in ignore patterns that are always active:

• odoo.modules.loading: Modules loaded
• odoo.service.server: HTTP service
• DeprecationWarning:
• PendingDeprecationWarning:
• ResourceWarning: unclosed
• UserWarning: Fields
• odoo.tools.translate: unable to find translation

Custom patterns are checked in addition to the defaults. You do not need to repeat the defaults in your custom list.

Merging Errors

If the fingerprinting algorithm creates separate records for what is logically the same error, you can merge them. Merging combines occurrence counts into a primary error and hides the secondary errors.

curl -X POST https://ocore.example.com/api/errors/merge \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "primaryId": "PRIMARY_ERROR_UUID",
    "secondaryIds": ["SECONDARY_1_UUID", "SECONDARY_2_UUID"]
  }'

Up to 50 errors can be merged in a single request. Merged errors are hidden from the list view.

Error Fingerprinting

oCore uses a deterministic fingerprinting algorithm to group related errors into a single record, even when they occur at different line numbers or across different instances.

How Fingerprinting Works

Each detected error is hashed using SHA-256 over three components:

1. Exception type -- The Python exception class name (e.g., ValueError, AccessError)
2. Custom addon stack frames -- File paths and function names from your custom addon directories, excluding line numbers so that minor code changes do not create duplicate fingerprints
3. Module name -- The Odoo module where the error originated, extracted from the file path

This produces a stable hash that groups occurrences of the same root cause together.

Enrichment Fields

When available, fingerprints are enriched with additional context:

Affected Instances

Each fingerprint tracks up to 100 affected instance IDs. When the same error appears on a new instance, it is added to the affected instances list. This lets you quickly determine whether an error is isolated to one instance or widespread across your organization.

Occurrence Histogram

Occurrences are stored in hourly histogram buckets per fingerprint per instance. This enables:

• Sparkline charts on the error list showing 7-day occurrence trends
• Timeline views on error detail pages with up to 90 days of hourly data
• Spike detection to correlate error bursts with deployments or other events

Batch Histogram Loading

The error list uses batch histogram loading to fetch sparkline data for multiple fingerprints in a single query, reducing the number of database round trips.

Cooldown

To prevent re-processing the same error in rapid succession, oCore applies a 5-minute cooldown per fingerprint hash. If the same error is detected within 5 minutes of the last occurrence, the fingerprint is updated but the full detection pipeline is not re-run.

Deduplication Concurrency

When multiple background workers detect the same new error simultaneously, oCore handles the race condition gracefully. If a unique constraint violation occurs during fingerprint creation, the operation is retried as an update to the existing record.

Issue Grouping

Error fingerprints can be grouped together when the fingerprinting algorithm creates separate records for what is logically the same root cause.

Automatic Grouping

oCore automatically groups errors by:

• Status lifecycle -- New errors become active on second occurrence, auto-resolve after 3 days of inactivity, and transition to regressed if they reappear
• Severity classification -- Errors are classified as critical (full tracebacks), high (database/access errors), or medium (module warnings) based on detection type
• Module extraction -- The Odoo module is extracted from the file path to group errors by their source module

Manual Merging

When automatic grouping is not sufficient, you can manually merge fingerprints:

curl -X POST https://ocore.example.com/api/errors/merge \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "primaryId": "PRIMARY_ERROR_UUID",
    "secondaryIds": ["SECONDARY_1_UUID", "SECONDARY_2_UUID"]
  }'

Merging:

• Combines total occurrence counts into the primary fingerprint
• Sets merged_into_id on secondary fingerprints, hiding them from the list
• Supports up to 50 secondary fingerprints per merge request
• Runs in a database transaction for consistency

Aggregate Statistics

The error dashboard provides four aggregate stats computed across all your projects:

Soft Delete

Fingerprints use soft deletion (via deleted_at timestamp). Soft-deleted fingerprints are excluded from all queries, including upsert lookups, ensuring that deleted errors do not resurface.

Required Permissions

Troubleshooting

No errors detected for a running instance

• Verify that error_detection_enabled is set to true (or null for environment default) on the instance
• Check that the instance is in a production environment (dev/staging instances have detection off by default)
• Confirm the instance is running and producing logs
• Check that your error is not matched by a default or custom ignore pattern

GitHub Issues not being created

• Verify that the project has a connected GitHub repository (see Git Integration)
• Check that a GitHub App installation or Personal Access Token is configured
• Confirm the daily issue creation cap has not been reached
• Ensure the error fingerprint is not muted

Same error appearing as multiple entries

• The fingerprinting algorithm uses exception type, custom addon stack frames, and module name. If these differ (e.g., different call paths to the same root cause), separate fingerprints are created
• Use the Merge feature to combine duplicates into a single entry

Errors reappearing after resolution

• This is expected behavior. If a resolved error occurs again, it transitions to regressed status
• If a GitHub Issue exists, it is reopened with a regression label
• Investigate whether a recent deployment reintroduced the bug