Fix Uncategorized Expenses in Finance with Gemini-Powered AI

Use Multimodal Inputs: Combine Merchant, Text, and Receipt Images

Gemini’s strength for Finance is its ability to interpret multiple input types together. For each expense line, send merchant name, transaction amount, currency, employee free-text description, and a receipt image (where available). This helps Gemini distinguish, for example, between a hotel’s restaurant and room charges, or between personal and business items on the same receipt.

Structure your API payload to keep financial metadata separate from raw text, so prompts can explicitly reference them. Where your expense tool already extracts some OCR data from receipts, pass both the original image and the OCR text so Gemini can correct or enrich it.

Example Gemini prompt template (conceptual):
You are an expense classification assistant for the Finance department.
You receive: merchant, amount, date, employee description, and a receipt image.

Tasks:
1. Propose the most likely expense category (GL account) from this list: [...].
2. Propose cost center and project, if clearly inferable.
3. Flag potential policy violations (e.g. weekend spend, first-class travel).
4. Return a JSON object with fields: category_code, cost_center, project,
   policy_flags[], confidence_score, reasoning.

Now classify the following expense:
MERCHANT: {{merchant}}
AMOUNT: {{amount}} {{currency}}
DATE: {{date}}
DESCRIPTION: {{description}}
RECEIPT_IMAGE: <binary or URL>

Standardise Outputs with JSON Schemas and Confidence Thresholds

For reliable downstream posting, enforce a strict JSON schema for Gemini responses. Define required fields (e.g. category_code, confidence_score), permissible values (e.g. list of GL accounts), and default behaviours when the model is unsure.

On the consuming side (your middleware or expense tool), implement confidence thresholds. For example, if confidence_score >= 0.9, auto-apply the classification; if 0.7–0.9, route to a controller’s review queue with Gemini’s reasoning pre-filled; if < 0.7, keep the line as “uncategorized” but attach the suggestion for faster manual handling.

Expected Gemini response structure:
{
  "category_code": "6130_TRAVEL_HOTEL",
  "cost_center": "CC_102_MKT_DE",
  "project": "PRJ_4567_CAMPAIGN_Q4",
  "policy_flags": ["WEEKEND", "NO_APPROVAL_FOUND"],
  "confidence_score": 0.92,
  "reasoning": "Merchant is a hotel chain, date matches conference…"
}

Integrate Gemini into Existing Expense and ERP Workflows

Instead of building a separate tool, embed Gemini classification into the workflows your employees and controllers already use. Typical patterns include a middleware service between the expense app and ERP, or an API extension inside the expense management system.

Implementation sequence could look like this: (1) Employee submits an expense as usual; (2) Webhook triggers a Gemini API call with all inputs; (3) Middleware writes back suggested category, cost center, and flags into custom fields; (4) Approver or controller sees the pre-filled suggestions and can accept or adjust; (5) Final data posts to the ERP. This minimises change management and speeds up adoption.

Continuously Retrain Prompts with Feedback from Finance

Set up a simple loop where Finance feedback directly improves Gemini’s performance. Whenever a controller overrides a category or cost center, log both Gemini’s suggestion and the final choice. Regularly sample these overrides and update your prompt instructions (e.g. “for <merchant> in Germany, prefer cost center X unless description mentions Y”).

Even without model fine-tuning, iterative prompt refinement can move accuracy from “roughly helpful” to “good enough to automate most cases”. Schedule monthly prompt reviews with Finance and your technical team to adjust rules, add new merchant patterns, and refine policy checks.

Prompt refinement snippet:
Previous errors: Taxi rides for client visits were coded as "office travel".
Update instruction:
- If description contains words like "client", "customer", "meeting" and
  merchant type is taxi/ride-sharing, classify under
  6145_CLIENT_VISIT_TRAVEL instead of 6140_INTERNAL_TRAVEL.

Use Gemini for Policy Violation Detection and Anomaly Flags

Beyond basic categorisation, use Gemini to flag policy violations and anomalies in real time. For example, detect first-class or business-class fares from the ticket image, weekend or holiday expenses, or duplicate receipts submitted across different reports.

Design prompts that explicitly ask Gemini to reason about context: time of day, location vs. employee office, unusual amounts relative to typical spend for that merchant. Route flagged items into a specific queue with clear labels so controllers can quickly decide whether to approve, reject, or request more information.

Example policy check snippet:
In addition to classification, check for:
- Travel class (economy vs business/first) from the ticket or receipt.
- Weekend or public holiday dates.
- Multiple similar receipts in a short time window.
Return a "policy_flags" array with reasons, e.g. [
  "BUSINESS_CLASS_FLIGHT",
  "WEEKEND_EXPENSE"
]

Define KPIs and Dashboards to Track Impact

To prove value, track a small set of expense automation KPIs before and after Gemini deployment. Common metrics include: percentage of uncategorized lines, average time from submission to posting, manual touch rate per expense, and reclassification rate after monthly close.

Visualise these KPIs in your existing BI or ERP reporting, ideally by department and country. This helps you see where the model performs well, where additional training is needed, and where process issues (not AI) are causing delays. Use these insights to prioritise future improvements and justify further investment in AI-enabled Finance workflows.

Implemented pragmatically, Finance teams typically see 30–60% fewer uncategorized entries within the first 2–3 months, a 20–40% reduction in manual expense coding effort, and materially faster visibility into spend by cost center and project—enough to make a noticeable difference in month-end closing and budget steering.