# Frontend Guide — Smart Lead Import

A step-by-step guide for building the lead-import UI against the CRM API. The
import is a guided, multi-step wizard: **upload → map columns → review & clean →
distribute → import**. This document lists every request, the exact fields to
send, and what to render at each step.

> Audience: frontend developers. You do not need to read the backend code — the
> shapes here match the API exactly.

---

## Prerequisites

- **Auth**: every request needs `Authorization: Bearer <token>` (Sanctum). Get
  the token from `POST /api/login`.
- **Permission**: the logged-in user needs `lead_management.raw_lead.create` (or
  `.override`). Without it these endpoints return **403 Forbidden** — hide the
  import feature or show a "not allowed" state.
- **File limits**: `xlsx`, `xls`, or `csv`, max **20 MB**. Enforce this in the
  file picker so the user gets instant feedback instead of a 422.
- **Response envelope**: every endpoint returns
  ```json
  { "success": true, "message": "...", "data": { } }
  ```
  On validation errors you get HTTP 422 with
  ```json
  { "success": false, "message": "...", "errors": { "field": ["..."] } }
  ```

### A note on header keys (important)

The reader **slugs** the spreadsheet's header row: lowercased and snake_cased.
So a column titled `Full Name` becomes the key `full_name`, `Phone No.` becomes
`phone_no`. Always use the keys returned by `preview`/`analyze` (`data.headers`)
when you build `column_mapping` — never the raw display text the user typed in
Excel.

---

## The wizard at a glance

| Step | Endpoint | Purpose |
|------|----------|---------|
| 0 (optional) | `POST /api/lead/import/template` | Let the user download a correctly-formatted sample file |
| 1 | `POST /api/lead/import/preview` | Read headers + sample rows + total row count |
| 2 | `POST /api/lead/import/analyze` | After mapping, profile each field: formats, counts, anomalies |
| 3 | `POST /api/lead/import/with_mapping` | Run the import with rules + distribution |

---

## Step 0 — Download template (optional)

`POST /api/lead/import/template`

Returns an `.xlsx` file download (binary). Wire it to a "Download sample" button.
Send it as a normal authenticated request and treat the response as a blob.

```js
const res = await fetch('/api/lead/import/template', {
  method: 'POST',
  headers: { Authorization: `Bearer ${token}` },
});
const blob = await res.blob();
// trigger a browser download of blob as "lead_import_template.xlsx"
```

---

## Step 1 — Upload & preview

`POST /api/lead/import/preview` — multipart form.

**Send:** `file` (the uploaded file).

**Receive:**

```json
{
  "success": true,
  "data": {
    "headers": ["name", "phone", "email"],
    "sample_data": [
      { "name": "John Doe", "phone": "9876543210", "email": "john@x.com" }
    ],
    "available_fields": {
      "lead_fields": {
        "name": "Lead Name", "email": "Email", "country_code": "Country Code",
        "phone": "Phone", "source": "Source", "designation": "Designation"
      },
      "company_fields": {
        "company_name": "Company Name", "company_website": "Company Website",
        "company_address": "Company Address", "company_zipcode": "Company Zipcode"
      }
    },
    "total_rows": 250,
    "row_counts": { "total": 254, "blank": 3, "header": 1, "data": 250 }
  }
}
```

**What to render:**
- A mapping table: one row per `headers` entry, each with a dropdown of the
  **system fields** from `available_fields` (group them: Lead vs Company). The
  labels (`"Lead Name"`) are for display; the keys (`name`) are what you send.
- Show `sample_data` as a small preview grid so the user can eyeball their columns.
- Show the row summary from `row_counts`:
  - `data` = leads that will actually be considered (this is also `total_rows`).
  - `blank` = empty rows that are ignored.
  - `header` = duplicate header rows pasted mid-file that are ignored.

**Build `column_mapping`** as the user maps dropdowns. It is
`{ system_field: header_key }`, e.g.:

```json
{ "name": "name", "phone": "phone", "email": "email" }
```

Only include fields the user actually mapped. `name` and `phone` are effectively
required by the importer, so warn the user if they leave those unmapped.

---

## Step 2 — Analyze (review & clean)

Once the mapping is chosen, call analyze so the user can review data quality per
field and decide how to clean it.

`POST /api/lead/import/analyze` — multipart form.

**Send:** `file` (same file again) + `column_mapping` (from step 1).

> Multipart tip: send `column_mapping` as `column_mapping[name]=name`,
> `column_mapping[phone]=phone`, … or use a JSON body if you re-upload the file
> differently. With `FormData`, append each key: `fd.append('column_mapping[name]', 'name')`.

**Receive** (one entry per mapped field under `data.fields`):

```json
{
  "success": true,
  "data": {
    "total_rows": 250,
    "row_counts": { "total": 254, "blank": 3, "header": 1, "data": 250 },
    "fields": {
      "phone": {
        "excel_header": "phone",
        "filled": 248, "empty": 2, "distinct": 246,
        "samples": ["+91 98765 43210", "9876543210"],
        "detected_formats": [
          { "label": "Plain digits, 10 digits", "count": 190, "example": "9876543210" },
          { "label": "With country code (+), Formatted (separators), 12 digits", "count": 58, "example": "+91 98765 43210" }
        ],
        "available_formats": [
          { "key": "plain_10", "label": "Plain 10 digits (9876543210)", "spec": { } },
          { "key": "with_country_code", "label": "With country code (+919876543210)", "spec": { } },
          { "key": "digits_only", "label": "Digits only, keep as-is length", "spec": { } }
        ],
        "custom_format_template": { "strip_separators": true, "strip_leading_zeros": false, "country_code": "keep", "force_country_code": "", "expected_digits": null },
        "anomalies": { "checked": false }
      },
      "name": {
        "excel_header": "name",
        "filled": 250, "empty": 0, "distinct": 249,
        "samples": ["John Doe"],
        "detected_formats": [ { "label": "Text", "count": 250, "example": "John Doe" } ],
        "available_formats": [
          { "key": "trim", "label": "Trim whitespace", "spec": { } },
          { "key": "title_case", "label": "Title Case", "spec": { } },
          { "key": "upper_case", "label": "UPPERCASE", "spec": { } },
          { "key": "lower_case", "label": "lowercase", "spec": { } }
        ],
        "custom_format_template": { "trim": true, "collapse_spaces": true, "case": "none" },
        "anomalies": { "checked": true, "affected_rows": 12, "chars": ["@", "1", "2"], "examples": ["Ann@Marie", "Bob123"] }
      }
    }
  }
}
```

**What to render, per field:**

1. **Counts** — `filled` / `empty` / `distinct`. Good for a small stat line
   ("248 of 250 filled, 246 unique").
2. **Detected formats** — list `detected_formats` with the example and count so
   the user sees "190 rows look like `9876543210`, 58 look like `+91 98765 43210`".
   This is where mixed data becomes obvious.
3. **Format picker** — a dropdown from `available_formats` (use `label`, submit
   `key`). Optionally a "Custom…" option that seeds a form from
   `custom_format_template` (see the spec reference at the end).
4. **Anomalies** — only when `anomalies.checked === true`. If `affected_rows > 0`,
   show a warning: "12 rows contain characters that don't belong
   (`@`, `1`, `2`). Examples: Ann@Marie, Bob123." Then offer the user a choice:
   **Fix** (strip them), **Discard** (skip those rows), or **Ignore**.

Control characters appear in `chars` as escaped tokens like `\x09` (tab) — render
them literally, they are intentional.

From the user's choices on this screen you assemble `field_rules` (step 3).

---

## Step 3 — Import

`POST /api/lead/import/with_mapping` — multipart form.

**Send:**

| Field | Required | Description |
|-------|----------|-------------|
| `file` | yes | The same uploaded file |
| `column_mapping` | yes | `{ system_field: header_key }` from step 1 |
| `field_rules` | no | Per-field cleanup rules built in step 2 (see below) |
| `created_by_distribution` | no | Who owns the imported leads |
| `assigned_to_distribution` | no | Who the leads are assigned to |

### `field_rules` shape

Keyed by system field. Each entry may include any of:

```json
{
  "phone": { "format_key": "plain_10" },
  "name":  { "required": true, "on_anomaly": "fix", "format_key": "title_case" },
  "email": { "format": { "trim": true, "case": "lower" }, "on_anomaly": "ignore" }
}
```

- **`format_key`** — a `key` from that field's `available_formats`. **Or**
- **`format`** — a custom spec object (from `custom_format_template`, edited).
  Send one or the other, not both.
- **`required`** — `true` skips any row where this field is blank.
- **`on_anomaly`** — `"fix"` (strip disallowed chars, keep row), `"discard"`
  (skip rows with anomalies), or `"ignore"` (default). String fields only.

Order the backend applies per row: **required-skip → anomaly (fix/discard) →
format normalization → validation & duplicate check → insert.**

### Distribution objects

Both `created_by_distribution` and `assigned_to_distribution` share this shape:

```json
{
  "mode": "percentage",
  "users": [
    { "user_id": 5, "value": 60 },
    { "user_id": 6, "value": 40 }
  ]
}
```

- `mode: "percentage"` — `value`s must **sum to exactly 100**. Validate client-side.
- `mode: "count"` — `value`s are absolute counts; their **sum must not exceed**
  the distributable row count (`total_rows` from step 1/2). Validate client-side
  to avoid a 422.

Omit a distribution block entirely if the user doesn't want to distribute (leads
then default to the importer as creator, unassigned).

### Full example request (as JSON, for reference)

```json
{
  "file": "<multipart binary>",
  "column_mapping": { "name": "name", "phone": "phone", "email": "email" },
  "field_rules": {
    "phone": { "format_key": "plain_10" },
    "name":  { "required": true, "on_anomaly": "fix" }
  },
  "created_by_distribution":  { "mode": "percentage", "users": [ { "user_id": 5, "value": 60 }, { "user_id": 6, "value": 40 } ] },
  "assigned_to_distribution": { "mode": "count", "users": [ { "user_id": 7, "value": 250 } ] }
}
```

### Response

```json
{
  "success": true,
  "message": "Lead import with mapping completed",
  "data": {
    "total_imported": 240,
    "total_failed": 4,
    "total_skipped": 6,
    "total_normalized": 198,
    "errors": [
      { "row": 12, "error": "Duplicate phone number: 9876543210 (existing lead ID: 88)" }
    ],
    "skips": [
      { "row": 20, "reason": "Required field 'name' is blank" }
    ],
    "normalization_warnings": [
      { "row": 33, "field": "phone", "warning": "Phone has 9 digits, expected 10", "value": "987654321" }
    ]
  }
}
```

**What to render on the results screen:**
- Headline: `total_imported` succeeded, plus `total_failed`, `total_skipped`,
  `total_normalized` as secondary stats.
- `errors` — rows that could not be imported (validation failed or duplicate).
  Show `row` + `error`. These are the ones the user may want to fix and re-upload.
- `skips` — rows intentionally excluded by the user's own rules (blank required
  field, or discarded anomaly). Show `row` + `reason`.
- `normalization_warnings` — rows that were imported but where a value looked off
  after cleaning (e.g. wrong phone length). Show `row`, `field`, `warning`, `value`.

`row` numbers are 1-based against the file including its header row (so `row: 2`
is the first data row), which lets the user locate the row in their spreadsheet.

---

## Format spec reference (for the "Custom…" option)

When a user wants a format not in `available_formats`, start from the field's
`custom_format_template` and let them edit it, then send it as `format`.

### Phone spec

| Key | Values | Meaning |
|-----|--------|---------|
| `strip_separators` | bool | Remove spaces, `-`, `(`, `)`, `.` |
| `strip_leading_zeros` | bool | Drop leading `0`s from the national number |
| `country_code` | `keep` \| `drop` \| `force` | Keep the existing code, remove it, or force one |
| `force_country_code` | string e.g. `"+91"` | Used only when `country_code` is `force` |
| `expected_digits` | int \| null | If set and the digit count differs, the row is imported but a warning is recorded |

### Email spec

| Key | Values | Meaning |
|-----|--------|---------|
| `trim` | bool | Trim surrounding whitespace |
| `case` | `lower` | Lowercase the address |

### Text spec (name, company_name, designation, source, company_address)

| Key | Values | Meaning |
|-----|--------|---------|
| `trim` | bool | Trim surrounding whitespace |
| `collapse_spaces` | bool | Collapse runs of whitespace to a single space |
| `case` | `none` \| `lower` \| `upper` \| `title` | Case transform |

### Country-code spec

| Key | Values | Meaning |
|-----|--------|---------|
| `trim` | bool | Trim surrounding whitespace |
| `country_code_style` | `plus` \| `digits` | `+91` vs `91` |

---

## Preset keys cheat-sheet

Use these `format_key` values directly (no analyze round-trip needed if you want
sensible defaults):

| Field | Keys |
|-------|------|
| `phone` | `plain_10`, `with_country_code`, `digits_only` |
| `email` | `lowercase_trim` |
| `country_code` | `plus_prefixed`, `digits_only` |
| `name`, `company_name`, `company_address`, `designation`, `source` | `trim`, `title_case`, `upper_case`, `lower_case` |

Fields that support **anomaly** detection/cleanup (`on_anomaly`): `name`,
`designation`, `company_name`, `source`, `company_address`. Other fields return
`anomalies.checked = false` — don't show the anomaly controls for them.

---

## Recommended UX flow

1. User uploads a file → call **preview** → show mapping table + row summary.
2. User maps columns → call **analyze** → show per-field formats, counts, and
   anomaly warnings; user picks a format and an anomaly action per field.
3. User sets distribution (optional) with live client-side validation of the
   percentage/count rules.
4. User confirms → call **with_mapping** → show the results breakdown.
5. Offer "download failed/skipped rows" (built client-side from `errors`+`skips`)
   so the user can fix and re-import just those.

## Error handling checklist

- **403** — user lacks permission; hide/disable the feature.
- **422 on `file`** — wrong type or too large; validate before upload.
- **422 on distribution** — percentages ≠ 100, or count sum > `total_rows`;
  validate client-side first.
- **500** — surface `message` and let the user retry; the file may be corrupt.