Start here

Running an evaluation

The core loop, done by hand in the app. Everything here applies to any evaluation type — vendor proposals, applications, compliance submissions. If you are screening candidates from Greenhouse, start with Connecting an ATS instead; most of these steps happen automatically there.

Before you start

You need two things in hand:

  • A source document — whatever defines what "good" means. A job description, an RFP, a policy, a scoring rubric.
  • At least one target — the thing being evaluated. A proposal, a resume, an application, a submission.

Supported file types are .pdf, .docx, .doc, .txt, .rtf, .xlsx and .csv.

1. Add a pipeline

A pipeline is the evaluation workflow — the prompts, report templates and resources for one kind of decision. Open the catalog and add the pipeline that matches the decision you are making.

Screenshot docs/how-to/01-catalog-add-pipeline.png

The catalog, with a pipeline being added to the workspace.

2. Upload source documents

Open the pipeline and upload the source documents criteria will be extracted from. You can upload several — Reckn reads across all of them when it generates criteria.

Screenshot docs/how-to/02-upload-sources.png

Source document upload on the pipeline's documents tab.

Tip

Upload the document that actually governs the decision, not a summary of it. Criteria extracted from a one-paragraph brief will be thin, and a thin criteria set is where evaluations go vague.

3. Generate criteria

Create a criteria set and generate from the source documents. Reckn extracts the evaluation criteria it finds, each with a name and a description of what is being assessed.

This is a draft. Edit the wording, delete criteria that do not belong, and add anything the source document implies but never states. You can also generate a single additional criterion if you want Reckn to draft one for you.

Video docs/how-to/03-generate-criteria.mp4

Generating criteria from the uploaded sources, then editing the draft.

4. Set weights

Weights decide how much each criterion moves the result. Reckn derives them from pairwise comparisons — for each pair of criteria, which matters more, and by how much.

There are two ways to get there:

  • Simple — Reckn proposes a full set of comparisons and derives weights from them. Fastest path; you review the result.
  • Advanced — the same, with finer-grained reasoning across the comparison matrix.

Either way, you can override any individual comparison afterwards and the weights recalculate.

Screenshot docs/how-to/04-pairwise-weighting.png

The pairwise comparison matrix and the weights derived from it.

Note

Weighting is the layer you own. Reckn will propose, but the weights that ship are the ones you approved — and they are recorded that way in the audit trail.

5. Lock the criteria set

Locking freezes the set. From that point the criteria and weights cannot drift, and every run against the set is measured on identical terms. Runs require a locked set.

You can unlock a set later if the underlying requirements change. Runs already completed keep pointing at the version they were evaluated against.

Screenshot docs/how-to/05-lock-criteria.png

The lock confirmation, showing the criteria and weights being frozen.

6. Run an evaluation

Upload the targets — the documents being evaluated — then create a run against the locked criteria set. A run takes a name, the criteria set, and the target.

Targets come in a few shapes:

Target type What it is
DOCUMENTS One or more uploaded target documents. The common case.
TEXT Raw text pasted or passed straight in.
ID An identifier the pipeline knows how to resolve.
SLUG A named handle the pipeline knows how to resolve.
Video docs/how-to/06-create-run.mp4

Uploading targets and starting a run against the locked set.

A run moves through PENDING, PROCESSING, then COMPLETE or FAILED. The page updates live — you do not need to refresh. A failed run can be retriggered without recreating it.

7. Read the record

The overall assessment is the top layer. Underneath it, for every criterion, the record holds:

  • The assessment against that criterion
  • The weight applied to it
  • The reasoning behind the assessment, in full
  • A confidence range on that assessment
  • How it contributed to the rollup
Video docs/how-to/07-drill-into-result.mp4

Drilling from the overall assessment into per-criterion reasoning and confidence.

Tip

Sort by confidence before you sort by score. Low-confidence criteria are where a human reviewer changes the outcome; high-confidence ones rarely repay the time.

8. Export the report

Every completed run renders to a report — HTML for review in the browser, PDF for anything that has to leave the building. If the pipeline has more than one report template, choose the one that fits the audience.

Screenshot docs/how-to/08-export-report.png

Report export, with template and format selection.

Reusing a locked set

A locked criteria set is meant to be run many times. Mark one as the pipeline default and every new run starts against it. That is what keeps a batch of fifty evaluations comparable: same criteria, same weights, same version, recorded on each run.

Audit trail

Nothing above is a log line that scrolls away. Every criteria edit, weight change, lock, run and delivery is recorded with the actor who did it and when — whether that actor was a person or an API key.

Each run also produces an artifact: a hashed, versioned record of the inputs and outputs. It can be verified against its hash later, which is what turns "here is the result" into "here is the result, and here is proof it has not changed since."

Screenshot docs/how-to/09-activity-log.png

The activity log, filtered to one criteria set.