Scoring an attribute

Scoring reads one attribute of every entity in a collection, places each entity on levels you describe, and saves the results in a new column. For example, it can read a free-text “Education” attribute and fill a “Highest degree” column with bachelor’s, master’s or doctorate. Use it when the answer is a judgment rather than a value to look up, for example:

  • which suppliers are far enough along to shortlist
  • how mature each company’s disclosure is
  • whether a program is early, established or winding down

Every entity is judged against the same levels by the same rule, so the results are consistent across the collection.

Asking for a score

Ask your assistant in your conversation. Tell it:

  • which collection to score
  • which attribute to read
  • the levels, in order, lowest first
  • a name for the column the results go in

Your assistant sends the request and tells you scoring has started. The rest happens in the background: the new column appears on the collection’s page right away and fills in over the next few minutes. You can ask for several attributes at once; they are scored one after another. 

YOU ASK THE ASSISTANT

“score these companies on Disclosure, into a column called Disclosure
 level: no public figures, figures published but not audited, audited
 figures for last year, or audited figures for this year.”

THE ASSISTANT REPLIES

Scoring Disclosure for 4 entities into Disclosure level in the
background. The column fills in over the next few minutes.

A FEW MINUTES LATER, ON THE COLLECTION’S PAGE

entity      Disclosure level
------      ----------------
Northwind   L4: Audited figures, current year
Meridian    L2: Figures published, not audited
Brightline  L3 (split): Audited figures, prior year
Kestrel     (empty: nothing recorded in Disclosure)

What the column shows

Each cell holds the level, numbered from the lowest, and its description. The level is decided by the judge, a model separate from your assistant.

  • (split) means the judge was torn between the levels on either side, so the level shown sits between them. Check these first.
  • An empty cell means nothing is recorded in the attribute for that entity, so there was nothing to score.

Your assistant can tell you how confident the judge was about any level.

Writing good levels

  • Keep to one dimension. The levels should all describe the same thing, in order. To judge two things, score twice.
  • Describe situations, not degrees. “Discloses audited figures for the current year” is a level. “Good” is not, because it doesn’t say what the entity actually does.
  • Say where each level ends. Use 2 to 10 levels; 3 to 5 is usually plenty.
  • Describe the attribute. If it has guidance saying what belongs in it, the judge uses that too.

Scoring reads only what is recorded in that one attribute. It doesn’t do research or look at other attributes.

Scoring again

Ask again with the same column name and the same levels, for example after the attribute’s values change. Each cell gets the latest level, and a level that hasn’t changed stays as it is.

Nobody edits the results column, you or an assistant. To change a level, change what it was read from: correct the attribute’s values, or choose between competing claims, then score again. For different levels, use a new column name.

Saved levels can be used when filling a collection: “suppliers whose Disclosure level is audited figures” checks what you already decided instead of making a new judgment.