How it works

An assessment is an argument about what could go wrong with a system, built by the people who know it. Numbers are added as far as the group can, and the structure keeps a record of how each conclusion was reached.

The structure

Every assessment has the same four levels, from the Target of Assessment down to Defences and Transfers.

  • Target of Assessment. The system, product, service or process you are assessing.
  • Risk. A loss you care about, such as a data leak or an outage. It has an impact: what it costs each time it happens.
  • Attack. A way a risk can happen. It has a likelihood: how often per year.
  • Defence. A component or decision that reduces a risk. It has a status (implemented, planned or shelved) and a strength against each attack it is linked to.
  • Transfer. A defence that shifts the loss to someone else, such as an insurance policy, a contract clause or a disclaimer. It counts like any defence.
An example assessment The target, a customer portal, has two risks: a data leak and a service outage. Three attacks lead to them. SQL injection leads to the data leak, a DDoS attack leads to the service outage, and a phished admin login leads to both. SQL injection is countered by parameterised queries. The phished admin login is countered by two-factor sign-in and by a planned cyber insurance transfer. The DDoS attack has no defence yet. Customer portal target Data leak risk · €250k per event Service outage risk · €40k per event SQL injection attack · 0.3 a year Phished admin login attack · 0.8 a year DDoS attack · 2 a year Parameterised queries defence · −90% Two-factor sign-in defence · −80% Cyber insurance transfer · −50% no defence yet
Illustration with invented figures. The thick teal outline marks one attack shared by two risks. The dashed outline marks a planned transfer, and the dotted box an attack that has no defence yet.

An argument you can inspect

An attack is a claim about how a risk happens. A defence is a counter-claim: it says that something reduces that attack. The group argues the graph into shape, and the gaps show on the page: a risk with no attack under it, or an attack with no defence, is an empty branch that someone can be asked to fill.

Every change is recorded, and each saved version says who or what saved it and lists the changes that went into it, so the finished assessment shows how it got its shape. The history and version features keep that record.

One attack, several risks

The structure is a directed acyclic graph. The same attack, such as phishing for an admin login, is often a way into more than one risk, and the same defence, such as two-factor sign-in, often protects against more than one attack. In ArgueSecure the attack or defence is one node that appears in several places, so editing it in one place edits it everywhere. A badge on the node shows how many places it appears in.

A defence has its strength set per attack. One control can stop 80% of one kind of attack and 20% of another, and the assessment records both.

Lightweight and collaborative

A risk assessment depends on what many people know, and they rarely know the same things. Engineers know how the system fails, product owners know which losses matter, security people know which attacks are current, and management knows what the organisation can bear. ArgueSecure gives them one shared graph to add to, in a workshop room or over a few weeks, with comments and @mentions on the items they are discussing.

  • Start with a brainstorm. The first pass needs only titles: risks, ways they could happen, things that might help.
  • Add numbers when you have them. No prior quantitative data is needed to begin. A value that is still missing is flagged, and the rest of the assessment works.
  • Keep the reasoning. The claims and counter-claims, and the history of changes to them, stay in the assessment, so a reader can see how a conclusion was reached.

The idea comes from the 2016 paper behind the tool (doi:10.1109/REW.2016.027). It starts from a practical problem: in small companies and projects, a risk assessment is often skipped, done from a checklist, or handed to an outside consultant. Checklists cover known risks but miss the ones that come with new technology, and a consultant still needs the people who know the system and the people who hold the budget. The paper argues for a participative session as a supplement to both, run by the project team itself, since project teams are the most common participants in risk meetings and tend to prefer qualitative methods.

The same reasoning shaped the online version. The people whose knowledge is needed are busy and rarely free at the same time, so they should be able to add what they know from wherever they are, without installing anything, while the link from each defence back to the risk it answers is kept.

Why the structure is so plain

The first ArgueSecure tools were tried in a pilot study, a focus group with a bank's security staff and one with security researchers, a case study, and an online study at a requirements engineering conference. What people did in those sessions decided what the model kept:

  • No attacker and defender teams. Participants were asked to split into two sides and did not. Anyone can add a risk, an attack or a defence at any time.
  • No formal argument fields. Assumptions, warrants and qualifiers were left empty or misused, and the experts saw writing down common knowledge as wasted time. A claim and its grounds, in plain words, is enough.
  • Many-to-many links. People wanted one attack to lead to several risks and one defence to answer several attacks. The graph allows both, and a defence's strength is set per attack.
  • A shallow graph. Deep trees became hard to read, so categories became tags and the levels are fixed: risk, attack, defence.

What every expert in those studies did want was traceability: being able to see why a defence is there. That is the part ArgueSecure keeps most carefully. Today's ArgueSecure is a rewrite built on these concepts, not the 2016 software. See About for its background.

The numbers in plain terms

For every pair of a risk and an attack that leads to it, ArgueSecure works out an expected annual loss. The assessment's total is the sum over every pair that has its values.

expected annual loss = how often the attack happens per year
  × the loss each time the risk occurs
  × the share each defence leaves

Impact is the loss per occurrence. If a breach costs €200,000 to clean up, enter €200,000. Likelihood is a rate per year. A value of 0.5 means about one occurrence every two years, and 2 is valid and means twice a year. A defence that stops 80% of an attack leaves 20%, so its factor is 0.2.

Risk: Data leak
€250,000impact, per occurrence
Attack: Phished admin login
0.8likelihood, per year
Inherent, no defences
€200,0000.8 × €250,000
Current, two-factor sign-in implemented (−80%)
€40,000€200,000 × 0.2
Target, plus the planned insurance (−50%)
€20,000€40,000 × 0.5, 90% below inherent

Three baselines

The total is worked out three times, for three sets of defences. Inherent counts no defences. Current counts only the implemented ones. Target counts the implemented ones and the planned ones. Shelved defences never count in any of them. The report shows all three, with the reduction from one to the next.

Missing values

While an assessment is in progress, some values are blank. A risk with no impact, or an attack with no likelihood, leaves its pairs out of the total, and the total is shown as a lower bound such as ≥ €53,500 with the number of valued paths. Counting a blank as zero would make the risk look smaller than it is. A defence with no strength set counts as no reduction, which overstates what is left. Both defaults err towards showing more risk.

What the numbers do not claim

  • Results are amounts in your currency with a reduction percentage. They are never mapped to low, medium and high.
  • Stacked defences on one attack are assumed to fail independently. Controls that share a weak point make the real residual risk higher than shown, so read a figure built on many stacked controls as optimistic.
  • An attack linked to several risks is assumed to cause every linked loss when it succeeds. If two risks describe the same loss, merge them.

The model is specified in full in the calculation engine spec, and explained for users in the in-app help.

See an assessment in the tool

Start a new assessment from the Northwind Outfitters template, a small web shop part-way through its assessment, to see all of this in the tool. The features page lists what else is there.