CharacterErrorRate

class ignite.metrics.CharacterErrorRate(output_transform=<function CharacterErrorRate.<lambda>>, device=device(type='cpu'), skip_unrolling=False)

Calculates the Character Error Rate (CER).

CER is defined as the total number of errors (substitutions, deletions, and insertions) at the character level divided by the total number of characters in the reference sequence.

$$\text{CER} = \frac{S + D + I}{N} = \frac{S + D + I}{S + D + C}$$

where $S$ is the number of substitutions, $D$ is the number of deletions, $I$ is the number of insertions, $C$ is the number of correct characters, and $N$ is the total number of characters in the reference ($N = S + D + C$).

• update must receive input of the form (y_pred, y).

• y_pred and y both must be either str or list of str.

• When both inputs are plain str, they are treated as a single-element batch.

Parameters:

• output_transform (Callable) – a callable that is used to transform the Engine’s process_function’s output into the form expected by the metric.

• device (str | device) – specifies which device updates are accumulated on. By default, CPU.

• skip_unrolling (bool) – specifies whether output should be unrolled before being fed to update method.

Examples

For more information on how metric works with Engine, visit Attach Engine API.

from ignite.metrics.nlp import CharacterErrorRate

cer = CharacterErrorRate()

y_pred = ["the cat sat on the mat", "hello world"]
y = ["the cat sat on mat", "hello world"]

cer.update((y_pred, y))
print(round(cer.compute(), 4))

0.1379

New in version 0.5.2.

Methods

compute	Computes the metric based on its accumulated state.
reset	Resets the metric to its initial state.
update	Updates the metric's state using the passed batch output.

compute()

Computes the metric based on its accumulated state.

By default, this is called at the end of each epoch.

Returns:

the actual quantity of interest. However, if a Mapping is returned, it will be (shallow) flattened into engine.state.metrics when completed() is called.

Return type:

Any

Raises:

NotComputableError – raised when the metric cannot be computed.

reset()

Resets the metric to its initial state.

By default, this is called at the start of each epoch.

Return type:

None

update(output)

Updates the metric’s state using the passed batch output.

By default, this is called once for each batch.

Parameters:

output (Sequence[str]) – the is the output from the engine’s process function.

Return type:

None