Scala algorithm: Monitor success rate of a process that may fail
Published
Algorithm goal
In a distributed system, we wish to monitor processes which may succeed or fail; if a process fails too often, we should kill it. Implement an algorithm or data structure that, based on a set of observations of successes and failures, can tell us whether the process needs to be killed.
In software in practice, this is called flapping, where a service keeps on dying or not performing in some way; it could be due to a memory leak for instance, and such a service can recover by killing it and restarting it.
Assume the following inputs:
- A # of target successes (\(a\))
- A # of attempts minimum (\(b\)) (meaning, success ratio is \(a / b\) once \(b\) attempts are reached; as we'd otherwise have 100% failure rate if we only considered one attempt)
Test cases in Scala
assert(
!SuccessMonitor(targetSuccesses = 4, outOfAttempts = 5).succeeded.shouldKill
)
assert(
!SuccessMonitor(
targetSuccesses = 4,
outOfAttempts = 5
).succeeded.failed.shouldKill
)
assert(
!SuccessMonitor(
targetSuccesses = 4,
outOfAttempts = 5
).succeeded.failed.succeeded.shouldKill
)
assert(
!SuccessMonitor(
targetSuccesses = 4,
outOfAttempts = 5
).succeeded.failed.succeeded.failed.shouldKill
)
assert(
SuccessMonitor(
targetSuccesses = 4,
outOfAttempts = 5
).succeeded.failed.succeeded.failed.succeeded.shouldKill
)
assert(
!SuccessMonitor(
targetSuccesses = 4,
outOfAttempts = 5
).succeeded.succeeded.succeeded.failed.succeeded.shouldKill
)
assert(
!SuccessMonitor(
targetSuccesses = 4,
outOfAttempts = 5
).failed.failed.shouldKill
)
Algorithm in Scala
17 lines of Scala (compatible versions 2.13 & 3.0), showing how concise Scala can be!
Explanation
This is a State machine, which enables us to represent transitions in an immutable fashion. While the algorithm is fairly straightforward, the way to implement it in Scala in the neatest way is using an immutable Queue combined with pattern matching: basically, we dequeue from a list of results only when we had reached the # of attempts \(b\). (this is © from www.scala-algorithms.com)
Scala concepts & Hints
Pattern Matching
Pattern matching in Scala lets you quickly identify what you are looking for in a data, and also extract it.