Hi David,
1. Is it possible to store and execute previously
generated offline test? Something like a "replay" function?
Tests are stored as test logs. You can generate an offline test by
$ fmbt -l offline.log -o adapter=dummy test.conf
-o adapter=dummy disables whatever adapter is defined in test.conf.
The most robust way to execute the same test steps again is fmbt's interactive mode
(fmbt -i).
You can execute arbitrary actions by typing them to interactive mode prompt:
$ fmbt -l replay.log -i test.conf
fMBT> i:step 1
executing: i:step 1
adapter: i:step 1
model: ok
...
On the other hand, you can use fmbt-log to extract executed actions from logs:
$ fmbt-log -f '$as' offline.log
...thus you can replay test steps from offline.log by:
$ fmbt-log -f '$as' offline.log | fmbt -l replay.log -i test.conf
2. Maybe this one is a bit dumb but i still have the doubt.
Most of the examples I found use the aal adapter but I would like
to have the adapter and the model totally independent ( I might
use the same adapter for a different model).
You can write a separate AAL file that implements only adapter blocks for test steps.
Example:
pure_model.aal:
---8<---
aal "pure_model" {
language "python" {}
variables { x }
initial_state {
x = 1
}
input "step 1" {
guard { return x == 1 }
body { x = 2 }
}
input "step 2" {
guard { return x == 2 }
body { x = 1 }
}
}
--->8---
pure_adapter.aal:
---8<---
aal "pure_adapter" {
language "python" {}
variables {}
input "step 2" {
adapter { print "*** STEP 2" }
}
input "step 1" {
adapter { print "*** STEP 1" }
}
input "step 3" {
adapter { print "*** STEP 3" }
}
}
--->8---
test.conf that uses pure_model for test generation, while tests are implemented in
pure_adapter:
---8<---
model = aal_remote(remote_pyaal -l model.log pure_model.aal)
adapter = aal_remote(remote_pyaal -l adapter.log pure_adapter.aal)
heuristic = lookahead(4)
coverage = perm(1)
pass = steps(4)
--->8---
Note that pure_adapter.aal cannot use the variables defined in pure_model.aal.
On the other hand, you can implement a remote adapter with any programming language pretty
easily. A reference implementation in Python (adapter.py):
---8<---
import sys
LOGFILE = "/tmp/reference-adapter.txt"
def log(msg):
file(LOGFILE, "a").write("%s\n" % (msg,))
log("started")
# Read test step count
step_count = int(sys.stdin.readline().strip())
# Read names of test steps
steps = []
for i in xrange(step_count):
steps.append(sys.stdin.readline().rstrip())
log("%s steps read" % (step_count,))
# Test execution loop
while True:
# Read test step to be executed
try:
step_index = int(sys.stdin.readline().rstrip())
except ValueError:
break
step_name = steps[step_index]
# Execute test step
# if step_name == "i:foo":
# ...
# elif ...
log("executing %s" % (step_name,))
# Report the result
sys.stderr.write("%s\n" % (step_index,)) # report successful execution
# sys.stderr.write("0\n") would report failure
# sys.stderr.write(some_other_index) would report some other step
# Adapter exit
log("stopped")
--->8---
This is how you can configure a test run that uses the reference adapter:
---8<---
model = aal_remote(remote_pyaal -l model.log pure_model.aal)
adapter = remote_noencode(python adapter.py)
heuristic = lookahead(4)
coverage = perm(2)
pass = steps(20)
--->8---
remote_noencode communicates with spawned adapter process through stdin/out/error, and
according to the remote adapter protocol specified in
the Remote adapter section of
https://github.com/01org/fMBT/blob/master/doc/adapters.txt
(noencode means that names of actions will not be URL encoded as usual).
I hope this helps,
Antti