Tags

3.10.0

Incremented minor version for R6: Great spotted woodpecker release version

Overview of new functionalities (compared to previous, R5: Green kingfisher (3.6.0) release):
+ Model handling:
	- Model:
		- Updates on model parsing:
			- Critical log message is given if a mandatory field (model `name`/`id`, `startElementId`, `vertices`, `edges`, `name`/`id`/`sourceVertexId`/`targetVertexId` field of edges) is missing from the model
		- Updates on EFSM models:
			- Handle user-defined struct variables
			- Handle ternary conditional expressions
			- Handle global initialization actions
		- Handle CEFSM models
		- Log model information:
			- Add `--graphviz` flag for Graphviz drawing and visualize guard conditions and actions of EFSMs models
			- Log CEFSM attributes
	- Model conversions:
		- Updates on EFSM to FSM conversion:
			- Add `--graphviz` flag for Graphviz drawing
		- CEFSM to FSM conversion:
			- Handles 1-1, 1-many, many-1 communications
			- Handles blocking options (single, all, fixed)
			- Handles the expected messages clause
		- Updates on Add reset to the model:
			- Reliable/unreliable reset can be added to EFSM models as well (in this case variables will be also initialized using global initialization action)
		- Updates on Inject random faults to the model:
			- Generate all possible fault models of given change injection settings if `--all_faults` CLI/configuration flag is enabled. A report file is also created if this functionality is activated.
			- The case can be handled (with corresponding `sc_retries` flag) if next state fault would result in a non-strongly connected model
+ Test generation:
	- New test generation algorithms:
		- Model-based mutation testing:
			- It generates all possible mutant models for the given list of (first or higher order) change types
			- It uses a given list of test generation algorithms (with their relevant parameters) to generate test suites that tries to kill at least as many mutants models as it is defined by the `target_score` or if it is not possible, it tries to kill as many mutants as possible
			- Besides test set and summary csv, efficency and detection score reports are also saved
			- One can also save all generated mutant models with the `--save_mutants` flag
			- One can also use the `--mutation_map` flag to create a map containing all generated mutant models and the test suite which killed that mutant
	- Updates on existing test generation algorithms:
		- Optional subsequences (`--subsequences` flag):
			- Subsequences functionality has been added for the following test generation algorithms:
				- Harmonized State Identifiers (HSI): *Q* State Cover Set, Transition Tests (including extra state extensions), and *Z* Separating Family of Sequences
				- H-method: *Q* State Cover Set, Transition Tests (including extra state extensions), and Postamble sequences used in state verification
				- All-Transition-Transition (ATT): Main sequences and transition adjacent alternative test sequences
			- Additionally, when both the `--subsequences` and `--transition_list` flags are enabled, the tools now output not only the input lists, but also the associated ID and name lists in the same format for the above three methods. This extended output format is also supported by the All-Transition-State (ATS) algorithm.
		- Introduce Test Set, that is the container of test suites and their additional data (like method, and command-, parameter-, config history)
		- Add applied test generation parameters under the `parameters` field of the test suite file
		- Improve Graphviz graph visualization:
			- Adding input/output symbols to ATS subgraphs
			- Adding reset symbols for TT/ATS graphs if `consider_reset_transitions` is set to true either implicitly or explicitly during test generation
		- Unify CSV summary naming style:
			- Random Walk and Weighted Random Walk now use the shortened version for CSV file naming as other algorithms: `r_result.csv`, `wr_result.csv`
+ Application examples:
	- New sample test project, to test the Ory Hydra open source OpenIddict and OpenID Connect server through an API

Bugfixes:
+ Model handling:
	- Graphviz:
		- Fix empty output visualization
		- Fixing behaviour of `--graphviz` flag in State minimization conversion
		- Unifying `reset_visualization` usage
	- Model conversions:
		- EFSM->FSM conversion:
			- Fix global initialization of variables
			- Remove unnecessary "Empty output" warning messages for the converted model
			- Fix a rare, undefined behaviour issue related to input/output symbols
		- Inject random faults to the model:
			- `REMOVE_STATE` change can not remove start state anymore
			- Fix a rare, undefined behaviour issue related to input/output symbols
		- Add `ADD_IOSYMBOLS` under the `conversionMethod` field of the converted model if this conversion mode was applied
+ Test generation:
	- Transition Tour (TT):
		- Fixes in debug level logging: list of states is provided, list of transition is refactored
	- Harmonized State Identifiers (HSI) and H methods:
		- Write state names instead of state ids in the log of State Cover Set _Q_ 
	- Test unreliable reset symbols (RCS):
		- Test suites are saved now into the right place (`test_suites` subfolder)
		- Fix "Transition not found" error, when reset checking sequence generation is called indirectly by H/HSI for a model that has unreliable resets
	- Fix the bug, when CsvManager used uninitialized field in constructor
+ Test Execution:
	- `TestSuiteExecutor.execute()` throws `NonExistingStartStateException` when the start state of the model is no longer existing in states of the model
+ Simulation Script:
	- Fix working of reduced model generation
	- Fix working of tracking process of already progressed scenarios
	- Fix the Y axis of diagram generation in case of fractional numbers

Other improvements (useful for developers):
+ Refactoring TestSuiteParser; add specific parsers to have the used parameters of the given test generation algorithm parsed.
+ Refactoring of Graphviz file generations (creating base class, outsourcing graphviz drawings of test generation algorithms)
+ Initial type-checking no longer occurs during parsing, but instead during a separate pass right after parsing
+ Use constants in the code more extensively for better maintainability
+ CI tests:
	- Sample simulation script commands of User Guide have been added automatically into test pipeline
	- Commands of notes for contributors and Ory Hydra application example documentations have been added automatically into test pipeline
	- Regression tests for test generation algorithms have been added into test pipeline
	- Refactor test suite parser and test generation tests
	- Add new CI tests for EFSM->FSM conversion, remove deprecated manual EFSM tests
	- Update test model descriptions
+ Add verification timer for the debug mode of test generation algorithms
+ Extend notes for contributors with the following sections/subsections:
	- TestSuiteParser
	- Graphviz file generation
+ Add weights and costs to some delivered introduction sample models

--------------------------------

Many bugfixes/implementation of missing features even since 3.9.0 related mainly to new functionalities (Model-based mutation testing, CEFSM->EFSM transformation...etc.):
- Add error msg for missing mandatory JSON fields (482, MR 443)
- Explicitly added previous conversion methods, reset information and change data list to the transformed model in EFSM->FSM (536, 537, MR 442)
- Fix EFSM transform IO issues (526, MR 441)
- Fixed bug when CEFSM to EFSM would not add conversion method to the transformed model (533, MR 440)
- Fix TT node sequence debug log (524, MR 438)
- Fix wreorder warnings (325, MR 437)
- Fixed CEFSM test model issue (527, MR 436)
- Implement test suite checking test (517, MR 437)
- Use final iteration num for ats depth for test suite param (529, MR 435)
- Don't create empty reports folder in TestSuiteExecutor when used by MUT (523, MR 433)
- Set converted flag true for addiosymbols in model (528, MR 432)
- Fix total system memory query for windows platform (MR 430)
- Fix windows build (MR 429)
- Log warning message if estimated memory usage is higher than available system memory (522, MR 428)
- Fix mutation config path setting in sim script (521, MR 426)
- Fix RCS test suite params (520, MR 427)
- Set subfolder flag for test set writer in MUT, fixes in MUT test suite handling (506, MR 423)
- Fix architecture figure (512, MR 424)
- Resolve "[conversion: ] CEFSM->EFSM transformation: Handle many to one communication Follow-up tasks" (485, 488, 502, 510, 511, 513, MR 422)
- Resolve "[bug: ] Test suite file is not written in case of test generation" (517, MR 421)
- [sim script/mut: ] Add parameters of model-based mutation testing into simulation script (507)
- Bugfix for "[bug: ] ATS params added to the test suite of AS" (516, MR 420)
- Update architecture figure of MTR (515, MRs 419+424)
- Create application example for the Ory Hydra test project (505, MR 414)
- "Add the ability to Test Set Parser/Writer to handle test suites in a subfolder" (503, MR 412)
- Fix new compiler warning (MR 416)
- Mutation test generation algorithm (337, 490, 491, MR 305)
- Remove unused logger name U-IG (MR 413)

3.9.29

3.9.28

3.9.27

3.9.26

3.9.25

3.9.24

3.9.23

3.9.22

3.9.21

3.9.20

3.9.19

3.9.18

3.9.17

3.9.16

3.9.15

3.9.14

3.9.13

3.9.12

3.9.11