There is a difference of 24.49% in the number of reported conflicts.
Merge scenarios had at least one conflict with semistructured merge on average 5.66% of the considered merge scenarios, with a standard deviation of 5.39% . Considering aggregated numbers of all projects of our sample, this corresponds to 5.64% of the considered merge scenarios. Besides that, merge scenarios had at least one conflict with unstructured merge on average 6.52% of the considered merge scenarios, with a standard deviation of 6.22% . Considering aggregated numbers of all projects of our sample, this corresponds to 6.5% of the considered merge scenarios.
The tools differ on average 2.19% of the considered merge scenarios, with a standard deviation of 2.45% . Considering aggregated numbers of all projects of our sample, this corresponds to 2.49% of the considered merge scenarios.
The tools differ on average 34.62% of the considered merge scenarios, with a standard deviation of 34.05% . Considering aggregated numbers of all projects of our sample, this corresponds to 34.02% of the considered merge scenarios.
On average 1.76% of the considered merge scenarios, had semistructured merge additional falses positives, with a standard deviation of 1.62% . Considering aggregated numbers of all projects of our sample, this corresponds to 1.66% of the considered merge scenarios.
On average 18.02% of the number of semistructured merge reported conflicts are additional falses positives, with a standard deviation of 17.41% . Considering aggregated numbers of all projects of our sample, this corresponds to 14.09% of the number of semistructured merge reported conflicts.
On average 4.24% of the considered merge scenarios, had unstructured merge additional falses positives, with a standard deviation of 4.49% . Considering aggregated numbers of all projects of our sample, this corresponds to 4.27% of the considered merge scenarios.
On average 40.89% of the number of unstructured merge reported conflicts are additional falses positives, with a standard deviation of 13.94% . Considering aggregated numbers of all projects of our sample, this corresponds to 45.63% of the number of unstructured merge reported conflicts.
Statistical significance by scenarios, consider a confidence level of 0.95 (p-value = 0.05):
## Warning in wilcox.test.default(aFP_ST$unstructured, aFP_SS
## $Semistructured, : cannot compute exact p-value with zeroes
##
## Wilcoxon signed rank test with continuity correction
##
## data: aFP_ST$unstructured and aFP_SS$Semistructured
## V = 99, p-value = 0.00388
## alternative hypothesis: true location shift is not equal to 0
Strength/magnitude of the statistical claim (effect size) by scenarios:
##
## Cliff's Delta
##
## delta estimate: 0.3377778 (medium)
## 95 percent confidence interval:
## lower upper
## -0.09307555 0.66208097
Statistical significance by conflicts, consider a confidence level of 0.95 (p-value = 0.05):
##
## Wilcoxon signed rank test
##
## data: aFP_ST$unstructured and aFP_SS$Semistructured
## V = 110, p-value = 0.002625
## alternative hypothesis: true location shift is not equal to 0
Strength/magnitude of the statistical claim (effect size) by conflicts:
##
## Cliff's Delta
##
## delta estimate: 0.6844444 (large)
## 95 percent confidence interval:
## lower upper
## 0.2672163 0.8855660
On average 3.25% of the considered merge scenarios, had semistructured merge additional falses negatives, with a standard deviation of 3.49% . Considering aggregated numbers of all projects of our sample, this corresponds to 3% of the considered merge scenarios.
On average 34.4% of the number of semistructured merge conflicts are semistructured merge additional falses negatives, with a standard deviation of 20.32% . Considering aggregated numbers of all projects of our sample, this corresponds to 30.32% of the number of semistructured merge conflicts.
On average 0.32% of the considered merge scenarios, had unstructured merge additional falses negatives, with a standard deviation of 0.49% . Considering aggregated numbers of all projects of our sample, this corresponds to 0.36% of the considered merge scenarios.
On average 1.24% of the number of unstructured merge conflicts are unstructured merge additional falses negatives, with a standard deviation of 1.74% . Considering aggregated numbers of all projects of our sample, this corresponds to 2.34% of the number of unstructured merge conflicts.
Statistical significance by scenarios, consider a confidence level of 0.95 (p-value = 0.05):
## Warning in wilcox.test.default(aFN_SS$Semistructured, aFN_UN
## $Unstructured, : cannot compute exact p-value with zeroes
##
## Wilcoxon signed rank test with continuity correction
##
## data: aFN_SS$Semistructured and aFN_UN$Unstructured
## V = 105, p-value = 0.001097
## alternative hypothesis: true location shift is not equal to 0
Strength/magnitude of the statistical claim (effect size) by scenarios:
##
## Cliff's Delta
##
## delta estimate: 0.72 (large)
## 95 percent confidence interval:
## lower upper
## 0.3522688 0.8951494
Statistical significance by conflicts, consider a confidence level of 0.95 (p-value = 0.05):
## Warning in wilcox.test.default(aFN_SS$Semistructured, aFN_UN
## $Unstructured, : cannot compute exact p-value with ties
## Warning in wilcox.test.default(aFN_SS$Semistructured, aFN_UN
## $Unstructured, : cannot compute exact p-value with zeroes
##
## Wilcoxon signed rank test with continuity correction
##
## data: aFN_SS$Semistructured and aFN_UN$Unstructured
## V = 105, p-value = 0.001094
## alternative hypothesis: true location shift is not equal to 0
Strength/magnitude of the statistical claim (effect size) by conflicts:
##
## Cliff's Delta
##
## delta estimate: 0.8977778 (large)
## 95 percent confidence interval:
## lower upper
## 0.5291413 0.9813313