How often conflicts occur with semistructured and unstructured merge?

There is a difference of 24.49% in the number of reported conflicts.

Merge scenarios had at least one conflict with semistructured merge on average 5.66% of the considered merge scenarios, with a standard deviation of 5.39% . Considering aggregated numbers of all projects of our sample, this corresponds to 5.64% of the considered merge scenarios. Besides that, merge scenarios had at least one conflict with unstructured merge on average 6.52% of the considered merge scenarios, with a standard deviation of 6.22% . Considering aggregated numbers of all projects of our sample, this corresponds to 6.5% of the considered merge scenarios.

How long does take semistructured and unstructured merge?

How often semistructured and unstructured merge differ?

By All Scenarios

The tools differ on average 2.19% of the considered merge scenarios, with a standard deviation of 2.45% . Considering aggregated numbers of all projects of our sample, this corresponds to 2.49% of the considered merge scenarios.

By Conflicting Scenarios

The tools differ on average 34.62% of the considered merge scenarios, with a standard deviation of 34.05% . Considering aggregated numbers of all projects of our sample, this corresponds to 34.02% of the considered merge scenarios.

Boxplot

When compared to semistructured merge, does unstructured merge reduce unnecessary integration effort by reporting fewer false positives?

Semistructured Merge Additional False Positives

By All Scenarios

On average 1.76% of the considered merge scenarios, had semistructured merge additional falses positives, with a standard deviation of 1.62% . Considering aggregated numbers of all projects of our sample, this corresponds to 1.66% of the considered merge scenarios.

By Conflicts

On average 18.02% of the number of semistructured merge reported conflicts are additional falses positives, with a standard deviation of 17.41% . Considering aggregated numbers of all projects of our sample, this corresponds to 14.09% of the number of semistructured merge reported conflicts.

Unstructured Merge Additional False Positives

By All Scenarios

On average 4.24% of the considered merge scenarios, had unstructured merge additional falses positives, with a standard deviation of 4.49% . Considering aggregated numbers of all projects of our sample, this corresponds to 4.27% of the considered merge scenarios.

By Conflicts

On average 40.89% of the number of unstructured merge reported conflicts are additional falses positives, with a standard deviation of 13.94% . Considering aggregated numbers of all projects of our sample, this corresponds to 45.63% of the number of unstructured merge reported conflicts.

Statistical significance by scenarios, consider a confidence level of 0.95 (p-value = 0.05):

## Warning in wilcox.test.default(aFP_ST$unstructured, aFP_SS
## $Semistructured, : cannot compute exact p-value with zeroes
## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  aFP_ST$unstructured and aFP_SS$Semistructured
## V = 99, p-value = 0.00388
## alternative hypothesis: true location shift is not equal to 0

Strength/magnitude of the statistical claim (effect size) by scenarios:

## 
## Cliff's Delta
## 
## delta estimate: 0.3377778 (medium)
## 95 percent confidence interval:
##       lower       upper 
## -0.09307555  0.66208097

Statistical significance by conflicts, consider a confidence level of 0.95 (p-value = 0.05):

## 
##  Wilcoxon signed rank test
## 
## data:  aFP_ST$unstructured and aFP_SS$Semistructured
## V = 110, p-value = 0.002625
## alternative hypothesis: true location shift is not equal to 0

Strength/magnitude of the statistical claim (effect size) by conflicts:

## 
## Cliff's Delta
## 
## delta estimate: 0.6844444 (large)
## 95 percent confidence interval:
##     lower     upper 
## 0.2672163 0.8855660

When compared to semistructured merge, does unstructured merge compromise integration correctness by having more false negatives?

Semistructured Merge Additional False Negatives

By All Scenarios

On average 3.25% of the considered merge scenarios, had semistructured merge additional falses negatives, with a standard deviation of 3.49% . Considering aggregated numbers of all projects of our sample, this corresponds to 3% of the considered merge scenarios.

By Conflicts

On average 34.4% of the number of semistructured merge conflicts are semistructured merge additional falses negatives, with a standard deviation of 20.32% . Considering aggregated numbers of all projects of our sample, this corresponds to 30.32% of the number of semistructured merge conflicts.

Unstructured Merge Additional False Negatives

By All Scenarios

On average 0.32% of the considered merge scenarios, had unstructured merge additional falses negatives, with a standard deviation of 0.49% . Considering aggregated numbers of all projects of our sample, this corresponds to 0.36% of the considered merge scenarios.

By Conflicts

On average 1.24% of the number of unstructured merge conflicts are unstructured merge additional falses negatives, with a standard deviation of 1.74% . Considering aggregated numbers of all projects of our sample, this corresponds to 2.34% of the number of unstructured merge conflicts.

Statistical significance by scenarios, consider a confidence level of 0.95 (p-value = 0.05):

## Warning in wilcox.test.default(aFN_SS$Semistructured, aFN_UN
## $Unstructured, : cannot compute exact p-value with zeroes
## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  aFN_SS$Semistructured and aFN_UN$Unstructured
## V = 105, p-value = 0.001097
## alternative hypothesis: true location shift is not equal to 0

Strength/magnitude of the statistical claim (effect size) by scenarios:

## 
## Cliff's Delta
## 
## delta estimate: 0.72 (large)
## 95 percent confidence interval:
##     lower     upper 
## 0.3522688 0.8951494

Statistical significance by conflicts, consider a confidence level of 0.95 (p-value = 0.05):

## Warning in wilcox.test.default(aFN_SS$Semistructured, aFN_UN
## $Unstructured, : cannot compute exact p-value with ties
## Warning in wilcox.test.default(aFN_SS$Semistructured, aFN_UN
## $Unstructured, : cannot compute exact p-value with zeroes
## 
##  Wilcoxon signed rank test with continuity correction
## 
## data:  aFN_SS$Semistructured and aFN_UN$Unstructured
## V = 105, p-value = 0.001094
## alternative hypothesis: true location shift is not equal to 0

Strength/magnitude of the statistical claim (effect size) by conflicts:

## 
## Cliff's Delta
## 
## delta estimate: 0.8977778 (large)
## 95 percent confidence interval:
##     lower     upper 
## 0.5291413 0.9813313

Summary False Positives and False Negatives