MergeSort

Motivation

Sorting is a problem that is well solved by divide and conquer algorithms. Merge sort is a elegant example which can be parallelized in a straight-forward manner.

Finally, parallelization of the combine step, while not trivial, is possible (and left as an optional fun exercise for those so inclined).

Background

In computer science, merge sort refers to a sorting algorithm which splits an array of data into continuously smaller halves until the arrays reach a size of 1, after which the elements are continuously compared and merged into a sorted array.

For more information on how this process works, visit the wikipedia page on merge sort.

Video: CS 50 Shorts: Merge Sort

Video: HackerRank: Merge Sort

Video: Merge Sort Combine Step

Visualization

If you are unclear on how merge sort works, take a look at the visualgo explanation and visualization of merge sort.

Work and Span

The divide step doesn't do any real significant work- it just calculates the midpoint. The combine step does all the work of merge sort. The top-level combine (when its two recursive calls have completed) must do N work. The next level does two ~N/2 combines. The level below that performs four ~N/4 combines. And so on. The table is $N$ wide and $\log N$ high. Thus the work of merge sort is $O(N\log N)$.

For N=16:

	<- - - - - - - - - - - - - - N - - - - - - - - - - - - - ->
	mid
	mid								mid
	mid				mid				mid				mid
	mid		mid		mid		mid		mid		mid		mid		mid
^	base	base	base	base	base	base	base	base	base	base	base	base	base	base	base	base
|	combine [0, 2)		combine [2, 4)		combine [4, 6)		combine [6, 8)		combine [8, 10)		combine [10, 12)		combine [12, 14)		combine [14, 16)
lg N	combine [0, 4)				combine [4, 8)				combine [8, 12)				combine [12, 16)
|	combine [0, 8)								combine [8, 16)
v	combine [0, 16)

Technically, the span also include log(N) midpoint calculations from the divide steps, this is insignificant next to N, so we focus on just the work/span of the combines.

The grand vast majority of the span for the classic sequential combine merge sort is the work of one combine for each of the levels (highlighted in blue). This works out to be the geometric series:

$N+{\frac {N}{2}}+{\frac {N}{4}}+{\frac {N}{8}}+{\frac {N}{16}}+\cdots +1$

which is:

$2N$

As a challenge problem, we parallelize the combine step, resulting in a span for merge sort of an outstanding $\lg ^{k}{}n$.

The Core Questions

• What are the tasks?
• What is the data?
• Is the data mutable?
• If so, how is it shared?

Mistakes To Avoid

Warning: Be sure to calculate the midpoint correctly.

Warning: When checking the base case, remember to account for [minInclusive, maxExclusive). It is all too easy to get an off by 1 error and stack overflow.

Warning: When transitioning to sequential, make sure to NOT sort the entire array when you are only responsible for a range [min,max).

Code to Investigate and Implement

You will need to implement merge sort sequentially and in parallel, but you will need to do both implementations recursively. The kernel method should call itself using recursion, but each public mergesort method should only call its kernel once to do the work.

Combine Step

Both the sequential and parallel merge sorts will be passed a Combiner. When you are done with the divide and conquer phases, invoke combiner.combineRange(a, b, comparator) to merge the two sorted sub-problem results. For the base case, combiner.createSingleItemArray(item) is provided.

interface Combiner<T>

public interface Combiner<T> {
	IntFunction<T[]> arrayGenerator();

	T[] combine(T[] a, T[] b, Comparator<T> comparator) throws InterruptedException, ExecutionException;

	default T[] createSingleItemArray(T item) {
		T[] result = arrayGenerator().apply(1);
		result[0] = item;
		return result;
	}
}

class SequentialCombiner<T>

class:	SequentialCombiner.java
methods:	constructor arrayGenerator combine
package:	sort.merge.exercise
source folder:	student/src/main/java

constructor

Maintain the passed arrayGenerator parameter in a (private final) instance variable for later use.

arrayGenerator

Return the arrayGenerator previously passed to the constructor.

combine

Implement the combine step.

Sort

interface Sorter<T>

public interface Sorter<T> {
	default void sort(T[] data, Comparator<T> comparator) throws InterruptedException, ExecutionException {

		throw new NotYetImplementedException();

	}

	void sortRange(T[] data, Comparator<T> comparator, int min, int maxExclusive)
			throws InterruptedException, ExecutionException;
}

class:	Sorter.java
methods:	sort
package:	sort.core
source folder:	student/src/main/java

The Sorter<T> interface provides two methods: sort and sortRange. We will leave sortRange to the classes which implement Sorter.

sort

You will provide a default implementation of the sort method, so that all implementing classes do not have to.

method: default void sort(T[] data, Comparator<T> comparator) (sequential implementation only)

You should simply invoke the sortRange method with the correct arguments. Consider whether you should use data.length-1 or data.length as your max, given that we've named the variable maxExclusive.

SequentialMergeSorter

class:	SequentialMergeSorter.java
methods:	sortRangeOutOfPlace
package:	sort.merge.exercise
source folder:	student/src/main/java

Implement the classic divide and conquer sequential algorithm here.

Warning: Be sure to leverage the provided private final Combiner<T> combiner; field to perform the combine step.

ParallelMergeSorter

class:	ParallelMergeSorter.java
methods:	sortRangeOutOfPlace
package:	sort.merge.exercise
source folder:	student/src/main/java

Adapt your sequential algorithm to take advantage of what can be parallelized.

What can be in parallel?

What is dependent on the parallel tasks completing?

Warning:Leverage the provided private final IntPredicate isParallelDesired; field to test whether or not parallelism is warranted for the current range length.

Warning:Fall back to the provided private final Sorter<T> lowerOverheadSorter; field when the isParallelDesired predicate indicates that parallelism is not longer warranted.

(Optional) Challenge Parallel Combiner

You can divide and conquer the combine step in merge sort. The work should remain ${\mathcal {O}}(n\lg {}n)$ while the critical path length can be cleanly improved from ${\mathcal {O}}(n)$ down to ${\mathcal {O}}(\lg ^{3}{}n)$.

For details on how to complete this challenge, check out: MergeSort_Parallel_Combiner

Testing Your Solution

class:	__MergeSorterTestSuite.java
package:	sort.merge.exercise
source folder:	testing/src/test/java

Client

class:	MergeSortClient.java	CLIENT
package:	sort.merge.client
source folder:	student/src/main/java

MergeSortClient

String[] data = createShuffledLetterNumberPairs(); System.out.println("unsorted: " + Arrays.toString(data)); Combiner<String> combiner = new SequentialCombiner<>(String.class); IntPredicate isParallelDesired = new GreaterThanThreshold(31); Sorter<String> sorter = new ParallelMergeSorter<>(combiner, isParallelDesired); sorter.sort(data, Comparator.naturalOrder()); System.out.println(" sorted: " + Arrays.toString(data));

Performance

class:	SortTiming.java
package:	sort.merge.performance
source folder:	src/main/java

Pledge, Acknowledgments, Citations

file:

studio-merge-sort-pledge-acknowledgments-citations.txt

More info about the Honor Pledge