Motivation

Sorting is a problem that is well solved by divide and conquer algorithms. Merge sort is a elegant example which can be parallelized in a straight-forward manner.

Finally, parallelization of the combine step, while not trivial, is possible (and left as an optional fun exercise for those so inclined).

Background

In computer science, merge sort refers to a sorting algorithm which splits an array of data into continuously smaller halves until the arrays reach a size of 1, after which the elements are continuously compared and merged into a sorted array.

For more information on how this process works, visit the wikipedia page on merge sort.

Video: CS 50 Shorts: Merge Sort

Video: HackerRank: Merge Sort

Video: Merge Sort Combine Step

Visualization

If you are unclear on how merge sort works, take a look at the visualgo explanation and visualization of merge sort.

Work and Span

The divide step doesn't do any real significant work- it just calculates the midpoint. The combine step does all the work of merge sort. The top-level combine (when its two recursive calls have completed) must do N work. The next level does two ~N/2 combines. The level below that performs four ~N/4 combines. And so on. The table is $N$ wide and $\log N$ high. Thus the work of merge sort is $O(N\log N)$ .

For N=16:

	<- - - - - - - - - - - - - - N - - - - - - - - - - - - - ->
	mid
	mid								mid
	mid				mid				mid				mid
	mid		mid		mid		mid		mid		mid		mid		mid
^	base	base	base	base	base	base	base	base	base	base	base	base	base	base	base	base
\|	combine [0, 2)		combine [2, 4)		combine [4, 6)		combine [6, 8)		combine [8, 10)		combine [10, 12)		combine [12, 14)		combine [14, 16)
lg N	combine [0, 4)				combine [4, 8)				combine [8, 12)				combine [12, 16)
\|	combine [0, 8)								combine [8, 16)
v	combine [0, 16)

Technically, the span also include log(N) midpoint calculations from the divide steps, this is insignificant next to N, so we focus on just the work/span of the combines.

The grand vast majority of the span for the classic sequential combine merge sort is the work of one combine for each of the levels (highlighted in blue). This works out to be the geometric series:

$N+{\frac {N}{2}}+{\frac {N}{4}}+{\frac {N}{8}}+{\frac {N}{16}}+\cdots +1$

which is:

$2N$

As a challenge problem, we parallelize the combine step, resulting in a span for merge sort of an outstanding $\lg ^{k}{}n$ .

The Core Questions

What are the tasks?
What is the data?
Is the data mutable?
If so, how is it shared?

Mistakes To Avoid

Warning: Be sure to calculate the midpoint correctly.

Warning: When checking the base case, remember to account for [minInclusive, maxExclusive). It is all too easy to get an off by 1 error and stack overflow.

Warning: When transitioning to sequential, make sure to NOT sort the entire array when you are only responsible for a range [min,max).

Code to Use

Both the sequential and parallel merge sorts will be passed a Combiner. When you are done with the divide and conquer phases, invoke combiner.combineRange(a, b, comparator) to merge the two sorted sub-problem results. For the base case, combiner.createSingleItemArray(item) is provided.

public interface Combiner<T> {
	IntFunction<T[]> generator();

	T[] combine(T[] a, T[] b, Comparator<T> comparator) throws InterruptedException, ExecutionException;

	default T[] createSingleItemArray(T item) {
		T[] result = generator().apply(1);
		result[0] = item;
		return result;
	}
}

Code to Implement

You will need to implement merge sort sequentially and in parallel, but you will need to do both implementations recursively. The kernel method should call itself using recursion, but each public mergesort method should only call its kernel once to do the work.

Combine Step

Combiner<T>

public interface Combiner<T> {
	IntFunction<T[]> arrayGenerator();

	T[] combine(T[] a, T[] b, Comparator<T> comparator) throws InterruptedException, ExecutionException;

	default T[] createSingleItemArray(T item) {
		T[] result = arrayGenerator().apply(1);
		result[0] = item;
		return result;
	}
}

SequentialCombiner<T>

constructor

arrayGenerator

combine

Sorter

class:	Sorter.java
methods:	sort
package:	sort.core
source folder:	student/src/main/java

The Sorter<T> interface provides two methods: sort and sortRange. We will leave sortRange to the classes which implement Sorter.

sort

You will provide a default implementation of the sort method, so that all implementing classes do not have to.

method: default void sort(T[] data, Comparator<T> comparator) (sequential implementation only)

You should simply invoke the sortRange method with the correct arguments. Consider whether you should use data.length-1 or data.length as your max, given that we've named the variable maxExclusive.

SequentialCombiner

Merge sort's combine step leverages the fact that it can rely on the two sub-ranges are provided in sorted order. Use this fact to fill the provided temporary buffer with the sorted contents of the entire range [min, maxExclusive).

Performance note: in an effort to make this part of the assignment more appropriately approachable, a temporary buffer will be created, combined into, and copied back from. Merge sort can be built such that depths of the recursion alternate which buffer is read from and written to. The added complexity this performance improvement would incur is not deemed valuable from a pedagogical stand point, but it is worth noting.

(Provided) combineRange

The public method combineRange(data, comparator, min, mid, maxExclusive) is provided. It creates a temporary buffer and passes it to combineRangeIntoBuffer(buffer, data, comparator, min, mid, maxExclusive) which you will implement to fill the temporary buffer with the combined sorted contents. The contents of the temporary buffer is then copied back to the data array with System.arraycopy().

combineRange(T[] data, Comparator<T> comparator, int min, int mid, int maxExclusive)

@Override
public void combineRange(T[] data, Comparator<T> comparator, int min, int mid, int maxExclusive) {
	int rangeLength = maxExclusive - min;
	T[] buffer = ArrayUtils.createArray(componentType, rangeLength);
	combineRangeIntoBuffer(buffer, data, comparator, min, mid, maxExclusive);
	System.arraycopy(buffer, 0, data, min, maxExclusive - min);
}

(Exercise) combineRangeIntoBuffer

class:	SequentialCombiner.java
methods:	combineRangeIntoBuffer
package:	sort.merge.exercise
source folder:	student/src/main/java

method: private static <T> void combineRangeIntoBuffer(T[] buffer, T[] data, Comparator<T> comparator, int min, int mid, int maxExclusive) (sequential implementation only)

Here you will implement the classic sequential merge combine outlined in the Background section. You will read from data between min and maxExclusive and write to buffer starting at 0.

SequentialMergeSorter

class:	SequentialMergeSorter.java
methods:	sortRangeOutOfPlace
package:	sort.merge.exercise
source folder:	student/src/main/java

Implement the classic divide and conquer sequential algorithm here.

Warning: Be sure to leverage the provided private final Combiner<T> combiner; field to perform the combine step.

ParallelMergeSorter

class:	ParallelMergeSorter.java
methods:	sortRangeOutOfPlace
package:	sort.merge.exercise
source folder:	student/src/main/java

Adapt your sequential algorithm to take advantage of what can be parallelized.

What can be in parallel?

What is dependent on the parallel tasks completing?

Warning:Leverage the provided private final IntPredicate isParallelDesired; field to test whether or not parallelism is warranted for the current range length.

Warning:Fall back to the provided private final Sorter<T> sequentialSorter; field when the isParallelDesired predicate indicates that parallelism is not longer warranted.

(Optional) Challenge Parallel Combiner

You can divide and conquer the combine step in merge sort. The work should remain ${\mathcal {O}}(n\lg {}n)$ while the critical path length can be cleanly improved from ${\mathcal {O}}(n)$ down to ${\mathcal {O}}(\lg ^{3}{}n)$ .

For details on how to complete this challenge, check out: MergeSort_Parallel_Combiner

Testing Your Solution

class:	__MergeSorterTestSuite.java
package:	sort.merge.exercise
source folder:	testing/src/test/java

Client

class:	MergeSortClient.java	CLIENT
package:	sort.merge.client
source folder:	student/src/main/java

MergeSortClient

String[] data = createShuffledLetterNumberPairs();
System.out.println("unsorted: " + Arrays.toString(data));
Combiner<String> combiner = new SequentialCombiner<>(String.class);
IntPredicate isParallelDesired = new GreaterThanThreshold(31);
Sorter<String> sorter = new ParallelMergeSorter<>(combiner, isParallelDesired);
sorter.sort(data, Comparator.naturalOrder());
System.out.println("  sorted: " + Arrays.toString(data));

Performance

class:	SortTiming.java
package:	sort.merge.performance
source folder:	src/main/java

Pledge, Acknowledgments, Citations

file:

studio-merge-sort-pledge-acknowledgments-citations.txt

More info about the Honor Pledge

MergeSort

Contents