MapReduce Frameworks Lab

Bottleneck MapReduce Framework

Matrix MapReduce Framework

combineAndFinishAll

method: Map<K, R> combineAndFinishAll(Map<K, A>[][] input) (parallel implementation required)

In this stage, you will take the matrix you just completed and combine all of the separate rows down to one array. Afterward, you will convert this combined array of maps into one final map. This method should run in parallel.

As mentioned previously, you should go directly down the matrix to access the same bucket across the different slices you created in the mapAndAccumulateAll step. For all of the maps in a column, you should go through each entry and combine it down into one row. You will need to make use of the Collector’s finisher again, but you will also need to make use of the combiner. You can access the Collector’s combiner using the combiner() method. Although the combine step differs from the bottlenecked framework, the finish step should mirror what you did previously.

Hint: You can use the provided MultiWrapMap class to return the final row as a valid output. You should also combine before you finish.

slide

Testing Your Solution

Correctness

There is a top-level test suite comprised of sub test suites which can be invoked separately when you want to focus on one part of the assignment.

class:	FrameworksLabTestSuite.java
package:	mapreduce.framework.lab
source folder:	testing/src/test/java

Bottlenecked

class:	BottleneckedFrameworkTestSuite.java
package:	mapreduce.framework.lab.bottlenecked
source folder:	testing/src/test/java

MapAll

class:	BottleneckedFrameworkTestSuite.java
package:	mapreduce.framework.lab.bottlenecked
source folder:	testing/src/test/java

AccumulateAll

class:	BottleneckedAccumulateAllTestSuite.java
package:	mapreduce.framework.lab.bottlenecked
source folder:	testing/src/test/java

FinishAll

class:	BottleneckedFinishAllTestSuite.java
package:	mapreduce.framework.lab.bottlenecked
source folder:	testing/src/test/java

Holistic

class:	BottleneckedHolisticTestSuite.java
package:	mapreduce.framework.lab.bottlenecked
source folder:	testing/src/test/java

Matrix

class:	MatrixFrameworkTestSuite.java
package:	mapreduce.framework.lab.matrix
source folder:	testing/src/test/java

MapAccumulateAll

class:	MatrixMapAccumulateAllTestSuite.java
package:	mapreduce.framework.lab.matrix
source folder:	testing/src/test/java

CombineFinishAll

class:	MatrixCombineFinishAllTestSuite.java
package:	mapreduce.framework.lab.matrix
source folder:	testing/src/test/java

Holistic

class:	MatrixHolisticTestSuite.java
package:	mapreduce.framework.lab.matrix
source folder:	testing/src/test/java

Rubric

As always, please make sure to cite your work appropriately.

Total points: 100

Bottlenecked framework subtotal: 40

• Correct mapAll (10)
• Correct accumulateAll (20)
• Correct finishAll (10)

Matrix framework subtotal: 60

• Correct mapAndAccumulateAll (30)
• Correct combineAndFinishAll (30)

-->