Difference between revisions of "MapReduce Reducer Assignment"

From CSE231 Wiki
Jump to navigation Jump to search
Line 3: Line 3:
  
 
=Background=
 
=Background=
[https://docs.oracle.com/javase/8/docs/api/java/util/stream/Collector.html interface Collector<T,A,R>]
+
After much debate, we decided to go with [https://docs.oracle.com/javase/8/docs/api/java/util/stream/Collector.html interface Collector<T,A,R>] from the standard Java streams framework as the basis for the MapReduce assignment. 
 +
 
 +
It might have been slightly less confusing at the outset if we provided our own custom interface.  Imaging the mythical non-existant interface below:
 +
 
 +
<nowiki>public interface Reducer<V, A, R> {
 +
A createMutableContainer();
 +
void accumulate(A container, V item);
 +
A combine(A containerA, A containerB);
 +
R reduce(A container);
 +
}</nowiki>
 +
 
 +
 
  
 
[https://commons.apache.org/proper/commons-lang/apidocs/org/apache/commons/lang3/mutable/MutableInt.html class MutableInt]
 
[https://commons.apache.org/proper/commons-lang/apidocs/org/apache/commons/lang3/mutable/MutableInt.html class MutableInt]

Revision as of 17:50, 22 February 2018

Motivation

Collector is fundamental to the MapReduce Framework lab.

Background

After much debate, we decided to go with interface Collector<T,A,R> from the standard Java streams framework as the basis for the MapReduce assignment.

It might have been slightly less confusing at the outset if we provided our own custom interface. Imaging the mythical non-existant interface below:

public interface Reducer<V, A, R> {
	A createMutableContainer();
	void accumulate(A container, V item);
	A combine(A containerA, A containerB);
	R reduce(A container);
}


class MutableInt

Mythical Reducer:

public class CollectorReducerAdapter<V, A, R> implements Reducer<V, A, R> {
	private final Collector<V,A,R> collector;
	public CollectorReducerAdapter(Collector<V,A,R> collector) {
		this.collector = collector;
	}
	@Override
	public A createMutableContainer() {
		return collector.supplier().get();
	}
	@Override
	public void accumulate(A container, V item) {
		collector.accumulator().accept(container, item);
	}
	@Override
	public A combine(A containerA, A containerB) {
		return collector.combiner().apply(containerA, containerB);
	}
	@Override
	public R reduce(A container) {
		return collector.finisher().apply(container);
	}
}

Code To Implement

ClassicReducer

class: ClassicReducer.java Java.png
methods: supplier
accumulator
combiner
package: mapreduce.collector.studio
source folder: student/src/main/java

IntSumCollector

class: IntSumCollector.java Java.png
methods: supplier
accumulator
combiner
finisher
package: mapreduce.collector.intsum.studio
source folder: student/src/main/java

Testing Your Solution

Correctness

class: CollectorStudioTestSuite.java Junit.png
package: mapreduce
source folder: testing/src/test/java